TensorFlow.js

1. What is it?

TensorFlow.js is a library for machine learning in JavaScript

Develop ML models in JavaScript, and use ML directly in the browser or in Node.js.

2. How it works?

• Run existing models
  Use off-the-shelf JavaScript models or convert Python TensorFlow models to run in the browser or under Node.js.

• Retrain existing models
  Retrain pre-existing ML models using your own data.

• Develop ML with JavaScript
  Build and train models directly in JavaScript using flexible and intuitive APIs.

3. TensorFlow.js API

• High level Layers API (like Keras)
• Low level Ops API (Mathmetical)

Notice that backend (CPU, WebGL, WASM) doen't mean server side here, it means the hardware where tensorflow will be executed.

4. Performance

5. Run existing models

• 3.1: What are pre-trained models?

Pre-trained Model

• tfjs-model

Example 1: Question and Answer Model

NPM Example

import '@tensorflow/tfjs-core'
import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as qna from '@tensorflow-models/qna'

const searchText = 'We believe cats are the real stars of YouTube'
const question = 'What are the real stars of YouTube?'

qna.load().then(model => {
    model.findAnswers(question, searchText).then(answers => {
        console.log('Answers: ', answers) // Cats!
    })
})

• Demo
• Model Document

Example 2: Object Detection (coco-ssd)

NPM Example

import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as cocoSsd from '@tensorflow-models/coco-ssd'

const img = document.getElementById('img')

// Load the model.
cocoSsd.load().then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
        console.log('Predictions: ', predictions)
    })
})

The predictions format be like

[{
  bbox: [x, y, width, height],
  class: 'person',
  score: 0.8380282521247864
}, {
  bbox: [x, y, width, height],
  class: 'kite',
  score: 0.74644153267145157
}]

• Demo
• Model Document

Advance Pre-tranined Model

https://www.kaggle.com/models?tfhub-redirect=true

Structure

• model.json - bunch of metadata about model type, its architecture, and configuration details.
• shardXofN.bin - all of trained parameters that the model has learned in order to perform a certain task.

Example: MoveNet Singlepose Lightning

https://www.kaggle.com/models/google/movenet/tfJs/singlepose-lightning

Offline model

Save to local storage for offline access!

await model.save('localstorage://demo/newModelName')

Check for saved models in local storage:

console.log(JSON.stringify(await tf.io.listModels()))

Load just like before but using localstorage url:

await tf.loadLayersModel('localstorage://demo/newModelName')

6. Retrain existing models

Real project

Teachable Machine

Transfer learning

Transfer learning (TL) is a technique in machine learning (ML) in which knowledge learned from a task is re-used in order to boost performance on a related task.

Advantages

1. Faster to train
2. Reuses knowledge learnt from the base model
3. Less example data required to classify new examples
4. Very well suited for web browser environment

Check tutorial below to understand how teachable machine does trainsfer learning.

• Recognize custom objects with TensorFlow.js.
• Using layers models for transfer learning

7. Tensors

Dimension (Rank)

const t0 = tf.scalar(6)

const t1 = tf.tensor1d([1, 2, 3])

const t2 = tf.tensor2d([
    [5, 0, 7],
    [8, 0, 3],
    [0, 2, 9]
])

const t3 = tf.tensor3d([
    [
        [1, 2, 3], [4, 5, 6], [7, 8, 9]
    ],
    [
        [0, 0, 0], [0, 0, 0], [0, 0, 0]
    ],
    [
        [2, 4, 6], [8, 6, 4], [2, 4, 6]
    ]
])

// ...

Data Type (DType)

Shape

Size

Memory Management

• tf.tidy
• tf.dispose

Check api document for tensor usage.

Example: Normalization

function normalize(inputs) {
  return tf.tidy(function () {
    const maxs = tf.max(inputs, 0)
    const mins = tf.min(inputs, 0)
    return tf.div(tf.sub(inputs, mins), tf.sub(maxs, mins))
  })
}

8. Develop ML with JavaScript

Example: Single Neuron Linear Regression Model

ax + b = y

Check the simple demo repo here.

9. Refference

• Course
  • https://www.youtube.com/playlist?list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui
• TensorFlow.js
  • https://www.tensorflow.org/js
  • https://www.tensorflow.org/js/guide/tensors_operations
• Noun
  • Epoch, Batch size, Iteration, Learning Rate
  • Transfer learning
  • CUDA
  • AVX
  • Keras
• Statistic & Mathmetics
  • Matrix
  • Mean Square Error (MSE)
  • Linear Regression
  • Descision Boundary
  • Logistic Regression
  • Cross Entropy
  • Marginal Distribution
  • Sigmoid Function
  • Conditional Distribution
  • Softmax Function
• Algorithm
  • K Nearest Neighbor (KNN)
  • Gradient Descent (GD)
    • Stochastic Gradient Descent (SGD)