# TensorFlow.js


## 1. What is it?

> TensorFlow.js is a library for machine learning in JavaScript

> Develop ML models in JavaScript, and use ML directly in the browser or in Node.js.

## 2. How it works?

- **Run existing models**
Use off-the-shelf JavaScript models or convert Python TensorFlow models to run in the browser or under Node.js.

- **Retrain existing models**
Retrain pre-existing ML models using your own data.

- **Develop ML with JavaScript**
Build and train models directly in JavaScript using flexible and intuitive APIs.

## 3. TensorFlow.js API

- High level Layers API (like Keras)
- Low level Ops API (Mathmetical)


Notice that backend (CPU, WebGL, WASM) doen't mean server side here, it means the hardware where tensorflow will be executed.

## 4. Performance



## 5. Run existing models

- [3.1: What are pre-trained models?](https://www.youtube.com/watch?v=iTlj3gMYzw8&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=9&ab_channel=GoogleforDevelopers)

### Pre-trained Model

- [tfjs-model](https://github.com/tensorflow/tfjs-models)

#### Example 1: Question and Answer Model

NPM Example

import '@tensorflow/tfjs-core'
import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as qna from '@tensorflow-models/qna'

const searchText = 'We believe cats are the real stars of YouTube'
const question = 'What are the real stars of YouTube?'

qna.load().then(model => {
    model.findAnswers(question, searchText).then(answers => {
        console.log('Answers: ', answers) // Cats!
    })
})

- [Demo](https://storage.googleapis.com/tfjs-models/demos/mobilebert-qna/index.html)
- [Model Document](https://github.com/tensorflow/tfjs-models/tree/master/qna)

#### Example 2: Object Detection (coco-ssd)


NPM Example

import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as cocoSsd from '@tensorflow-models/coco-ssd'

const img = document.getElementById('img')

// Load the model.
cocoSsd.load().then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
        console.log('Predictions: ', predictions)
    })
})

The predictions format be like

[{
  bbox: [x, y, width, height],
  class: 'person',
  score: 0.8380282521247864
}, {
  bbox: [x, y, width, height],
  class: 'kite',
  score: 0.74644153267145157
}]

- [Demo](https://tensorflow-js-object-detection.glitch.me/)
- [Model Document](https://github.com/tensorflow/tfjs-models/tree/master/coco-ssd)

### Advance Pre-tranined Model

https://www.kaggle.com/models?tfhub-redirect=true

#### Structure

- model.json - bunch of metadata about model type, its architecture, and configuration details.
- shardXofN.bin - all of trained parameters that the model has learned in order to perform a certain task.



#### Example: MoveNet Singlepose Lightning


https://www.kaggle.com/models/google/movenet/tfJs/singlepose-lightning

#### Offline model

Save to local storage for offline access!

await model.save('localstorage://demo/newModelName')

Check for saved models in local storage:

console.log(JSON.stringify(await tf.io.listModels()))

Load just like before but using localstorage url:

await tf.loadLayersModel('localstorage://demo/newModelName')

####

## 6. Retrain existing models

### Real project

[Teachable Machine](https://teachablemachine.withgoogle.com/)

### Transfer learning

> Transfer learning (TL) is a technique in machine learning (ML) in which knowledge learned from a task is re-used in order to boost performance on a related task.


Advantages

1. Faster to train
2. Reuses knowledge learnt from the base model
3. Less example data required to classify new examples
4. Very well suited for web browser environment

Check tutorial below to understand how teachable machine does trainsfer learning.
- [Recognize custom objects with TensorFlow.js.](https://www.youtube.com/watch?v=VHlCxLlTZBg&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=35&ab_channel=GoogleforDevelopers)
- [Using layers models for transfer learning](https://www.youtube.com/watch?v=PN4asCDITNg&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=36&ab_channel=GoogleforDevelopers)

## 7. Tensors

### Dimension (Rank)

const t0 = tf.scalar(6)

const t1 = tf.tensor1d([1, 2, 3])

const t2 = tf.tensor2d([
    [5, 0, 7],
    [8, 0, 3],
    [0, 2, 9]
])

const t3 = tf.tensor3d([
    [
        [1, 2, 3], [4, 5, 6], [7, 8, 9]
    ],
    [
        [0, 0, 0], [0, 0, 0], [0, 0, 0]
    ],
    [
        [2, 4, 6], [8, 6, 4], [2, 4, 6]
    ]
])

// ...

<table>
<tbody>
<tr>
<td>
<img src="https://hackmd.io/_uploads/HkCt7XDfR.png" alt="截圖 2024-05-07 上午11.44.00" />
</td>
<td>
<img src="https://hackmd.io/_uploads/SkW37QDM0.png" alt="截圖 2024-05-07 上午11.44.33" />
</td>
<td>
<img src="https://hackmd.io/_uploads/S1Oe4mvGR.png" alt="截圖 2024-05-07 上午11.45.41" />
</td>
</tr>
<tr>
<td>
<img src="https://hackmd.io/_uploads/BJVaHQwGR.png" alt="截圖 2024-05-07 上午11.53.09" />
</td>
<td>
<img src="https://hackmd.io/_uploads/SJOWImvfA.png" alt="截圖 2024-05-07 上午11.54.15" />
</td>
<td>
<img src="https://hackmd.io/_uploads/B1TGUXPzA.png" alt="截圖 2024-05-07 上午11.54.31" />
</td>
</tr>
</tbody>
</table>

### Data Type (DType)

### Shape


### Size


### Memory Management

- [tf.tidy](https://js.tensorflow.org/api/latest/?_gl=1*17jqq0u*_ga*MTU0MzEyMTQyNC4xNzEzNzYxMjkz*_ga_W0YLR4190T*MTcxNTA1NDQ4My44LjEuMTcxNTA1NDUwOS4wLjAuMA..#tidy)
- [tf.dispose](https://js.tensorflow.org/api/latest/?_gl=1*17jqq0u*_ga*MTU0MzEyMTQyNC4xNzEzNzYxMjkz*_ga_W0YLR4190T*MTcxNTA1NDQ4My44LjEuMTcxNTA1NDUwOS4wLjAuMA..#dispose)

Check [api document](https://js.tensorflow.org/api/3.12.0/#Tensors) for tensor usage.

### Example: Normalization


function normalize(inputs) {
  return tf.tidy(function () {
    const maxs = tf.max(inputs, 0)
    const mins = tf.min(inputs, 0)
    return tf.div(tf.sub(inputs, mins), tf.sub(maxs, mins))
  })
}

## 8. Develop ML with JavaScript


### Example: Single Neuron Linear Regression Model

ax + b = y



Check the simple demo [repo](https://github.com/Lilybon/tfjs-neuron-linear-regression) here.

## 9. Refference

- **Course**
- https://www.youtube.com/playlist?list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui
- **TensorFlow.js**
- https://www.tensorflow.org/js
- https://www.tensorflow.org/js/guide/tensors_operations
- **Noun**
- [Epoch, Batch size, Iteration, Learning Rate](https://medium.com/%E4%BA%BA%E5%B7%A5%E6%99%BA%E6%85%A7-%E5%80%92%E5%BA%95%E6%9C%89%E5%A4%9A%E6%99%BA%E6%85%A7/epoch-batch-size-iteration-learning-rate-b62bf6334c49)
- [Transfer learning](https://en.wikipedia.org/wiki/Transfer_learning)
- [CUDA](https://zh.wikipedia.org/zh-tw/CUDA)
- [AVX](https://zh.wikipedia.org/zh-tw/AVX%E6%8C%87%E4%BB%A4%E9%9B%86)
- [Keras](https://zh.wikipedia.org/zh-tw/Keras)
- **Statistic & Mathmetics**
- [Matrix](https://zh.wikipedia.org/zh-tw/%E7%9F%A9%E9%98%B5)
- [Mean Square Error (MSE)](https://zh.wikipedia.org/zh-tw/%E5%9D%87%E6%96%B9%E8%AF%AF%E5%B7%AE)
- [Linear Regression](https://zh.wikipedia.org/zh-tw/%E7%B7%9A%E6%80%A7%E5%9B%9E%E6%AD%B8)
- [Descision Boundary](https://zh.wikipedia.org/zh-tw/%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C)
- [Logistic Regression](https://zh.wikipedia.org/zh-tw/%E9%82%8F%E8%BC%AF%E6%96%AF%E8%AB%A6%E8%BF%B4%E6%AD%B8)
- [Cross Entropy](https://zh.wikipedia.org/zh-tw/%E4%BA%A4%E5%8F%89%E7%86%B5)
- [Marginal Distribution](https://zh.wikipedia.org/zh-tw/%E8%BE%B9%E7%BC%98%E5%88%86%E5%B8%83)
- [Sigmoid Function](https://zh.wikipedia.org/zh-tw/S%E5%9E%8B%E5%87%BD%E6%95%B0)
- [Conditional Distribution](https://zh.wikipedia.org/zh-tw/%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83)
- [Softmax Function](https://zh.wikipedia.org/zh-tw/Softmax%E5%87%BD%E6%95%B0)
- **Algorithm**
- [K Nearest Neighbor (KNN)](https://zh.wikipedia.org/zh-tw/K-%E8%BF%91%E9%82%BB%E7%AE%97%E6%B3%95)
- [Gradient Descent (GD)](https://zh.wikipedia.org/zh-tw/%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95#:~:text=%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%EF%BC%88%E8%8B%B1%E8%AA%9E%EF%BC%9AGradient,%E9%BB%9E%E9%80%B2%E8%A1%8C%E7%96%8A%E4%BB%A3%E6%90%9C%E7%B4%A2%E3%80%82)
- Stochastic Gradient Descent (SGD)