# TensorFlow.js
## 1. What is it?
> TensorFlow.js is a library for machine learning in JavaScript
> Develop ML models in JavaScript, and use ML directly in the browser or in Node.js.
## 2. How it works?
- **Run existing models**
Use off-the-shelf JavaScript models or convert Python TensorFlow models to run in the browser or under Node.js.
- **Retrain existing models**
Retrain pre-existing ML models using your own data.
- **Develop ML with JavaScript**
Build and train models directly in JavaScript using flexible and intuitive APIs.
## 3. TensorFlow.js API
- High level Layers API (like Keras)
- Low level Ops API (Mathmetical)
Notice that backend (CPU, WebGL, WASM) doen't mean server side here, it means the hardware where tensorflow will be executed.
## 4. Performance
## 5. Run existing models
- [3.1: What are pre-trained models?](https://www.youtube.com/watch?v=iTlj3gMYzw8&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=9&ab_channel=GoogleforDevelopers)
### Pre-trained Model
- [tfjs-model](https://github.com/tensorflow/tfjs-models)
#### Example 1: Question and Answer Model
NPM Example
```javascript
import '@tensorflow/tfjs-core'
import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as qna from '@tensorflow-models/qna'
const searchText = 'We believe cats are the real stars of YouTube'
const question = 'What are the real stars of YouTube?'
qna.load().then(model => {
model.findAnswers(question, searchText).then(answers => {
console.log('Answers: ', answers) // Cats!
})
})
```
- [Demo](https://storage.googleapis.com/tfjs-models/demos/mobilebert-qna/index.html)
- [Model Document](https://github.com/tensorflow/tfjs-models/tree/master/qna)
#### Example 2: Object Detection (coco-ssd)
NPM Example
```javascript
import '@tensorflow/tfjs-backend-cpu'
import '@tensorflow/tfjs-backend-webgl'
import * as cocoSsd from '@tensorflow-models/coco-ssd'
const img = document.getElementById('img')
// Load the model.
cocoSsd.load().then(model => {
// detect objects in the image.
model.detect(img).then(predictions => {
console.log('Predictions: ', predictions)
})
})
```
The predictions format be like
```javascript
[{
bbox: [x, y, width, height],
class: 'person',
score: 0.8380282521247864
}, {
bbox: [x, y, width, height],
class: 'kite',
score: 0.74644153267145157
}]
```
- [Demo](https://tensorflow-js-object-detection.glitch.me/)
- [Model Document](https://github.com/tensorflow/tfjs-models/tree/master/coco-ssd)
### Advance Pre-tranined Model
https://www.kaggle.com/models?tfhub-redirect=true
#### Structure
- model.json - bunch of metadata about model type, its architecture, and configuration details.
- shardXofN.bin - all of trained parameters that the model has learned in order to perform a certain task.
#### Example: MoveNet Singlepose Lightning
https://www.kaggle.com/models/google/movenet/tfJs/singlepose-lightning
#### Offline model
Save to local storage for offline access!
```javascript
await model.save('localstorage://demo/newModelName')
```
Check for saved models in local storage:
```javascript
console.log(JSON.stringify(await tf.io.listModels()))
```
Load just like before but using localstorage url:
```javascript
await tf.loadLayersModel('localstorage://demo/newModelName')
```
####
## 6. Retrain existing models
### Real project
[Teachable Machine](https://teachablemachine.withgoogle.com/)
### Transfer learning
> Transfer learning (TL) is a technique in machine learning (ML) in which knowledge learned from a task is re-used in order to boost performance on a related task.
Advantages
1. Faster to train
2. Reuses knowledge learnt from the base model
3. Less example data required to classify new examples
4. Very well suited for web browser environment
Check tutorial below to understand how teachable machine does trainsfer learning.
- [Recognize custom objects with TensorFlow.js.](https://www.youtube.com/watch?v=VHlCxLlTZBg&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=35&ab_channel=GoogleforDevelopers)
- [Using layers models for transfer learning](https://www.youtube.com/watch?v=PN4asCDITNg&list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui&index=36&ab_channel=GoogleforDevelopers)
## 7. Tensors
### Dimension (Rank)
```javascript
const t0 = tf.scalar(6)
const t1 = tf.tensor1d([1, 2, 3])
const t2 = tf.tensor2d([
[5, 0, 7],
[8, 0, 3],
[0, 2, 9]
])
const t3 = tf.tensor3d([
[
[1, 2, 3], [4, 5, 6], [7, 8, 9]
],
[
[0, 0, 0], [0, 0, 0], [0, 0, 0]
],
[
[2, 4, 6], [8, 6, 4], [2, 4, 6]
]
])
// ...
```
<table>
<tbody>
<tr>
<td>
<img src="https://hackmd.io/_uploads/HkCt7XDfR.png" alt="截圖 2024-05-07 上午11.44.00" />
</td>
<td>
<img src="https://hackmd.io/_uploads/SkW37QDM0.png" alt="截圖 2024-05-07 上午11.44.33" />
</td>
<td>
<img src="https://hackmd.io/_uploads/S1Oe4mvGR.png" alt="截圖 2024-05-07 上午11.45.41" />
</td>
</tr>
<tr>
<td>
<img src="https://hackmd.io/_uploads/BJVaHQwGR.png" alt="截圖 2024-05-07 上午11.53.09" />
</td>
<td>
<img src="https://hackmd.io/_uploads/SJOWImvfA.png" alt="截圖 2024-05-07 上午11.54.15" />
</td>
<td>
<img src="https://hackmd.io/_uploads/B1TGUXPzA.png" alt="截圖 2024-05-07 上午11.54.31" />
</td>
</tr>
</tbody>
</table>
### Data Type (DType)
### Shape
### Size
### Memory Management
- [tf.tidy](https://js.tensorflow.org/api/latest/?_gl=1*17jqq0u*_ga*MTU0MzEyMTQyNC4xNzEzNzYxMjkz*_ga_W0YLR4190T*MTcxNTA1NDQ4My44LjEuMTcxNTA1NDUwOS4wLjAuMA..#tidy)
- [tf.dispose](https://js.tensorflow.org/api/latest/?_gl=1*17jqq0u*_ga*MTU0MzEyMTQyNC4xNzEzNzYxMjkz*_ga_W0YLR4190T*MTcxNTA1NDQ4My44LjEuMTcxNTA1NDUwOS4wLjAuMA..#dispose)
Check [api document](https://js.tensorflow.org/api/3.12.0/#Tensors) for tensor usage.
### Example: Normalization
```javascript
function normalize(inputs) {
return tf.tidy(function () {
const maxs = tf.max(inputs, 0)
const mins = tf.min(inputs, 0)
return tf.div(tf.sub(inputs, mins), tf.sub(maxs, mins))
})
}
```
## 8. Develop ML with JavaScript
### Example: Single Neuron Linear Regression Model
```
ax + b = y
```
Check the simple demo [repo](https://github.com/Lilybon/tfjs-neuron-linear-regression) here.
## 9. Refference
- **Course**
- https://www.youtube.com/playlist?list=PLOU2XLYxmsILr3HQpqjLAUkIPa5EaZiui
- **TensorFlow.js**
- https://www.tensorflow.org/js
- https://www.tensorflow.org/js/guide/tensors_operations
- **Noun**
- [Epoch, Batch size, Iteration, Learning Rate](https://medium.com/%E4%BA%BA%E5%B7%A5%E6%99%BA%E6%85%A7-%E5%80%92%E5%BA%95%E6%9C%89%E5%A4%9A%E6%99%BA%E6%85%A7/epoch-batch-size-iteration-learning-rate-b62bf6334c49)
- [Transfer learning](https://en.wikipedia.org/wiki/Transfer_learning)
- [CUDA](https://zh.wikipedia.org/zh-tw/CUDA)
- [AVX](https://zh.wikipedia.org/zh-tw/AVX%E6%8C%87%E4%BB%A4%E9%9B%86)
- [Keras](https://zh.wikipedia.org/zh-tw/Keras)
- **Statistic & Mathmetics**
- [Matrix](https://zh.wikipedia.org/zh-tw/%E7%9F%A9%E9%98%B5)
- [Mean Square Error (MSE)](https://zh.wikipedia.org/zh-tw/%E5%9D%87%E6%96%B9%E8%AF%AF%E5%B7%AE)
- [Linear Regression](https://zh.wikipedia.org/zh-tw/%E7%B7%9A%E6%80%A7%E5%9B%9E%E6%AD%B8)
- [Descision Boundary](https://zh.wikipedia.org/zh-tw/%E5%86%B3%E7%AD%96%E8%BE%B9%E7%95%8C)
- [Logistic Regression](https://zh.wikipedia.org/zh-tw/%E9%82%8F%E8%BC%AF%E6%96%AF%E8%AB%A6%E8%BF%B4%E6%AD%B8)
- [Cross Entropy](https://zh.wikipedia.org/zh-tw/%E4%BA%A4%E5%8F%89%E7%86%B5)
- [Marginal Distribution](https://zh.wikipedia.org/zh-tw/%E8%BE%B9%E7%BC%98%E5%88%86%E5%B8%83)
- [Sigmoid Function](https://zh.wikipedia.org/zh-tw/S%E5%9E%8B%E5%87%BD%E6%95%B0)
- [Conditional Distribution](https://zh.wikipedia.org/zh-tw/%E6%9D%A1%E4%BB%B6%E6%A6%82%E7%8E%87%E5%88%86%E5%B8%83)
- [Softmax Function](https://zh.wikipedia.org/zh-tw/Softmax%E5%87%BD%E6%95%B0)
- **Algorithm**
- [K Nearest Neighbor (KNN)](https://zh.wikipedia.org/zh-tw/K-%E8%BF%91%E9%82%BB%E7%AE%97%E6%B3%95)
- [Gradient Descent (GD)](https://zh.wikipedia.org/zh-tw/%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95#:~:text=%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%B3%95%EF%BC%88%E8%8B%B1%E8%AA%9E%EF%BC%9AGradient,%E9%BB%9E%E9%80%B2%E8%A1%8C%E7%96%8A%E4%BB%A3%E6%90%9C%E7%B4%A2%E3%80%82)
- Stochastic Gradient Descent (SGD)
Sign in with Wallet
Connect another wallet