# Building Machine Learning Pipeline(1)
<!-- Put the link to this slide here so people can follow -->
slide: https://hackmd.io/@31vLb4j3SY6ZER7aLvwPXg/B1SwShYWB#/
---
## Speaker - Me
- Data scientist
- Gola to become a Machine Learning Architect :heart:
---
### Our Loyal friend: Pandas :dog:
- Essentially Plays the python ETL role
- Effective in visualization and dashboard
- After ETL and data cleaning one can easily train the ML model( *dataframe.values* )
---
### However difficult in productization
- Operates on numpy matrices, not DataFrames
- Does not preserve feature names
- Omits column data types
- Ignores almost anything useful provided by pandas.
---
{%youtube E8Nj7RwXf0s %}
---
### Usage flow
---
```graphviz
digraph {
compound=true
rankdir=RL
graph [ fontname="Source Sans Pro", fontsize=20 ];
node [ fontname="Source Sans Pro", fontsize=18];
edge [ fontname="Source Sans Pro", fontsize=12 ];
subgraph core {
c [label="Hackmd-it \ncore"] [shape=box]
}
c -> sync [ltail=session lhead=session]
subgraph cluster1 {
concentrate=true
a [label="Text source\nGithub, Gitlab, ..."] [shape=box]
b [label="HackMD Editor"] [shape=box]
sync [label="sync" shape=plaintext ]
b -> sync [dir="both"]
sync -> a [dir="both"]
label="An edit session"
}
}
```
---
### Architecture of extension
---
---
## Content script
- Bind with each page
- Manipulate DOM
- Add event listeners
- Isolated JavaScript environment
- It doesn't break things
---
# :fork_and_knife:
---
<style>
code.blue {
color: #337AB7 !important;
}
code.orange {
color: #F7A004 !important;
}
</style>
- <code class="orange">onMessage('event')</code>: Register event listener
- <code class="blue">sendMessage('event')</code>: Trigger event
---
# :bulb:
---
- Dead simple API
- Only cares about application logic
---
```typescript
import * as Channeru from 'channeru'
// setup channel in different page environment, once
const channel = Channeru.create()
```
---
```typescript
// in background script
const fakeLogin = async () => true
channel.answer('isLogin', async () => {
return await fakeLogin()
})
```
<br>
```typescript
// in inject script
const isLogin = await channel.callBackground('isLogin')
console.log(isLogin) //-> true
```
---
# :100: :muscle: :tada:
---
### Wrap up
- Cross envornment commnication
- A small library to solve messaging pain
- TypeScript Rocks :tada:
---
### Thank you! :sheep:
You can find me on
- GitHub
- Twitter
- or email me
{"metaMigratedAt":"2023-06-14T22:50:52.706Z","metaMigratedFrom":"YAML","title":"Building Machine Learning Pipeline(1)","breaks":true,"description":"In this talk I will show how to build a resuable, automated ML pipeline with Scikit-learn and Pandas.","contributors":"[{\"id\":\"df5bcb6f-88f7-498e-9911-1eda2efc0f5e\",\"add\":3099,\"del\":318}]"}