[TOC]

## [Agenda](https://gdg.community.dev/events/details/google-gdg-taipei-presents-devfest-taipei-2023/)

|Time|Room|Title|
|--|--|--|
|9:00 AM|Room 101|Opening - EvA ChU、Eric ShangKuan|
|9:15 AM|Room 101|Keynote - 展望 Generative AI 的現在與未來 - 薛良斌、KJ Wu (吳貴融)、林雅芳 (Tina Lin)|
|10:50 AM|Room 101|如何結合Google Colab及Intel OpenVINO來玩轉AIGC - 許哲豪|
|10:50 AM|Room 307|Thread-blocking I/O is blocking your future (JVM)|
|10:50 AM|Room 203|因為怕改壞，所以把 TDD 技能點滿了 - Hsuanhao|
|10:50 AM|Room 306|OpenTelemetry on GCP實戰 - Shawn Ho|
|10:50 AM|Room 301|Firebase Auth 與無密碼驗證的 WebAuthn API - Richard Lee|
|11:40 AM|Room 105|Future-Proofing Enterprise Conversations: Integrating NeMo-GuardRails & PaLM2 - JimmyLiao|
|1:00 PM|Room 101|多模態模型-Gemini - Jerry|
|1:00 PM|Room 306|像個資深工程師一樣寫程式 － Act like a Flutter Senior - Aki Yu|
|1:00 PM|Room 105|用 LLM 來整理您的知識庫 - Johnny Sung (宋岡諺)|
|1:00 PM|Room 202|防量子攻擊的開發套件 - Yang Lin|
|1:50 PM|Room 101|Advanced Knowledge Management with Generative AI - Michael Chi|
|1:50 PM|Room 202|如何進一步強化App資安 - 林育峰|
|1:50 PM|Room 105|GCP 監控、自動化與合規 - Johnny Yeng|
|1:50 PM|Room 304|Evolution of Transformers - Anjana|
|2:40 PM|Room 304|How to evaluate large language models - Daniel Eberts|
|3:30 PM|Room 304|Large Language Model:Prompt Engineering, Fine-Tuning. - Jit|

## [Devfest Taipei 講者投影片](https://drive.google.com/drive/folders/1xnOLdSoXowV82WLZu4NQrekSBHBt_dSw?usp=drive_link)

## Note

### Opening - EvA ChU、Eric ShangKuan

Google 智慧台灣計畫

### Keynote - 展望 Generative AI 的現在與未來 - 薛良斌、KJ Wu (吳貴融)、林雅芳 (Tina Lin)

:::spoiler Note
#### KJ Wu (吳貴融)

1. Internet
2. Mobile
3. AI is the **thrid big shift**

**Google Cloud Platform Foundation**
- Modern information
- Data Analytics
- AI/ML
- Security

A decade of experience in infrasturce and development (Container)

Consumers & enterprises different need (GCP)

Cloud AI Portfolio

#### 薛良斌

Hallucination (幻覺)

Guardrails (護欄)

**Insight**
1. LLM 的本質就是幻覺 (It's not a Bug, It's a Feature)
2. LLM 不會思考
3. 人類對幻覺充滿期待

**可能性**
1. 十倍生產力
2. 自動化 (透過 AI 自動化以前需要作決定的重複性任務)
3. 代理人 (AI 擔任虛擬代理人腳色，與人或AI進行對話或進行互動)
4. 多國語言 + 本土化
5. 模型混搭
:::

### Thread-blocking I/O is blocking your future (JVM)
:::spoiler Note
#### Thechniqure to build high throughput with kotlin application.
> Gary Lo

#### Spring boot
- Two Types:
- Spring MVC : Synchronous blocking I/O with onre request per thread model
- Spring WebFlux : Non-blocking web framework with massive numbers of request model
#### Threading concept
- What is thread blocking I/O
- Wating for system call for continue parallel programing
- Keep waiting low efficiency
- Non-blocking thread I/O
- No need to wait for system call
- with Callback function can opeerate more efficient
- More thread ==> More memory
- More Context switch ==> more chance to Cache Miss in cpu.
#### How blocking IO get worse in microservice architechture age ?
- Practical microservice architecture: cross service dependencies.
- Max throughput (2000 threads Avg. I/O 500 m/s):
= 200 /0.5
= 400 req/s
- What if worse coding make trigger performance degrade ?
- Other thread keep waiting for the main thread finish.
#### More Threads ?
- Spring MVC [pic]
- Hwo about Spring webflux ? [pic]
#### If there is Spring webflux avaliable, why developers don't use it ?
- Premature optimization is the root of all evil.
- Performance Optimization I/O [pic]
#### Java reative I/O reactor
#### Time line [pic]
#### Java v.s Kotlin coroutine
#### Virtual Thread - Now concept
:::

### Firebase Auth 與無密碼驗證的 WebAuthn API

:::spoiler Note
> 1. 驗證登入
> 2. 授權

#### 為甚麼要驗證登入?
1. 將資料與特定使用者綁定
2. 限制使用者數量
3. 實作後台管理

* 自製登入系統的缺點?
* 匿名使用者也應該要登入系統


* 再從匿名使用者轉成永久帳號


* 單要素驗證: 只要帳號/密碼就可以登入
* 可共享
* 但密碼不夠強
* 容易被猜到
* 需要提供忘記密碼


* 多重要素驗證
* 簡訊驗證碼
* Email 連結登入
* 第三方登入 (Google/Facebook/Apple)
* WebAuthn




#### WebAuthn

允許使用者利用公鑰加密技術進行身分驗證

Passkey




**會上傳範例程式碼/Powerpoint**

:::

### Future-Proofing Enterprise Conversations: Integrating NeMo-GuardRails & PaLM2

:::spoiler Note
Decision Tree / NLP Engine

NeMo-GuardRails

Challenges
1. Chat with your Data (Public / Private dataset)
2. Hallucination
3. Content Moderation / Safety

RAG with Google Vertex AI
1. Create a corpus
2. Ingestion
3. Querying
4. Show response
5. Show cited paddages
6. Reuse corpus

Content Safety
- `SafetySetting`
- Blocklist


Garak

:::

### 多模態模型-Gemini

:::spoiler Note



:::

### Evolution of transformers

:::spoiler Note
#### Turning words to numbers that make sense
- Word to vec (Skip Gram)
- Neural Network

#### How to handle sequential data?
- Recurrent Neural Networks
- Long Short Term Memory (LSTM)

#### The Sequence 2 Sequence Problem
- Encoder-Decoder Models
- Drawback: Data can't do parallel

#### Transformer
- Positional Encoding
- Attention Mechanism: using cosine similarity to let model understand the word relation between different word in same sentence
- Multi-head Attention

[ControlNet-XS: Designing an Efficient and Effective Architecture for Controlling Text-to-Image Diffusion Models](https://arxiv.org/abs/2312.06573)

:::

### How to evaluate large language models

:::spoiler Note

#### Manual evaluation

[Mistral](https://docs.mistral.ai/)

[Chi-square test](https://zh.wikipedia.org/zh-tw/%E5%8D%A1%E6%96%B9%E6%A3%80%E9%AA%8C)


#### With a metric

Example: Text summariztion
- ROUGE score: n-gram / matching
- BERTScore: Difference to cosine similarity of embedding vectors

- ROUGE Socore
- Simpler
- Bert Score
- More Precise with human judgement.



#### Without Metric
- KAIST - [Prometheus](https://github.com/kaistAI/Prometheus)
- Naverai baai judgeLM
ask another llm to be the judge
inputs:
metric (rubric) - could be customrized
reference gold standard
description of criteria


:::

### Large Language Model: Prompt Engineering

:::spoiler Note

#### Prompt Engineering

- Open-ended prompts
- Closed-ended prompts
- Instruction-based prompts


- Chain-of-thought (CoT)
- [Reasoning and Action (ReAct)](https://arxiv.org/pdf/2210.03629.pdf)

Prompt injection (positively / negatively)

Ethical considerations
- Jailbreaking
- Prompt leaking
- Token smuggling

#### Finetune
- LoRA (Low-Rank Adaptation)


[Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning](https://arxiv.org/abs/2302.02662)

[Prompt engineering Wiki](https://en.wikipedia.org/wiki/Prompt_engineering)

:::