Try   HackMD

Chatbots Lab

Location: Room 223A, 德田館
Time: 19:00 ~ 21:00, Mon. and Thu.

``AI is not here to replace us;
it’s here to amplify us.''
-- Mustafa Suleyman

Those who guard their lips preserve their lives,
but those who speak rashly will come to ruin.
-- Proverbs 13:3 (NIV)

Course Information

Instructor

Objectives

  • Understand the fundamentals of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG).
  • Build QA chatbots leveraging RAG to answer questions based on specialized knowledge bases.
  • Deploy the QA chatbot to a cloud server (e.g., GCP) providing with simple web UI.
  • Develop evaluation mechanisms to assess the chatbot performance and optimize its responses.

Prerequisites

  • Proficiency in Python programming.

Tech Stack

  • Python fundamentals
  • Basic operations on Google Colab to developing chatbots
  • LLM APIs
  • Vector database: ChromaDB
  • Langchain frameworks: RetrievalQA
  • Web app using Gradio, Streamlit, Flask
  • Evaluation frameworks: Langsmith, RAGAS
  • Deployment as cloud service: GCP (or Azure, AWS)
  • Virtualization using Docker
  • LLMOps

Recording Classroom Lectures Policy

Recording of classroom lectures is prohibited unless advance written permission is obtained from the class instructor and any guest presenter(s).

Syllabus

Day 0 Introduction slides

  • Overview of LLMs and RAG
    • What are LLMs? How do they work?
    • Introduction to RAG: Retrieval + Generation synergy
    • Use cases for QA chatbots
  • Setting Up the Development Environment
    • Configure Google Colab and local Python environment
    • Install and test libraries
    • Convert your knowledge base (e.g., PDF) into text for processing
    • Hands-on: Run a pre-trained LLM to generate text

Day 1 Building the RAG System slides

  • Vectorizing the Knowledge Base
    • Convert text to embedding vectors using embedding models
    • Set up a vector database (e.g., ChromaDB) for retrieval
    • Hands-on: Retrieve relevant documents for sample queries
  • Integrating LLM for Answer Generation
    • Combine retrieved documents with LLM to generate answers
    • Hands-on: Build a simple RAG pipeline

Day 2 Evaluation and Optimization slides

  • Evaluating Chatbot Performance
    • Automated metrics: BLEU, ROUGE, Precision/Recall
    • Manual evaluation: Design a scoring rubric (accuracy, fluency, relevance)
    • Hands-on: Evaluate your chatbot with a test set
  • Optimizing the RAG System
    • Improve retrieval (e.g., tweak embedding model)
    • Enhance generation (e.g., refine prompts)
    • Hands-on: Optimize based on evaluation results

Day 3 Deploying to the Cloud slides

  • Google Cloud Platform (GCP)
    • Create a new project with a virtual machine (also add a new firewall rule for service)
    • BASH: basic commands (ssh, cp, mkdir, mv, apt, and so)
  • Docker Techniques
    • Pack the app using Dockerfile: build an image and run a container
  • Deploying the Chatbot
    • Hands-on: Test your chatbot with sample queries

Day 4 Demenstration slides

  • Workshop for students' chatbots
  • Future works

References

Courses

LLM

Frameworks

Embeddeing Models

Vector Databases

Cloud Serverless

MISC

Gradebook

Last changed by 
 
Zheng-Liang Lu
Writing for future us.
0
727
Published on HackMD