LLM for Devs - Week 4 Project

# LLM for Devs - Week 4 Project ### Exploring Agents and Multi-Agent Systems In this assignment, you will choose **one** of four project paths to deepen your understanding of agents and multi-agent systems powered by large language models (LLM). Each path includes both a Jupyter notebook component to be completed in Google Colab. Select **one** of the following project paths based on relevance to your group project: 1. **Production Path** - Develop and deploy a multi-agent system as a web application using Chainlit, Streamlit, or a custom front-end with an optional [FastAPI](https://fastapi.tiangolo.com/) (or similar) backend 2. **Combining Techniques Path** - If your group project involves information retrieval, implement a system using [RAG](https://arxiv.org/abs/2005.11401) with a vector store and agents 3. **Library Evaluation Path** - If you're considering using an agent library for your group project, implement a multi-agent system using three different libraries that support agents (e.g., [CrewAI](https://www.crewai.com/), [LangGraph](https://python.langchain.com/v0.1/docs/langgraph/), [Langroid](https://langroid.github.io/langroid/), [AutoGen](https://microsoft.github.io/autogen/)) and compare/contrast performance 4. **Group Project Path** - This is pretty straightforward. Use this week to continue work on your group project. For building multi-agent systems, we highly recommend using the [CrewAI](https://www.crewai.com/) framework. [CrewAI](https://www.crewai.com/) provides a simple, prompting-based way to build multi-agent systems. We have examples using [CrewAI](https://www.crewai.com/) in [the Week 4 repo here](https://github.com/Devs-Do-AI/Week-4---Agents). We have a few resources for getting started with [CrewAI](https://www.crewai.com/) in the Resources section [here](https://hackmd.io/ww9jnZm0TlSdx5c8J5EY_w?both#Resources). ## 1 - Production Path In this project, you will develop a multi-agent system relevant to your group project and deploy it as a user-friendly web application. We recommend [Chainlit](https://docs.chainlit.io/get-started/overview) if it is a chatbot or [Streamlit](https://streamlit.io/) if it uses a non-chat UI. Alternatively, if you prefer to build the front-end using a different technology (e.g., a JavaScript web application), you may need to deploy your agents as an API using a library like [FastAPI](https://fastapi.tiangolo.com/). The system should include at least 2 specialized agents that collaborate to accomplish a task. ### 📓 Jupyter Notebook Component (2-5 hours) - Set up a Google Colab notebook - Implement 2 agents with clearly defined roles using [CrewAI](https://www.crewai.com/) that are relevant to your group project - Have the agents communicate and pass information between each other - Test the system with a few example queries ### 💻 Web Application Component (2-5 hours) - Create a user-friendly web application using [Chainlit](https://docs.chainlit.io/get-started/overview), Streamlit, or the technology of your choice - Integrate the agent system into the web application - Implement user input functionality and display the results generated by the agents - Deploy the web application to Hugging Face Spaces for easy access and sharing - [Optional] Apply appropriate styling and layout to enhance the user experience ### API Deployment Component (Optional, 2-5 hours) If you choose to build the front-end using a different technology (e.g., a JavaScript web application), follow these steps: - Refactor the agent system into a clean, modular FastAPI (or your preferred framework) application - Implement API endpoints for each agent and the overall system - Incorporate guardrails to prevent the agents from going off-track - Deploy the API to a cloud platform like [AWS](https://aws.amazon.com/), [Google Cloud](https://cloud.google.com/), or [Hugging Face Spaces](https://huggingface.co/spaces) - Integrate the API with your custom-built front-end application Remember to document your code, provide clear instructions for running the application, and share the link to your deployed web application or API. ## 2 - Combining Techniques Path In this project, you will implement a retrieval-augmented generation (RAG) system that uses a vector database to provide knowledge to a multi-agent system relevant to your group project. ### 🗒️ Jupyter Notebook Component (4-6 hours) - Set up a Google Colab notebook - Choose a dataset relevant to your group project to use as the knowledge base - Create embeddings and a vector database from the dataset using a library like [FAISS](https://github.com/facebookresearch/faiss) - Implement a retriever agent that uses the vector database to find relevant information for a query - Implement a generator agent that takes the retrieved information and the query to generate an answer - Allow the agents to collaborate until a satisfactory answer is generated - Test the system with example queries relevant to your project ### 💻 [Optional] Production Component (4-6 hours) - Refactor the RAG system into a [FastAPI](https://fastapi.tiangolo.com/) application - Use a hosted vector database like [Pinecone](https://www.pinecone.io/) or [Weaviate](https://weaviate.io/) - Implement API endpoints for queries - Report results including example queries and answers relevant to your project - Stretch goal: Implement a feedback loop where user feedback is used to improve the vector database and agent prompts over time ## 3 - Library Evaluation Path In this project, you will implement the same multi-agent system relevant to your group project using three libraries that support agents (e.g, [CrewAI](https://www.crewai.com/), [LangGraph](https://python.langchain.com/v0.1/docs/langgraph/), [Langroid](https://langroid.github.io/langroid/), [AutoGen](https://microsoft.github.io/autogen/), etc.), and compare both the developer experience and results. ### 🗒️ Jupyter Notebook Component (4-6 hours) - Set up a Google Colab notebook - Choose a task relevant to your group project for a multi-agent system to perform - Implement the system using three libraries that support agents (e.g, [CrewAI](https://www.crewai.com/), [LangGraph](https://python.langchain.com/v0.1/docs/langgraph/), [Langroid](https://langroid.github.io/langroid/), [AutoGen](https://microsoft.github.io/autogen/), etc.) - Have at least 2 agents with clear roles and communication - For each implementation, document your experience as a developer - Consider ease of use, flexibility, documentation quality, roadblocks, etc. - Test each implementation with example queries relevant to your project - Document and compare the quality of the results from each implementation ## 4 - Group Project Path This is pretty straightforward. Just continue to work on your group project. ## Deliverables - Google Colab notebook with all code, explanations, and example outputs - If applicable, include the following as separate blocks within your Colab notebook - Link to GitHub repo - Link to any production deployments - Any architecture diagrams linked to or embedded in the notebook - A short writeup with key learnings and reflections ## Resources ### CrewAI [Multi-Agent systems with CrewAI (DeepLearning.AI)](https://learn.deeplearning.ai/courses/multi-ai-agent-systems-with-crewai/lesson/1/introduction) (the most fully-featured walkthrough of the library) [Week 4 - Agents repository with CrewAI examples](https://github.com/Devs-Do-AI/Week-4---Agents) (lessons 1-6 build from simple crews to more sophisticated crews) #### YouTube videos [CrewAI Tutorial: Complete Crash Course for Beginners](https://www.youtube.com/watch?v=sPzc6hMg7So) [How I Made AI Assistants Do My Work For Me: CrewAI](https://www.youtube.com/watch?v=kJvXT25LkwA&pp=ygUHY3JldyBhaQ%3D%3D) ### LangGraph [LangGraph playlist](https://www.youtube.com/watch?v=5h-JBkySK34&list=PLfaIDFEXuae16n2TWUkKq5PgJ0w6Pkwtg) ### Prompt Engineering [PromptingGuide](https://promptingguide.ai/techniques/) ## Project Submission If you chose a path that requires submission of a Google Colab notebook, submit it using the form [here](https://forms.gle/DKeRAuYkvDQGjs9P9). Please make sure sharing permissions are turned on for everyone with the link. The notebooks are due 11:59PM Pacific Time the evening before the class session.