## Summary Notes

### Wednesday Jan 10th, 2024

9:00 AM Nirav's introduction.

Mithun discussion: What are LLM?
- what is intelligence?
- ChatGPT is not intelligent
- What are LLM?
- What is a Turing test?
- Can chatGPT pass a test?
- Why are we here today?
- What are the issues we will come up against?
- Privacy concerns: all information passed to ChatGPT is public, not good for HIPAA, FERPA and HSR proteted data
- "No guardrails" in terms of biases

10:00 AM

- [Ollama](https://github.com/jmorganca/ollama) Nirav demo running in CyVerse (running on [GPU 06:](http://gpu06.cyverse.org:8080))
- Goal for the codeathon: create an open source chat bot that resembles ChatGPT.
- Allows:
- Professors to upload data
- Students to ask questions
- Timeline of deliverables:
- Wed: model definition, GUI, Cloud Infrastructure, Auth, Course info
- Thu: minimum viable product
- Fri: testing
- Testing material:
- Class RNR355 material is going to be used for testing
- vs ChatGPT4:
- Known limitation using openAI: max 10 files, 25MB per file, USD20 subscription
- Thoughts:
- Cost
- Data ingenstion (mathematics, pdfs, images)
- Efficiency- does every information retreival need LLMs?
- The realities of time and effort constraints- important to focus on interpretability existing tools
- Focus on quality of answers.

10:45

Tyson
- Winter project: set up information retreival chatbot for course materials
- React app that interfaces with OpenAI backend
- ChatGPT 4.0 chat is accurate, accuracy falls and speed increases if ChatGPT 3 is used
- Tutorbot, vs Teacherbot- future stretch goal of creating something that can work with private data between teachers and students for personalized teaching materials
- [Flowise AI](https://flowiseai.com/)
- Could be used to create what we aim to do
- Can work with both open access and paywalled models

11:00

Nirav
- Storyboarding for the chatbot and use case, for a walk through
- Two use cases- course materials, website
- Grouper- how to restrict chatbot access to each course, so authorization is restricted
- What other universities are doing?
- Reading Math equations
- Is LLM training data private or public?
- How deep to follow URL links in a document?
- Use Local API for privacy
- Low code-no code for Professors
- Internal
- Need to integrate other commercial solutions if plausible
- D2L is moving to Canvas (LMS chatbot)
- Office 365 (high prices for including chatbots)

Mithun
- Laying down basis for day achievable goals
- Described some initial tasks for the Team leads
- Prompt engineering
- Each team decides more realistic tasks

1:30PM

### Propeller Team (LLM)

- how a pipeline works.

1. Create Vector DB; Chunks of documents, pass them through LLM in order to create Vector DB.
2. Retrieval: Get the closest document to the query
3. Generate: Depending on the task: Summary/ QA/ Translating Document etc.

Terminology:

Retrieval Augmented Generation (RAG)

- prime the LLM with context, which can help combat hallucinations
- entire text book is too long to provide in the prompt, rather we use the vector db to pick relevant chunks for dynamic prompt

*Technologies:*
- [Chroma](https://www.trychroma.com/)
- [LangChain](https://www.langchain.com/)
- [Nvidia Nemo](https://www.nvidia.com/en-us/ai-data-science/products/nemo/get-started/)

### Rudder (Back-End)
- items

### Crow's nest (GUI)
- items
-

### Anchor (Testing/QA)
- items
-