Notes on Large Language Model

# Notes on Large Language Model ###### tags: `AGI` ## Table of Contents [toc] ## TODOs & Notes on the go - [x] move local notes from obsidian to hackmd - [x] take notes from web links in safari tab groups - [x] organize this catalog for better readability ## Models ### OpenAI ### Llama - [huggingface doc](https://huggingface.co/docs/transformers/main/model_doc/llama) ### Alpaca - [github repo](https://github.com/tatsu-lab/stanford_alpaca) ### [Hugging Face](https://huggingface.co/models?sort=likes) ## Techniques and Frameworks ### Prompt Engineering - [guide hub](https://www.promptingguide.ai) - [guide repo](https://github.com/dair-ai/Prompt-Engineering-Guide#guides) - [ReAct blog](https://www.promptingguide.ai/techniques/react) - [ReaAct example](https://python.langchain.com/en/latest/modules/agents/agents/examples/react.html) - [ReAct paper](https://arxiv.org/pdf/2210.03629.pdf) - [MemPrompt](https://memprompt.com), memory-assisted prompt editing with user feedback - [inception example](https://twitter.com/goodside/status/1583262455207460865?s=20&t=8Hz7XBnK1OF8siQrxxCIGQ), instead of just asking for it, implant it as a false memory of the last thing the model has said, as though it were the model’s idea. ### Chain of thought - [chain-of-thought paper](https://arxiv.org/pdf/2201.11903.pdf), asking agents to elicit intermediate reasoning steps - [self-ask paper](https://ofir.io/self-ask.pdf), improving on chain-of-thought technique by asking follow-ups before answering the original question - [self-ask example](https://python.langchain.com/en/latest/modules/agents/agents/examples/self_ask_with_search.html) - [self-consistency paper](https://arxiv.org/pdf/2203.11171.pdf), improving on chain-of-thought with the most consistent path rather than the greediest - [step-by-step paper](https://arxiv.org/pdf/2112.00114.pdf), asking agents to perform step-by-step operationch - [reflexion paper](https://arxiv.org/pdf/2303.11366.pdf), autonomous agent with dynamic memory and self-reflection - [RCI paper](https://arxiv.org/pdf/2303.17491.pdf), an agent that Recursively Criticizes and Improves its response ### RLHF - [huggingface blog](https://huggingface.co/blog/rlhf) - [awesome rlhf repo](https://github.com/opendilab/awesome-RLHF) - [DeepSpeed readme](https://github.com/microsoft/DeepSpeed/blob/master/blogs/deepspeed-chat/README.md) - [paper collection](https://github.com/opendilab/awesome-RLHF) ### Automation - [autogpt repo](https://github.com/Significant-Gravitas/Auto-GPT) - [babyagi repo](https://github.com/yoheinakajima/babyagi) ### LangChain - read [Notes on LangChain](/tlk4zzBIQrGtEqMgOPxaaw) for a quick onboarding - read [Notes on LlamaIndex](/DQSfbVqITm2IaUewiUz3Nw) for a quick onboarding - [langchain python doc](https://python.langchain.com/en/latest/index.html) - [langchain blogs](https://blog.langchain.dev) - [llamaindex python doc](https://gpt-index.readthedocs.io/en/latest/index.html) ### OpenAPI tools - [openapi generator](https://github.com/openapitools/openapi-generator) - [tools list](https://openapi.tools/#data-validators) ### Other frameworks - [haystack](https://github.com/deepset-ai/haystack), an open-source framework for building search systems that work intelligently over large document collections ### Providers - [SteamShip](https://www.steamship.com), build, host and share AI apps - [ColossalAI](https://colossalai.org), replicate ChatGPT - [Pinecone](https://www.pinecone.io), vector database hoster ## Products ### companionship bot - to be added... ### document assistant - [mendable.ai](https://www.mendable.ai), chatbot for software documentation - [ResearchAIde](https://www.researchaide.org), chatbot for academic papers - [petal](https://www.petal.org), document analysis platform you can chat with - [chatpdf](https://www.chatpdf.com), webapp for Q&As on pdfs - [perplexity](https://www.perplexity.ai), web plugin for Q&As on web documents - [recast](https://www.letsrecast.ai), text article -> audio podcast - [anthropic](https://www.anthropic.com/index/100k-context-windows): LLMs with 100k tokens as context ### productivity - [raycast](https://www.raycast.com), smart shortcuts in spotlight search - [shortcutAI](https://shortcutai.com/docs), ChatGPT in Apple Notes ### copilot - [tabby repo](https://github.com/TabbyML/tabby?utm_source=tldrai), self-hosted AI coding assistant - [apollo+openai](https://zapier.com/apps/apollo/integrations/openai?utm_source=google&utm_medium=cpc&utm_campaign=gaw-gbl-nua-search-one_two_service_2023&utm_adgroup=dsa_one_service_activecampaign&utm_term=&utm_content=_pcrid_656581312204_pkw__pmt__pdv_c_slid__pgrid_142946974782_ptaid_dsa-1944672399734_&gad=1&gclid=CjwKCAjwjYKjBhB5EiwAiFdSfukuFV36hlSfR0EWdqNFceMUpShDaIQX_UXVltihnnNovmpzP3Zk5RoCH0sQAvD_BwE), ChatGPT in your airpods ### generic - [open assistant repo](https://github.com/LAION-AI/Open-Assistant), conversational AI for everyone ### No code - [builder.ai](https://www.builder.ai), app builder ## Lists and Tables - [ML papers of the week](https://github.com/dair-ai/ML-Papers-of-the-Week?utm_source=tldrai), updated weekly - [AI Tool Arsenal spreadsheets](https://docs.google.com/spreadsheets/d/1_K6oMcysbZ-VIjA8h1Q60xEa6RtGjN4y/edit?usp=sharing&ouid=111659874883047729679&rtpof=true&sd=true), Mar 6 - [DevPost](https://devpost.com/software), explore projects from portfolios and hackathons