# Notes on LangChain

###### tags: `AGI` `langchain`

:::info
:information_source: this note is made from official [**documentation**](https://python.langchain.com/en/latest/index.html) of langchain

:thumbsup: Essential Links: [**installation**](https://python.langchain.com/en/latest/reference/installation.html), [**references**](https://python.langchain.com/en/latest/reference.html), [**code base**](https://github.com/hwchase17/langchain/tree/master/langchain), [**mindmap**](https://github.com/mayooear/langchain-js-tutorial/blob/main/visual-image/langchainjsfundamentals.png)
:::

## Table of Contents
[toc]

## Overview
### [Modules](https://docs.langchain.com/docs/category/components)
- **Schema**
- **Text**
- **ChatMessages** includes SystemChatMessage, HumanChatMessage and AIChatMessage in a dictionary
- **Examples** input/output pairs for training and evaluating models
- **Document** consists of `page_content` and `metadata`
- **Models**
- **Language Model** inputs text and outputs text
- **Chat Model** inputs a list of ChatMessages and outputs a ChatMessage
- **Text Embedding Model** inputs text and outputs embedding representations for documents or queries
- **Prompts**
- **Prompt Value** represents an input to a model
- **Prompt Template** constructs a PromptValue
- **Example Selector** selects examples to include in PromptValue
- **Output Parser** instructs the model how to format output and parse the output into desired formatting
- **Indexes**
- **Document Loader** loads documents from various sources
- **Text Splitter** splits text into smaller chunks
- **VectorStore** stores documents as index
- **Retrievers** fetches relevant documents to combine with models
- **Memory**
- **Chat Message History** stores/updates and retrieves data in the process of a conversation
- **Chains**
- [**Chain**](https://github.com/hwchase17/langchain/blob/master/langchain/chains/base.py#L28) an end-to-end wrapper around multiple components
- **LLMChain**
- consists of a PromptTemplate, a model and an output parser
- **Index Chains** work with documents/indexes
- **Router Chains** call next chain to execute
- **Agents**
- [**Tool**](https://python.langchain.com/en/latest/reference/modules/tools.html#langchain.tools.BaseTool), a text-in-text-out function
- **Toolkit**, a group of tools for a particular problem
- [**Agent**](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.Agent), a wrapper around a model, inputs a prompt, uses a tool, and outputs a response
- [**Agent Executor**](https://github.com/hwchase17/langchain/blob/master/langchain/agents/agent.py#L620), a wrapper around an agent and a set of tools; responsible for calling the agent and using the tools; can be used as a chain

### Relationship
- **Atomic Components**
- Model
- Memory
- Prompt
- Index
- **Compound Components**
- Tools
- Agents
- Chains
- **Relationship**
- agents can select tools
- chains can route chains
- all compound components can use chains
- all compound components can be made out of chains
- all compound components can use all atomic components
- **Paradigms**
- tools use index, prompt and model
- agents use tools, prompt and model (possibly with memory)
- chains use agents, prompt and model (possibly with memory)
- chains use other chains or get used by tools

## Modules
### Models
- use LLMs with [`OpenAI(model_name="name")`](https://python.langchain.com/en/latest/modules/models/llms/getting_started.html)
- test LLMs with [`FakeListLLM`](https://python.langchain.com/en/latest/modules/models/llms/examples/fake_llm.html)
- stream LLM response with [streaming](https://python.langchain.com/en/latest/modules/models/llms/examples/streaming_llm.html) parameter in LLM configuration
- track token usage using [`get_openai_callback`](https://python.langchain.com/en/latest/modules/models/llms/examples/token_usage_tracking.html)
- see [chat models](https://python.langchain.com/en/latest/modules/models/chat/getting_started.html) and [text embedding models](https://python.langchain.com/en/latest/modules/models/text_embedding.html) for more
### Prompts
- [StringPromptTemplate](https://python.langchain.com/en/latest/modules/prompts/prompt_templates/examples/custom_prompt_template.html)
- [FewShotPromptTemplate](https://python.langchain.com/en/latest/modules/prompts/prompt_templates/examples/few_shot_examples.html)
- [`LengthBasedExampleSelector`](https://python.langchain.com/en/latest/reference/modules/example_selector.html#langchain.prompts.example_selector.LengthBasedExampleSelector), useful when you are worried about constructing a prompt that will go over the length of the context window
- [`MaxMarginalRelevanceExampleSelector`](https://python.langchain.com/en/latest/reference/modules/example_selector.html#langchain.prompts.example_selector.MaxMarginalRelevanceExampleSelector), useful for selecting the most relevant examples measured by cosine similarity between the inputs and embeddings
- [`SemanticSimilarityExampleSelector`](https://python.langchain.com/en/latest/reference/modules/example_selector.html#langchain.prompts.example_selector.SemanticSimilarityExampleSelector), useful for selecting the semantically most similar examples
- [Prompt Serialization](https://python.langchain.com/en/latest/modules/prompts/prompt_templates/examples/prompt_serialization.html)
- simple_prompt.json
- few_shot_prompt.json
- loading simple prompt from json file
- [Output Parsers](https://python.langchain.com/en/latest/modules/prompts/output_parsers/getting_started.html)
- must implement
- `get_format_instructions() -> str`, returns a string of instructions for how an output of an LLM should be formatted
- `parse(str) -> Any`, inputs a string and ouputs some structure
- [`StructuredOutputParser`](https://python.langchain.com/en/latest/modules/prompts/output_parsers/examples/structured.html), useful for ouputing a simple json schema
- [`PydanticOutputParser`](https://python.langchain.com/en/latest/modules/prompts/output_parsers/examples/pydantic.html), useful for outputing a complex json schema
- [`RetryOutputParser`](https://python.langchain.com/en/latest/modules/prompts/output_parsers/examples/retry.html), useful for making sure the output is complete
- [`OutputFixingParser`](https://python.langchain.com/en/latest/modules/prompts/output_parsers/examples/output_fixing_parser.html), useful for fixing the structure of the output
- check out [Kor](https://eyurtsev.github.io/kor/index.html) for more types of extractions
- Community prompts from [`langchain-hub/prompts`](https://github.com/hwchase17/langchain-hub/tree/master/prompts)

### Memory
- **`ChatMessageHistory`** underpins most memory modules; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/chat_message_histories/in_memory.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ChatMessageHistory), [examples](https://python.langchain.com/en/latest/modules/memory/getting_started.html#chatmessagehistory)
- **Conversation Buffer Memories**
- **`ConversationBufferMemory`** wraps around ChatMessageHistory and extracts the messages in a variable; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/buffer.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationBufferMemory), [examples](https://python.langchain.com/en/latest/modules/memory/getting_started.html#conversationbuffermemory)
- **`ConversationBufferWindowMemory`** essentially a `ConversationBufferMemory` with sliding window; only keeps the last `k` interactions; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/buffer_window.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationBufferWindowMemory) and [guides](https://python.langchain.com/en/latest/modules/memory/types/buffer_window.html)
- **`ConversationTokenBufferMemory`** essentially `ConversationBufferMemory` that uses token length rather than number of interactions to determine when to flush interactions; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/token_buffer.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationTokenBufferMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/token_buffer.html)
- **`ConversationSummaryBufferMemory`** essentially a `ConversationTokenBufferMemory` that summarizes instead of flushing old interactions; uses token length rather than number of interactions to determine when to flush interactions; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/summary_buffer.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationSummaryBufferMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/summary_buffer.html)
- **`ConversationEntityMemory`** extracts information on entities and builds up its knowledge about that entity over time; entities are determined by LLM; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/entity.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationEntityMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/entity_summary_memory.html)
- **`ConversationKGMemory`** uses a knowledge graph to recreate memory; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/kg.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationKGMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/kg.html)
- **`ConversationSummaryMemory`** creates a summary of the conversation over time; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/summary.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.ConversationSummaryMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/summary.html)
- **`VectorStoreRetrieverMemory`** stores memories in a vectorDB and queries the top-K most "salient" docs; does not explicitly track the order of interactions; [code](https://github.com/hwchase17/langchain/blob/master/langchain/memory/vectorstore.py), [docs](https://python.langchain.com/en/latest/reference/modules/memory.html#langchain.memory.VectorStoreRetrieverMemory), and [guides](https://python.langchain.com/en/latest/modules/memory/types/vectorstore_retriever_memory.html)
- adding memory to an agent, [guides](https://python.langchain.com/en/latest/modules/memory/examples/agent_with_memory.html)
- adding memory to an agent with external message store, [guides](https://python.langchain.com/en/latest/modules/memory/examples/agent_with_memory_in_db.html)
- customize conversational memory, [guides](https://python.langchain.com/en/latest/modules/memory/examples/conversational_customization.html)
- customize memory class, [guides](https://python.langchain.com/en/latest/modules/memory/examples/custom_memory.html)
- use multiple memory class in the same chain, [guides](https://python.langchain.com/en/latest/modules/memory/examples/multiple_memory.html)
- memory with db: [dynamodb](https://python.langchain.com/en/latest/modules/memory/examples/dynamodb_chat_message_history.html), [momento cache](https://python.langchain.com/en/latest/modules/memory/examples/momento_chat_message_history.html), [mongodb](https://python.langchain.com/en/latest/modules/memory/examples/mongodb_chat_message_history.html), [postgres](https://python.langchain.com/en/latest/modules/memory/examples/postgres_chat_message_history.html), [redis](https://python.langchain.com/en/latest/modules/memory/examples/redis_chat_message_history.html)
- **[zep](https://python.langchain.com/en/latest/modules/memory/examples/zep_memory.html)**
- Long-term memory persistence, with access to historical messages irrespective of your summarization strategy.
- Auto-summarization of memory messages based on a configurable message window. A series of summaries are stored, providing flexibility for future summarization strategies.
- Vector search over memories, with messages automatically embedded on creation.
- Auto-token counting of memories and summaries, allowing finer-grained control over prompt assembly.

### Indexes
- constructs indexes to use them later as retrievers
- builds vectorstore from documents
- splits documents into chunks
- creates embeddings out of chunks
- saves documents and embeddings into vectorstore
- builds retriever from vectorstore
- builds `RetrievalQA` chain using retriever
- check out [examples](https://python.langchain.com/en/latest/modules/indexes/getting_started.html#walkthrough) for more
- Document Loaders
- [`UnstructuredFileLoader`](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/unstructured_file.html)
- [`UnstructuredURLLoader`](https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/url.html)
- check out [more](https://python.langchain.com/en/latest/modules/indexes/document_loaders.html)
- Vectorstores
- check out [this medium post](https://medium.com/sopmac-ai/vector-databases-as-memory-for-your-ai-agents-986288530443) for conceptual understanding
- check out [llamaindex's note](https://gpt-index.readthedocs.io/en/latest/examples/vector_stores/WeaviateIndexDemo.html) for loading indexes from vectorstores
- [pinecone](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/pinecone.html), cloud only
- [weaviate](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/weaviate.html), open source
- [chroma](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/chroma.html), open source
- [qdrant](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/qdrant.html), local db
- [Retrivers](https://python.langchain.com/en/latest/modules/indexes/retrievers.html)
- a generic interface to combine documents with llms
- exposes a `get_relevant_documents` method that inputs a query string and outputs a list of documents
- see [vectorstore retriever](https://python.langchain.com/en/latest/modules/indexes/retrievers/examples/vectorstore.html) for example
- check out [list](https://python.langchain.com/en/latest/modules/indexes/retrievers.html) for more retrievers

### Chains
- **LLM Chains**
- **`LLMChain`**
- formats the `PromptTemplate` using the input key values provided, passes the formatted string to `llm` and returns the `llm` output,
- i.e. `outputs = chain(inputs)` where both inputs and outputs are dicts;
- sets `verbose=True` when constructing chain for debugging;
- [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.LLMChain), [guides](https://python.langchain.com/en/latest/modules/chains/generic/llm_chain.html) and [examples](https://python.langchain.com/en/latest/modules/chains/getting_started.html)
- **`LLMCheckerChain`** answers questions with self-verification; [code](https://github.com/hwchase17/langchain/blob/9c0cb90997db9eb2e2a736df458d39fd7bec8ffb/langchain/chains/llm_checker/base.py#L65) and [examples]((https://python.langchain.com/en/latest/modules/chains/examples/llm_checker.html))
- **`LLMMathChain`** does complex word math problems; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/llm_math/base.py) and [examples](https://python.langchain.com/en/latest/modules/chains/examples/llm_math.html)
- **`LLMRequestsChain`** gets HTML results from a URL and then an LLM to parse results; [code] and [examples](https://python.langchain.com/en/latest/modules/chains/examples/llm_requests.html)
- **`LLMSummarizationCheckerChain`**
- uses a sequential chain to revise the ouput until everything checks out
- sequential chain is made up of a fact extraction chain, fact checking chain, fact reivision chain, revision checking chain,
- beneficial to run this checker multiple times to minimize hallucination by setting `max_checks`
- [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/llm_summarization_checker/base.py) and [examples](https://python.langchain.com/en/latest/modules/chains/examples/llm_summarization_checker.html)
- **Index Chains** or **`CombineDocumentsChain`** are used for interacting with indexes. The purpose these chains is to combine your own data (stored in the indexes) with LLMs; [guide](https://docs.langchain.com/docs/components/chains/index_related_chains)
- **`StuffDocumentsChain`** is the simplest method, whereby you simply stuff all the related data into the prompt as context to pass to the language model; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/combine_documents/stuff.py)
- pros: only makes one single call to the LLM
- cons: restricted by the context length of the LLM
- **`MapReduceDocumentsChain`** runs an initial prompt on each chunk of data, then a different prompt to combine all the initial outputs; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/combine_documents/map_reduce.py)
- pros: can scale to larger and more documents; calls can be parallelized
- cons: requires many more calls to the LLM; loses some information during the final combined call
- **`MapRerankDocumentsChain`** does the same thing as `map_reduce` chain, but keeps a score for how certain it is in its answer and returns the output with the highest score instead of combining them; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/combine_documents/map_rerank.py)
- pros: similar to `map_reduce` chain but requires fewer calls
- cons: cannot combine information between documents; most useful when you expect there to be a single simple answer in a single document
- **`RefineDocumentsChain`** runs an initial prompt on the first chunk of data and generates some output, then refines the output as each new chunk of data is fed; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/combine_documents/refine.py)
- pros: allows for more relevant context; less lossy than `map_reduce` chain
- cons: calls to the LLM are codependent, hence cannot be paralleled like `map_reduce`; require some potential ordering of the documents
- **`load_summarize_chain`** summarizes a list of documents using one of the above `CombineDocumentsChain`；[code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/summarize/__init__.py) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/summarize.html)
- **Conversational Chains** (with memory)
- **`ConversationChain`** loads context from memory during conversation; essentially an `LLMChain` with memory capacity; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.ConversationChain), [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/conversation/base.py) and [examples](https://python.langchain.com/en/latest/modules/chains/getting_started.html#add-memory-to-chains)
- **`ConversationalRetrievalChain`** enables memory capability with `ConversationBufferMemory`; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.ConversationalRetrievalChain) , [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/conversational_retrieval/base.py) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/chat_vector_db.html)
- **`ChatVectorDBChain`** chats with a vector db and an llm; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.ChatVectorDBChain) and [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/conversational_retrieval/base.py)
- **Retrieval Chains** (over an index)
- **`RetrievalQAChain`** answers queries over an index; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.RetrievalQA) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html)
- **`RetreivalQAWithSourcesChain`** answer queries with sources over an index; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.RetrievalQAWithSourcesChain) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa_with_sources.html)
- **Question Answering Chains** (Q&As over a list of documents)
- **`QAGenerationChain`** answers queries over a list of documents; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.QAGenerationChain) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/question_answering.html)
- **`QAWithSourcesChain`** answers queries with sources over a list of documents; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.QAWithSourcesChain) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/qa_with_sources.html)
- **Router chains**
- **`RouterChain`** and **`MultiRouteChain`** dynamically selects the next chain to use for a given input; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/router/base.py) and [guides](https://python.langchain.com/en/latest/modules/chains/generic/router.html?highlight=RouterChain#router-chains)
- **`LLMRouterChain`** uses LLM to route between options; implements `RouterChain`; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/router/llm_router.py) and [guides]((https://python.langchain.com/en/latest/modules/chains/generic/router.html#llmrouterchain))
- **`EmbeddingRouterChain`** conducts similarity search on embeddings to route between options;implements `RouterChain`;[code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/router/embedding_router.py) and [guides]((https://python.langchain.com/en/latest/modules/chains/generic/router.html#embeddingrouterchain))
- **`MultiPromptChain`** selects the prompt which is most relevant for a given question and then use it to answer the question; implements `MultiRouteChain`; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/router/multi_prompt.py) and [guides](https://python.langchain.com/en/latest/modules/chains/examples/multi_prompt_router.html)
- **`MultiRetrievalQAChain`** selects the retrieval QA chain which is most relevant for a given question and then use it to answer the question; implements `MultiRouteChain`; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/router/multi_retrieval_qa.py) and [guides](https://python.langchain.com/en/latest/modules/chains/examples/multi_retrieval_qa_router.html)
- **Reasoning Chains**
- **`FlareChain`** (Forward-Looking Active REtrieval augmented generation) retrieves relevant documents and use them to improve answers if `max_generation_len` is reached or `min_prob` (uncertainty) is breached; see [guides](https://python.langchain.com/en/latest/modules/chains/examples/flare.html)， [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.FlareChain) and the [original repo](https://github.com/jzbjyb/FLARE/tree/main) for more
- **`PALChain`** implements Program-Aided Language Model that aids logical and arithmetic reasoning; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/pal/base.py), [examples](https://python.langchain.com/en/latest/modules/chains/examples/pal.html) and [paper](https://arxiv.org/pdf/2211.10435.pdf)
- **API Chains**
- **`APIChain`** interacts with APIs to retreieve relevant information based on an LLM, an API doc and a query question; see [examples](https://python.langchain.com/en/latest/modules/chains/examples/api.html) for how it's used and [code](https://github.com/hwchase17/langchain/tree/master/langchain/chains/api) for how to write custom API documentation for llms; has **great potential** for development
- **`OpenAPIEndpointChain`** calls an API endpoint and gets back a response in natural language; needs a descriptive API schema to work; internally uses `APIRequesterChain` and `APIResponderChain`; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/api/openapi/chain.py), [exmaples](https://python.langchain.com/en/latest/modules/chains/examples/openapi.html) and [discussion](https://github.com/hwchase17/langchain/pull/2974)
- **Other Chains**
- **`SequentialChain`** calls a series of component chains in deterministic order; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.SequentialChain) and [guides](https://python.langchain.com/en/latest/modules/chains/generic/sequential_chains.html)
- **`TransformChain`** transforms input_variables into output_variables by a self defined transform_func; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.TransformChain) and [guides](https://python.langchain.com/en/latest/modules/chains/generic/transformation.html)
- **`AnalyzeDocumentChain`** takes in a single document, splits it up, and then runs it through a `CombineDocumentsChain`; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.AnalyzeDocumentChain), [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/combine_documents/base.py) and [guides](https://python.langchain.com/en/latest/modules/chains/index_examples/analyze_document.html)
- **`ConstitutionalChain`** applies a set of rules to the output of an existing chain to guard against unexpected behaviors; can use default legal and ethical principles or customize your own; [docs](https://python.langchain.com/en/latest/reference/modules/chains.html#langchain.chains.ConstitutionalChain) and [examples](https://python.langchain.com/en/latest/modules/chains/examples/constitutional_chain.html)
- **`SQLDatabaseChain`** uses SQLAlchemy to connect to SQL db and answer queries over the db; [code](https://github.com/hwchase17/langchain/blob/master/langchain/chains/sql_database/base.py) and [examples](https://python.langchain.com/en/latest/modules/chains/examples/sqlite.html)
- **`QAEvalChain`** for question answering benchmarking; [guide](https://python.langchain.com/en/latest/use_cases/evaluation/question_answering.html) for simple Q&As and [guide](https://python.langchain.com/en/latest/use_cases/evaluation/data_augmented_question_answering.html) for data augmented Q&As; [code](https://github.com/hwchase17/langchain/blob/master/langchain/evaluation/qa/eval_chain.py#L12) and more [examples](https://python.langchain.com/en/latest/use_cases/evaluation.html#the-examples)
- **Usage of Chains**
- [generate text over vectorstore](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_text_generation.html) by retreiving the k most relevant documents for context
- [customize your own chain](https://python.langchain.com/en/latest/modules/chains/generic/custom_chain.html) by implementing the `Chain` class with`input_keys`, `output_keys` and `_call`; [docs](https://github.com/hwchase17/langchain/blob/master/langchain/chains/base.py#L28), [guides] and [examples](https://python.langchain.com/en/latest/modules/chains/getting_started.html#create-a-custom-chain-with-the-chain-class)
- [*provide async support*](https://python.langchain.com/en/latest/modules/chains/generic/async_chain.html) for `LLMChain`, `ChatVectorDBChain` and QA chains
- [*references*](https://python.langchain.com/en/latest/reference/modules/chains.html) for documentation on different types of chains

### Tools
- [List of Tools](https://python.langchain.com/en/latest/modules/agents/tools/getting_started.html)
- Components of a Tool
- `name` (str), is required and must be unique within a set of tools provided to an agent
- `description` (str), is optional but recommended, as it is used by an agent to determine tool use
- `return_direct` (bool), defaults to False
- `args_schema` (Pydantic BaseModel), is optional but recommended, can be used to provide more information (e.g., few-shot examples) or validation for expected parameters.
- `func` (Callable), inputs a str and ouputs a str
- Ways to define a tool
- using the `Tool` dataclass (str in str out); see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#tool-dataclass)
- subclassing the `BaseTool` class (more control); see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#subclassing-the-basetool-class)
- using the `tool` decorator; see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#using-the-tool-decorator)
- **custom Structureds Tools** by explicitly defining the parameter for `StructuredTool.from_function()`; see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#structuredtool-dataclass) for more
- Ways to use a tool
- **modify existing tools** by assigning new values to the relevant field
- **define priorities** among tools by adding explicit statements in the `description` field; see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#defining-the-priorities-among-tools) for more
- **return directly** from tools by setting `return_direct=True` when instantiating the `Tool` dataclass; see [examples](https://python.langchain.com/en/latest/modules/agents/tools/custom_tools.html#using-tools-to-return-directly) for more
- **feed multiple inputs** to tools by specifying how to parse multiple inputs in the `description` field or defining multiple inputs in the parameter for `.from_function()`; see [examples](https://python.langchain.com/en/latest/modules/agents/tools/multi_input_tool.html) for more
- **restrict arguments** be defining Tool input schema; see [guides](https://python.langchain.com/en/latest/modules/agents/tools/tool_input_validation.html) for more

### Agents
- Base Agent Types
- **`react-docstore`** uses the ReAct framework to interact with a docstore; requires a `Search` tool that searches for a document and a `Lookup` tool that looks up a term in the most recently found document; [paper](https://arxiv.org/pdf/2210.03629.pdf) and [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/react.html)
- **`self-ask-with-search`** utilizes a single tool that should be named `Intermediate Answer` and be able to lookup factual answers to questions; [paper]((https://ofir.io/self-ask.pdf)) and [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/self_ask_with_search.html)
- **`conversational-react-description`** uses the ReAct framework to determine which tool to use, and uses memory to remember the previous conversation interactions; is used specifically for [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/conversational_agent.html)
- **`zero-shot-react-description`** uses the ReAct framework to determine which tool to use based solely on the tool’s description; requires description to be provided for each tool; is used specifically for `MRKL` agents; [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/mrkl.html)
- Chat Agent Types (optimized for chat models)
- **`chat-zero-shot-react-description`**; [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/mrkl_chat.html)
- **`chat-conversational-react-description`** [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/chat_conversation_agent.html)
- **`structured-chat-zero-shot-react-description`**; [guides](https://python.langchain.com/en/latest/modules/agents/agents/examples/structured_chat.html)
- How to create agents
- loads an agent executor given tools and LLM with `initialize_agent`; [code](https://github.com/hwchase17/langchain/blob/master/langchain/agents/initialize.py#L12) and [docs](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.initialize_agent)
- [Custom Agent](https://python.langchain.com/en/latest/modules/agents/agents/custom_agent.html) by subclassing **`BaseSingleActionAgent`**; [code](https://github.com/hwchase17/langchain/blob/master/langchain/agents/agent.py#L45) and [docs](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.BaseSingleActionAgent)
- [Custom LLM Agent](https://python.langchain.com/en/latest/modules/agents/agents/custom_llm_agent.html) by subclassing **`LLMSingleActionAgent`**; [code](https://github.com/hwchase17/langchain/blob/master/langchain/agents/agent.py#L307) and [docs](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.LLMSingleActionAgent)
- [Custom LLM Agent (with a ChatModel)](https://python.langchain.com/en/latest/modules/agents/agents/custom_llm_chat_agent.html)
- [Custom MRKL Agent](https://python.langchain.com/en/latest/modules/agents/agents/custom_mrkl_agent.html) (Modular Reasoning, Knowledge and Language, pronounced "miracle") is a neuro-symbolic architecture that combine LLMs and external tools to solve complex problems by subclassing **`ZeroShotAgent`**; [code](https://github.com/hwchase17/langchain/blob/master/langchain/agents/mrkl/base.py#L35) and [docs](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.ZeroShotAgent)
- [Custom MultiAction Agent](https://python.langchain.com/en/latest/modules/agents/agents/custom_multi_action_agent.html) by subclassing **`BaseMultiActionAgent`**; [code](https://github.com/hwchase17/langchain/blob/master/langchain/agents/agent.py#L181) and [docs](https://python.langchain.com/en/latest/reference/modules/agents.html#langchain.agents.BaseMultiActionAgent)
- [Custom Agent with Tool Retrieval](https://python.langchain.com/en/latest/modules/agents/agents/custom_agent_with_tool_retrieval.html) by defining a retriever that filters the tools and prompts the LLM to select the right tool
- [Custom Agent with Plugin Retrieval](https://python.langchain.com/en/latest/use_cases/agents/custom_agent_with_plugin_retrieval.html) by combining tool retrieval and plugin chains
- Other Agents
- [plugnplai agent](https://python.langchain.com/en/latest/use_cases/agents/custom_agent_with_plugin_retrieval_using_plugnplai.html) with APIs from [Plug and Plai](https://www.plugnplai.com)
- [wikibase agent](https://python.langchain.com/en/latest/use_cases/agents/wikibase_agent.html) with data from [wikidata](http://wikidata.org)
- [multi-modal output agent](https://python.langchain.com/en/latest/use_cases/agents/multi_modal_output_agent.html) with image and text outputs
- **Agent Executors** take an agent and tools and use the agent to decide which tools to call and in what order
- configuring `initialize_agent`
- [handling parsing errors](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/handle_parsing_errors.html) with `handle_parsing_errors=True` or custom error functions
- [accessing intermediate steps](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/intermediate_steps.html) with `return_intermediate_steps=True`
- [capping the max number of iterations](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/max_iterations.html) with `max_iterations=n`
- [capping the timeout for an agent](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/max_time_limit.html) with `max_execution_time=n` (seconds)
- [interacting with data in vectorstore in an gentic manner](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/agent_vectorstore.html)
- use vectorstore as retriever in`RetrievalQA` chain
- use `RetrievalQA` chain as a tool in an agent_chain
- easy to answer multi-hop questions that depend on multiple vectorstores
- [using async API for agents](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/async_agent.html)
- [creating chatgpt clone](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/chatgpt_clone.html)
- [adding shared memory to an agent and its tools](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/sharedmemory_for_tools.html) by giving tools a `ReadOnlySharedMemory` wrapper of their agent's memory
- **Plan and execute Agent**
- accomplishes an objective by first planning what to do, then executing the sub tasks
- is inspired by [BabyAGI](https://github.com/yoheinakajima/babyagi) and then [Plan-and-Solve](https://arxiv.org/abs/2305.04091)
- see [guides](https://python.langchain.com/en/latest/modules/agents/plan_and_execute.html) for more
- Ongoing development
- implementing **reflexion**, see [discussion](https://github.com/hwchase17/langchain/pull/4737) and original [paper](https://arxiv.org/pdf/2303.11366.pdf) for more
- implementing **RCI** (Recursively Criticizes and Improves) agent, see [discussion](https://github.com/hwchase17/langchain/issues/2646) and original [paper](https://arxiv.org/pdf/2303.17491.pdf) with the following prompts:
> "Review your previous answer and find problems with your answer"
>
> "Based on the problems you found, improve your answer."
- check out [examples](https://python.langchain.com/en/latest/use_cases/personal_assistants.html#examples) and [Toolkits](https://python.langchain.com/en/latest/modules/agents/toolkits.html) for agents applied to particular use cases, i.e.

### Callbacks
- Types of callbacks
- Constructor callbacks
- defined in the constructor, eg. `LLMChain(callbacks=[handler])`
- used for all calls made on that object and scoped to that object only
- useful for logging and monitoring the entire chain
- Request callbacks
- defined in the `call()/run()/apply()` methods used for issuing a request, eg. `chain.call(inputs, callbacks=[handler])`
- used for that specific request only, and all sub-requests that it contains
- useful for streaming the output of a single request
- Existing callback handlers
- `StdOutCallbackHandler` logs all events to `stdout`; invoked by default when `verbose=True`
- check out more in [`langchain/callbacks`](https://github.com/hwchase17/langchain/tree/master/langchain/callbacks)
- creating a [custom handler](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#creating-a-custom-handler)
- creating an [async handler](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#async-callbacks)
- passing [multiple handlers](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#using-multiple-handlers-passing-in-handlers) in constructor callbacks and request callbacks
- Tracing and token counting
- setting the environment variable with `os.environ["LANGCHAIN_TRACING"] = "true"`
- using a context manager `with tracing_enabled()` to trace a particular block of code
- see more on [tracing](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#tracing)
- see more on [token counting](https://python.langchain.com/en/latest/modules/callbacks/getting_started.html#token-counting)

## Use Cases
### Autonomous Agents
- Concept
- executes a series of steps towards one or multiple gaols
- involves multi-step planning and reasoning
- check out more from chain-of-thought [papers](https://hackmd.io/@james-310110/HyEoTQ0Nn#Chain-of-thought) for more
- [Baby AGI with Tools](https://python.langchain.com/en/latest/use_cases/autonomous_agents/baby_agi_with_agent.html), [original repo](https://github.com/yoheinakajima/babyagi)
- pulls the first task from the task list.
- sends the task to the execution agent, which uses OpenAI's API to complete the task based on the context.
- enriches the result and stores it in Chroma/Weaviate.
- creates new tasks and reprioritizes the task list based on the objective and the result of the previous task.
- [AutoGPT](https://python.langchain.com/en/latest/use_cases/autonomous_agents/autogpt.html) and [WebSearch Research Assisstant](https://python.langchain.com/en/latest/use_cases/autonomous_agents/marathon_times.html), [original repo](https://github.com/Significant-Gravitas/Auto-GPT)
- determines what actions to take based on constraints, commands, resources, performance evaluation and response format
- uses a search tool, a write-file tool, a read-file tool and other tools such as web browsing tools, csv interaction tools, etc.
- [Meta-Prompt](https://python.langchain.com/en/latest/use_cases/autonomous_agents/meta_prompt.html), [original repo](https://github.com/ngoodman/metaprompt)
- builds self-improving agents by prompting the agent to reflect on its own performance and modify its own instructions
- not sure how it's better than the reflexion technique

### Agent Simulations
- Dialogue Agents
- [Camel](https://python.langchain.com/en/latest/use_cases/agent_simulations/camel_role_playing.html) implements the *Communicative Agents for “Mind” Exploration of Large Scale Language Model Society* [paper](https://arxiv.org/abs/2303.17760), where 2 agents interact with each other in a loop using formatted prompts and messages
- [Agent Debates with Tools](https://python.langchain.com/en/latest/use_cases/agent_simulations/two_agent_debate_tools.html) enables agents to use tools to inform their responses
- Speaker Selection
- [Decentralized Speaker Selection](https://python.langchain.com/en/latest/use_cases/agent_simulations/multiagent_bidding.html) allows agents to decide for themselves who speaks by bidding
- [Authoritarian Speaker Selection](https://python.langchain.com/en/latest/use_cases/agent_simulations/multiagent_authoritarian.html) where a privileged agent directs who speaks when
- Custom Environments
- two-agent env using Gymnasium, [examples](https://python.langchain.com/en/latest/use_cases/agent_simulations/gymnasium.html) and [docs](https://gymnasium.farama.org/api/env/)
- multi-agent env using PettingZoo, [examples](https://python.langchain.com/en/latest/use_cases/agent_simulations/petting_zoo.html) and [docs](https://pettingzoo.farama.org/content/environment_creation/)
- Dungeons & Dragons, [2-player game](https://python.langchain.com/en/latest/use_cases/agent_simulations/two_player_dnd.html) and [multi-player game](https://python.langchain.com/en/latest/use_cases/agent_simulations/multi_player_dnd.html)
- ==[Generative Agents](https://python.langchain.com/en/latest/use_cases/agent_simulations/characters.html) (Sims with GPT)==, [paper](https://arxiv.org/pdf/2304.03442.pdf)

### Code Comprehension
- concept
- answers questions in the context of an entire GitHub repository
- leverages [vectorstore](#Indexes), [`ConversationalRetrievalChain`](#Chains), and [LLM](#Models)
- workflow
1. ingest data
1. clone the target repository
2. load all files as documents using `TextLoader`
3. split files into chunks using `CharacterTextSplitter`
4. embed chunks using `OpenAIEmbeddings`
5. store the embedding in vectorstore using `DeepLake`
2. create chain
1. fetch the retriever from vectorstore using `db.as_retriever()`
2. build `ConversationalRetrievalChain`
3. answer questions
1. prepare questions with prompt formatter
2. send formatted prompt to the chain
- examples
- [analysis of twitter algorithm with Deep Lake](https://python.langchain.com/en/latest/use_cases/code/twitter-the-algorithm-analysis-deeplake.html)
- [analysis of langchain codebase with Deep Lake](https://python.langchain.com/en/latest/use_cases/code/code-analysis-deeplake.html)

### Other use cases
- [Question Answering over Docs](https://python.langchain.com/en/latest/use_cases/question_answering.html)
- [Chatbots](https://python.langchain.com/en/latest/use_cases/chatbots.html)
- [Querying Tabular Data](https://python.langchain.com/en/latest/use_cases/tabular.html)
- [Interacting with APIs](https://python.langchain.com/en/latest/use_cases/apis.html)

## Resources
- [Integrations](https://python.langchain.com/en/latest/integrations.html)
- [Deployments](https://python.langchain.com/en/latest/ecosystem/deployments.html)
- [Dependents](https://python.langchain.com/en/latest/dependents.html)
- [Projects](https://github.com/kyrolabs/awesome-langchain)
- [Youtube](https://python.langchain.com/en/latest/additional_resources/youtube.html)
- [Discord](https://discord.com/invite/6adMQxSpJS)