Try   HackMD

前言

Anything LLM是Mintplex開發的一款整合any LLM API的開源平台,不管是OpenAI、Azure AOAI、Gemini、Huggingface、Ollama等,都可以直接透過至個平台上做使用。這個平台也整合了各種向量資料庫(Chroma、Qdrant、Pinecone、Weaviate、Milvus),甚至是內建PDF解析、網頁爬蟲、YouTube影片分析,都能在這個平台上做到(真心佛)。

Anything LLM的出現大大降低AI應用的門檻,甚至不用親自寫code,就能做RAG的功能。雖說Mintplex是很佛心來著,讓比較沒基礎或非本科生的人能直接將LLM的功能應用最大化,但對於以此技術維生的工程師們也是殘忍,一口氣讓LLM的基本技能徹底貶值。

不過,畢竟在AI世界鬥爭的總是那幾間大集團,對我們這些不需參與競爭的工程師來說,真的是一大福音,接下來會詳細記錄如何從零開始架起Anything LLM。

由於Anything LLM是透過API key 呼叫各個LLM,因此用不到GPU,所以即使沒有超強顯示卡,也能在本機上架起(真的是太棒了)!

Anything LLM提供多種方式啟動專案,而我選擇docker方式來執行。我的電腦為macbook 2015,因此會以我的環境做示範及記錄。

STEP 1 安裝docker

由於我的電腦並未安裝docker,所以第一步先來安裝docker。我是跟著菜鳥教程-MacOS Docker安裝直接透過Homebrew進行安裝,若沒有Homebrew這篇文章也有其他方法可以安裝。

brew install --cask --appdir=/Applications docker

執行結果如下圖所示,只要出現successfully installed就代表安裝成功囉!

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

可以透過docker --version再次確認是否能使用docker指令,結果如下圖。

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

STEP 2 安裝Anything LLM

新建文件夹anythingllm和子目錄

mkdir  ./volume1/docker/anythingllm/data

進入anythingllm目錄

cd ./volume1/docker/anythingllm

新建env.txtx檔

touch env.txt

將環境設定寫入env.txt中

SERVER_PORT=3001
STORAGE_DIR="/app/server/storage"
UID='1000'
GID='1000'

目前資料結構如下圖所示

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

修改data目錄權限

chmod -R 777 data

修改env.txt文件權限

chmod 777 env.txt

安裝並運行AnythingLLM docker

docker run -d \
   --restart unless-stopped \
   --cap-add SYS_ADMIN \
   --name anythingllm \
   -p 3014:3001 \
   -v $(pwd)/data:/app/server/storage \
   -v $(pwd)/env.txt:/app/server/.env \
   mintplexlabs/anythingllm

NOTE:由於本機原本沒有anythingLLM,所以會先把這個docker pull下來,結果如下圖所示

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

透過docker ps -a查看docker狀態

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

瀏覽器輸入localhost:3014即可開始

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

STEP 3 初始設定

選擇LLM模型,填寫URL、API KEY就能使用囉

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

選擇單人使用還是多人共用,多人使用時要建立管理者帳號

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

建立第一個WORKSPACE

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

進入WORKSPACE之後就是標準的LLM介面

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

STEP 4 介面及使用說明

上圖[1]為系統設定,在這可以調整LLM、Embedding模型和vector DB等

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

上圖[2]為WORKSPACE的細節設定

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

上圖[3]為文件管理功能,在這可以上傳文件如PDF、txt、提供網址,甚至是YouTube影片位址。AnythingLLM在這會自動擷取文字內容,並轉成向量,是不是超方便!!!

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

**接著按Move to Workspace轉入,按Save and Embedding向量化並儲存

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

記得把文件pin到Workspace

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

STEP 5 進行測試囉

透過Anything LLM就能輕鬆體驗到RAG的功能了,且有附上Citation,如下圖所示:

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

NOTE:LLM的回答源自於以下段落,且內容有改寫

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

但從上圖的範例可以發現除了回答不準確之外,Citation也並不是很精準。由於我有將多個文件pin到Workspace,但其中一個Citation是與問題完全不相關的文件,因次這讓我懷疑RAG的真實性,以及是否是Embedding造成的問題。

AnythingLLM 測試2

我拿如何申辦ESTA舉例,比較有/無加入ESTA相關文件時,LLM的回覆是如何?

  • 無ESTA相關文件(圖1)

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More →

  • 有ESTA相關文件(圖2)

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More →

  • 正確答案(圖3)

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More →

從上面三張圖可以發現,圖1回覆是錯的($14 processing fee),而圖2加入相關文件後,的確修改成了正確答案($4 processing fee and $17 authorization fee),這也證實LLM的確會參考我們附上的文件。從這個測試可以發現使用英文文件與問答時,LLM表現較好,因此可以猜測到可能與Embedding有關。不過這兩個測試都有相同問題:Citation錯誤,這就讓我懷疑了RAG的真實性。

結論

Anything LLM的釋出的確讓我們能夠在地端架起整合多個LLM的聊天平台,不但可以上傳文件、爬蟲分析,還支援多使用者的管理。姑且不論效果如何,對於想整合LLM的一般使用者,門檻確實大幅降低;相對地,對於開發者來說,門檻也高了不少。整體來說,能透過一個平台就能體驗到RAG、vector DB等多種功能,對於一般想體驗的使用者來說,CP值還是滿高的,推薦可以玩玩看!

參考資料

AnythingLLM
用AnythingLLM构建专属知识库
包山包海的殺手級 LLM 整合平台 - Anything LLM