前言
Anything LLM是Mintplex開發的一款整合any LLM API的開源平台,不管是OpenAI、Azure AOAI、Gemini、Huggingface、Ollama等,都可以直接透過至個平台上做使用。這個平台也整合了各種向量資料庫(Chroma、Qdrant、Pinecone、Weaviate、Milvus…),甚至是內建PDF解析、網頁爬蟲、YouTube影片分析,都能在這個平台上做到(真心佛)。
Anything LLM的出現大大降低AI應用的門檻,甚至不用親自寫code,就能做RAG的功能。雖說Mintplex是很佛心來著,讓比較沒基礎或非本科生的人能直接將LLM的功能應用最大化,但對於以此技術維生的工程師們也是殘忍,一口氣讓LLM的基本技能徹底貶值。
不過,畢竟在AI世界鬥爭的總是那幾間大集團,對我們這些不需參與競爭的工程師來說,真的是一大福音,接下來會詳細記錄如何從零開始架起Anything LLM。
由於Anything LLM是透過API key 呼叫各個LLM,因此用不到GPU,所以即使沒有超強顯示卡,也能在本機上架起(真的是太棒了)!
Anything LLM提供多種方式啟動專案,而我選擇docker方式來執行。我的電腦為macbook 2015,因此會以我的環境做示範及記錄。
STEP 1 安裝docker
由於我的電腦並未安裝docker,所以第一步先來安裝docker。我是跟著菜鳥教程-MacOS Docker安裝直接透過Homebrew進行安裝,若沒有Homebrew這篇文章也有其他方法可以安裝。
執行結果如下圖所示,只要出現successfully installed就代表安裝成功囉!
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
可以透過docker --version
再次確認是否能使用docker指令,結果如下圖。
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
STEP 2 安裝Anything LLM
新建文件夹anythingllm和子目錄
進入anythingllm目錄
新建env.txtx檔
將環境設定寫入env.txt中
目前資料結構如下圖所示
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
修改data目錄權限
修改env.txt文件權限
安裝並運行AnythingLLM docker
NOTE:由於本機原本沒有anythingLLM,所以會先把這個docker pull下來,結果如下圖所示
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
透過docker ps -a
查看docker狀態
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
瀏覽器輸入localhost:3014即可開始
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
STEP 3 初始設定
選擇LLM模型,填寫URL、API KEY就能使用囉
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
選擇單人使用還是多人共用,多人使用時要建立管理者帳號
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
建立第一個WORKSPACE
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
進入WORKSPACE之後就是標準的LLM介面
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
STEP 4 介面及使用說明
上圖[1]為系統設定,在這可以調整LLM、Embedding模型和vector DB等
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
上圖[2]為WORKSPACE的細節設定
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
上圖[3]為文件管理功能,在這可以上傳文件如PDF、txt、提供網址,甚至是YouTube影片位址。AnythingLLM在這會自動擷取文字內容,並轉成向量,是不是超方便!!!
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
**接著按Move to Workspace
轉入,按Save and Embedding
向量化並儲存
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
記得把文件pin到Workspace
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
STEP 5 進行測試囉
透過Anything LLM就能輕鬆體驗到RAG的功能了,且有附上Citation,如下圖所示:
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
NOTE:LLM的回答源自於以下段落,且內容有改寫
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
但從上圖的範例可以發現除了回答不準確之外,Citation也並不是很精準。由於我有將多個文件pin到Workspace,但其中一個Citation是與問題完全不相關的文件,因次這讓我懷疑RAG的真實性,以及是否是Embedding造成的問題。
AnythingLLM 測試2
我拿如何申辦ESTA舉例,比較有/無加入ESTA相關文件時,LLM的回覆是如何?
-
無ESTA相關文件(圖1)
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
-
有ESTA相關文件(圖2)
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
-
正確答案(圖3)
Image Not Showing
Possible Reasons
- The image was uploaded to a note which you don't have access to
- The note which the image was originally uploaded to has been deleted
Learn More →
從上面三張圖可以發現,圖1回覆是錯的($14 processing fee),而圖2加入相關文件後,的確修改成了正確答案($4 processing fee and $17 authorization fee),這也證實LLM的確會參考我們附上的文件。從這個測試可以發現使用英文文件與問答時,LLM表現較好,因此可以猜測到可能與Embedding有關。不過這兩個測試都有相同問題:Citation錯誤,這就讓我懷疑了RAG的真實性。
結論
Anything LLM的釋出的確讓我們能夠在地端架起整合多個LLM的聊天平台,不但可以上傳文件、爬蟲分析,還支援多使用者的管理。姑且不論效果如何,對於想整合LLM的一般使用者,門檻確實大幅降低;相對地,對於開發者來說,門檻也高了不少。整體來說,能透過一個平台就能體驗到RAG、vector DB等多種功能,對於一般想體驗的使用者來說,CP值還是滿高的,推薦可以玩玩看!
參考資料
AnythingLLM
用AnythingLLM构建专属知识库
包山包海的殺手級 LLM 整合平台 - Anything LLM