# 為什麼會有這篇筆記
Kuwa GenAI OS從v0.2.0(2024/4/15發布)就開始安裝到目前最新版v0.3.2(2024/7/5發布)，不得不讚賞Kuwa GenAI OS團隊，一鍵就可在Windows上部署生成式AI GUI平台，並立即使用。
不過在不同硬體上，面對不同使用者需求，還需要額外調校。因為怕自己未來重新部署時，也忘記要調整什麼，所以就生出這一篇筆記。
如人飲水冷暖自知。看別人做事很容易，沒有實際自己摸索過，永遠不會知道固中道理。
Ｗindows可攜式版本安裝到：
* acer P3-131
* acer P259-MG-30ES
* Apple MacBook Pro(2020)
* 文書PC ASUSPRO M840MB
* 組裝PC i7-14700KF/Z790/GeForce RTX 4090
Docker v0.3.0安裝到：
* Apple MacBook Pro M3 Pro
Docker v0.3.2安裝到：
* Apple MacBook Pro(2020)
# Hello World
一鍵部署完，就可以跟模型說Hello，請他自我介紹驗明正身。
# 調校筆記
## proxy設定
如果Ｗindows可攜式版本在下載套件的過程有出現失敗、逾時等錯誤訊息，代表所處網路環境不穩或需要設定proxy代理伺服器才能對外連線。
若需使用Ollama或vLLM結合KUWA GenAI OS進行推論，則需設定no_proxy排除localhost和127.0.0.1/32。
v0.3.2以前於下面檔案第3列加入如下指令後，重新執行C:\kuwa\GenAI OS\windows\build.bat，則理論上可以正常下載相關套件。
C:\kuwa\GenAI OS\windows\src\variables.bat
```bash=3
set "PROXY=http://proxy.XXX.com.tw:3128"
set "HTTP_PROXY=http://proxy.XXX.com.tw:3128"
set "HTTPS_PROXY=http://proxy.XXX.com.tw:3128"
set "ALL_PROXY=http://proxy.XXX.com.tw:3128"
set "NO_PROXY=localhost,127.0.0.1/32,192.168.0.0/16"
set "proxy=http://proxy.XXX.com.tw:3128"
set "http_proxy=http://proxy.XXX.com.tw:3128"
set "https_proxy=http://proxy.XXX.com.tw:3128"
set "all_proxy=http://proxy.XXX.com.tw:3128"
set "no_proxy=localhost,127.0.0.1/32,192.168.0.0/16"
```
但若要使用Ollama或vLLM結合KUWA GenAI OS的RAG進行推論，則需使用以下git指令擷取程式碼並合併，合併後KUWA GenAI OS就會使用Windows的proxy設定和不使用代理伺服器的相關設定。就不用在上述的C:\kuwa\GenAI OS\windows\src\variables.bat加入proxy或no_proxy相關設定。
```bash
git cherry-pick 34099e8 39f6d93
```
## Log
C:\kuwa\GenAI OS\src\kernel\logs
C:\kuwa\GenAI OS\src\multi-chat\storage\logs
C:\kuwa\GenAI OS\windows\executors\taide\run.bat
executors下模型的run.bat加入參數`"--log" "debug"`
## Timeout秒數調整(120變成1200)
C:\kuwa\GenAI OS\windows\packages\php\php.ini
```php=409
max_execution_time = 1200
```
```php=883
default_socket_timeout = 1200
```
C:\kuwa\GenAI OS\windows\packages\nginx\conf\nginx.conf
```nginx=67
fastcgi_read_timeout 1200;
```
C:\kuwa\GenAI OS\src\executor\docqa\src\kuwa_llm_client.py
```python=76
async def chat_complete(self, auth_token: str = None, messages: list = [], timeout=1200, streaming=True):
```
C:\kuwa\GenAI OS\src\kernel\src\kuwa\kernel\routes\chat.py
```python=44
response = requests.post(dest[0], headers=headers, data=form, stream=True, timeout=1200)
```
## 重新install kernel
執行C:\kuwa\GenAI OS\windows\tool.bat，輸入cmd (會幫忙把環境變數設定好，這功能很重要，例如：本身沒裝git就可以拿到相關環境變數)
再執行指令:
```
cd ..\src\kernel
pip install .
```
## 限制IP
C:\kuwa\GenAI OS\windows\packages\nginx\conf\nginx.conf
```nginx=70
allow 192.168.0.0/16;
deny all;
```
C:\kuwa\GenAI OS\src\multi-chat\\.env
```php=52
ALLOWED_IPS=127.0.0.1/8,192.168.0.0/16#只能輸入網段
```
## 網站預設語言
C:\kuwa\GenAI OS\src\multi-chat\.env
```php=56
locale=zh_tw
fallback_locale=zh_tw
```
## 網站相關圖片
C:\kuwa\GenAI OS\windows\packages\nginx-1.24.0\html\images
C:\kuwa\GenAI OS\src\multi-chat\public\images
* 首頁
C:\kuwa\GenAI OS\src\multi-chat\resources\views\components\logo.blade.php
```php=5
<img class="w-[150px]" src="{{ asset('images/01-tpclogo-jpg.jpg') }}">
<!--</a>-->
<div class="flex flex-col ml-4 text-[50px]">
<span class="text-[60px] dark:text-green-300">這裡輸入你想要的網站名稱</span>
<span class="text-[72px]">生成式AI平台</span>
```
* 應用程式Logo
C:\kuwa\GenAI OS\src\multi-chat\resources\views\components\application-logo.blade.php
```php=2
<img src="{{ asset('images/01-tpclogo-jpg.jpg') }}" {{ $attributes }}>
```
* 首頁四分之一圖
C:\kuwa\GenAI OS\src\multi-chat\resources\views\components\welcome_body.blade.php
```php=88
<h2 class="text-xl font-semibold text-center text-gray-900 dark:text-white">開心使用地端生成式AI</h2>
<div id="chatroom"
class="flex-1 p-4 justify-center overflow-hidden flex flex-col scrollbar rounded-lg">
<div class="flex justify-center items-center">
<img class="w-auto" src="{{ asset('images/S__47554573.jpg') }}">
</div>
</div>
```
## 指定模型回答
DB_QA的executor後面加上`--model "taide"`指定TAIDE回答
## 置換Construct RAG的embedding-model
C:\kuwa\GenAI OS\src\toolchain\construct_vector_db.py
```python=24
def construct_db(
docs_path:str,
output_path:str,
chunk_size:int = 512,
chunk_overlap:int = 128,
embedding_model:str = 'thenlper/gte-base-zh'
):
```
改成
```python=24
def construct_db(
docs_path:str,
output_path:str,
chunk_size:int = 512,
chunk_overlap:int = 128,
embedding_model:str = 'intfloat/multilingual-e5-large'
):
```
```python=66
parser.add_argument("--embedding-model", help="the embedding model to use", type=str, default="thenlper/gte-base-zh")
```
改成
```python=66
parser.add_argument("--embedding-model", help="the embedding model to use", type=str, default="intfloat/multilingual-e5-large")
```
## RAG後的DB_QA回應的參考資料出現本機絕對路徑
C:\kuwa\GenAI OS\src\executor\docqa\src\docqa.py
```python=100
link = src if src.startswith("http") else pathlib.Path(src).as_uri()
result += f'{i+1}. [{title}]({link})\n\n```plaintext\n{content}\n```\n\n'
```
改成
```python=100
link = src if src.startswith("http") else pathlib.Path(src).name
result += f'{i+1}. ({link})\n\n```plaintext\n{content}\n```\n\n'
```
## 上傳檔案大小限制，假設開放200MB的錄音檔上傳。
v0.3.2已參數化至系統設定中。
C:\kuwa\GenAI OS\windows\packages\php\php.ini
```php=855
upload_max_filesize = 200M
```
```php=703
post_max_size = 250M
```
```php=435
memory_limit = 512M
;(若錄音檔超過512M則要動到此設定)
```
C:\kuwa\GenAI OS\src\multi-chat\app\Http\Controllers\RoomController.php
v0.3.1以前版本需手動修改程式碼。
```php=321
'file' => 'max:20480',
```
改成
```php=321
'file' => 'max:204800',
```
C:\kuwa\GenAI OS\windows\packages\nginx\conf\nginx.conf
```nginx=26
client_max_body_size 250m;
```
C:\kuwa\GenAI OS\src\multi-chat\resources\views\layouts\app.blade.php
v0.3.1以前版本需手動修改程式碼。
```php=43
20 * 1024 *
```
改成
```php=43
200 * 1024 *
```
## 上傳檔案儲存路徑
C:\kuwa\GenAI OS\src\multi-chat\public\storage\pdfs
## 開放多人使用TAIDE
1. 複製C:\kuwa\GenAI OS\windows\executors\chatgpt目錄並重新命名成給vllm用的，例如：taide_vllm。
2. 修改C:\kuwa\GenAI OS\windows\executors\taide_vllm\run.bat如下：
```bash
pushd C:\kuwa\GenAI OS\windows\executors\taide_vllm
call ..\..\src\variables.bat
popd
set EXECUTOR_ACCESS_CODE=taide8B-vllm
pushd ..\..\..\src\multi-chat
php artisan model:config "taide8B-vllm" "TAIDE" --image "..\..\windows\executors\taide_vllm\TAIDE.png"
popd
start /b "" "kuwa-executor" "chatgpt" "--access_code" "taide8B-vllm" --base_url "http://127.0.0.1:8000/v1" --model "taide/Llama3-TAIDE-LX-8B-Chat-Alpha1" "--system_prompt" "你是一個來自台灣的AI助理，你的名字是TAIDE，樂於以台灣人的立場幫助使用者，會用繁體中文回答問題。" --log debug "--no_override_api_key" --api_key dummy
```
3. 將C:\kuwa\GenAI OS\windows\executors\taide_vllm目錄複製多個，只要ACCESS_CODE相同，就可以在KUWA GenAI OS提供同時多人呼叫vLLM API。
4. 連線至http://127.0.0.1:9000/v1.0/worker/debug，就可以看到taide8B_vllm有6條線可以同時提供服務。
Ollama在KUWA GenAI OS則可使用C:\kuwa\GenAI OS\windows\executors\ollama\init.bat先進行設定，再將C:\kuwa\GenAI OS\windows\executors\ollama目錄複製多個，即可於KUWA GenAI OS提供同時多人呼叫Ollama API。
### Ollama
v0.2.0後支援Parallel requests及Multiple models。
查看目前安裝Ollama的版本：
```ollama -v```
從Hugging Face下載TAIDE官方*.gguf檔
https://huggingface.co/taide/Llama3-TAIDE-LX-8B-Chat-Alpha1-4bit
建Modelfile檔：
```
FROM ./taide-8b-a.3-q4_k_m.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"""
PARAMETER num_ctx 8192
PARAMETER num_keep 24
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
```
將*.gguf檔轉換成ollama能運作的模型：
```
ollama create Llama3-TAIDE-LX-8B-Chat-Alpha1-4bit -f Modelfile
```
列出ollama在地端能用的模型：
``ollama list``
用命令提示字元和模型聊天：
```
ollama run Llama3-TAIDE-LX-8B-Chat-Alpha1-4bit
```
跑API服務的方式：
``ollama serve``
看哪些模型被載入：
``ollama ps``
執行kuwa\GenAI OS\windows\executors下的init.bat
因為在公司需使用proxy才能上網，而localhost也被導到proxy，導致無法使用KUWA直接呼叫Ollama API。但在自己的Macbook pro環境搭配KUWA docker版使用起來還不錯。
### vLLM
```bash
docker run --runtime nvidia --gpus all \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HUGGING_FACE_HUB_TOKEN=<secret>" \
-p 127.0.0.1:8000:8000 \
--ipc=host \
vllm/vllm-openai:latest \
--model taide/Llama3-TAIDE-LX-8B-Chat-Alpha1 --dtype half
```
```json
{
"builder": {
"gc": {
"defaultKeepStorage": "20GB",
"enabled": true
}
},
"experimental": false,
"runtimes": {
"nvidia": {
"path": "/usr/bin/nvidia-container-runtime",
"runtimeArgs": []
}
}
}
```
## git指令
可從下面這個檔案裡
C:\kuwa\GenAI OS\windows\update.bat
抽絲剝繭，舉一反三成，但不保證成功：
```bash
git fetch
git status -uno
git stash
git pull
git stash list
git stash show
git stash pop
git stash clear
```
本地的版本若有改，在執行`git stash pop`時，理論上會自行標註並合併程式。可以在`git status -uno`時，看到那些檔案有異動，屆時可以打開原始碼，自行判斷要留github，還是本地端自己改的程式碼。
## v0.3.2升級v0.3.3後可能會出現的問題
~~如果大家都跟筆者一樣，喜歡沒事就執行C:\kuwa\GenAI OS\windows\update.bat，看看有沒有新功能上線的話，可能會在7月5日升級到v0.3.2，7月17日後更新到新的Commits。在更新完成後，~~
### 啟動KUWA GenAI OS登入後會出現nginx的502 Bad Gateway錯誤訊息。
* 解決方法如下：
複製
C:\kuwa\GenAI OS\windows\src\nginx.conf
取代
C:\kuwa\GenAI OS\windows\packages\nginx\conf\nginx.conf
再重啟KUWA GenAI OS即可解決502 Bad Gateway問題。
### 無法使用LLaMA3.1 8B Instruct和Gemma2 2B Instruct問題
* 檢查C:\kuwa\GenAI OS\windows\executors\llama3_1目錄下是否已下載完成llama3_1-8b-q4_k_m.gguf
* 檢查C:\kuwa\GenAI OS\windows\executors\llama3_1\run.bat裡的gguf是否與C:\kuwa\GenAI OS\windows\executors\llama3_1目錄下檔名一致。
* 用KUWA OS與LLaMA3.1 8B Instruct聊天，命令提示字元畫面出現如下訊息：
```
No machine for llama3.1-8b-instruct has founded, returning NOMACHINE code
```
* 查看C:\kuwa\GenAI OS\windows\logs\start.log，在gemma2有錯誤訊息：
```
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma2'
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "runpy.py", line 196, in _run_module_as_main
File "runpy.py", line 86, in _run_code
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\Scripts\kuwa-executor.exe\__main__.py", line 7, in <module>
sys.exit(main())
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\kuwa\executor\cli.py", line 93, in main
args, unknown_args = parser.parse_known_args( )
File "argparse.py", line 1866, in parse_known_args
File "argparse.py", line 2061, in _parse_known_args
File "argparse.py", line 2038, in consume_positionals
File "argparse.py", line 1943, in take_action
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\kuwa\executor\cli.py", line 84, in __call__
executor = executor_class()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\kuwa\executor\./example\llamacpp.py", line 147, in __init__
super().__init__()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\kuwa\executor\base_executor.py", line 58, in __init__
self.setup()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\kuwa\executor\./example\llamacpp.py", line 201, in setup
self.model = Llama(model_path=self.model_path, n_gpu_layers=self.args.ngl, n_ctx=self.context_window)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\llama_cpp\llama.py", line 341, in __init__
self._model = _LlamaModel(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\llama_cpp\_internals.py", line 57, in __init__
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: gemma-2-2b-it-Q8_0.gguf
```
* 查看C:\kuwa\GenAI OS\windows\logs\build.log在llama-cpp-python升級到0.2.87，有出現例外錯誤訊息：
```
Collecting llama-cpp-python==0.2.87,>=0.2.77 (from -r requirements.txt (line 12))
Using cached llama_cpp_python-0.2.87.tar.gz (80.5 MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\cli\base_command.py", line 105, in _run_wrapper
status = _inner_run()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\cli\base_command.py", line 96, in _inner_run
return self.run(options, args)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\cli\req_command.py", line 67, in wrapper
return func(self, options, args)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\commands\install.py", line 379, in run
requirement_set = resolver.resolve(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 397, in resolve
self._add_to_criteria(self.state.criteria, r, parent=None)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 156, in __bool__
return bool(self._sequence)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 174, in __bool__
return any(self)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 162, in <genexpr>
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 53, in _iter_built
candidate = func()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 186, in _make_candidate_from_link
base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 232, in _make_base_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 303, in __init__
super().__init__(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 158, in __init__
self.dist = self._prepare()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 235, in _prepare
dist = self._prepare_distribution()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 314, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\operations\prepare.py", line 527, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\operations\prepare.py", line 642, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\operations\prepare.py", line 72, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\distributions\sdist.py", line 56, in prepare_distribution_metadata
self._install_build_reqs(finder)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\distributions\sdist.py", line 126, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\distributions\sdist.py", line 103, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_internal\utils\misc.py", line 706, in get_requires_for_build_wheel
return super().get_requires_for_build_wheel(config_settings=cs)
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 166, in get_requires_for_build_wheel
return self._call_hook('get_requires_for_build_wheel', {
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\pyproject_hooks\_impl.py", line 321, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
File "C:\kuwa\GenAI OS\windows\packages\python-3.10.11-embed-amd64\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 77, in _build_backend
obj = import_module(mod_path)
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'scikit_build_core'
```
* 解決方法如下：
1. 執行C:\kuwa\GenAI OS\windows\tool.bat，輸入cmd (會幫忙把環境變數設定好，這功能很重要，例如：本身沒裝git就可以拿到相關環境變數)
2. 輸入指令``pip install scikit-build-core``
3. 輸入指令
```
pip install --upgrade https://github.com/abetlen/llama-cpp-python/releases/download/v0.2.87-cu123/llama_cpp_python-0.2.87-cp310-cp310-win_amd64.whl
```
4. KUWA OS測試聊天
# 參考資料
* Kuwa AI網站 https://kuwaai.tw/zh-Hant/os/intro
* Kuwa GenAI OS https://github.com/kuwaai/genai-os
* 陳伶志，「Kuwa AI 安裝過程筆記」https://hackmd.io/@cclljj/r1mIc3tNR
* 使用繁體中文評測各家 Embedding 模型的檢索能力 https://ihower.tw/blog/archives/12167
* YWC 科技筆記，抓別人的 GGUF 模型，用 Ollama 在本機執行！ https://ywctech.net/ml-ai/ollama-import-custom-gguf/
* TAIDE - 推動臺灣可信任生成式AI發展計畫 https://taide.tw/index
* https://github.com/NVIDIA/nvidia-container-toolkit/issues/150
# 致謝
* Kuwa GenAI OS開發團隊 胡詠翔先生回應我的各種問題。
* 公司同事處理NV4090 PC採購的行政流程。
Sign in with Wallet
Connect another wallet