1122final4j - HackMD

# Human Preference Based Multi Language Model ## Demands and Motivation People have different preferences for different language models. Based on the selection algorithm, we have created a platform that can provide models that are more in line with user preferences for dialogue. ## Development Environment - Language: Java 17 (Spring Boot Framework) - Database: Firestore (Firebase) - Language Model: GPT-3 (LangChain4j), Gemini-pro (GCP), Cohere - Web: HTML, CSS, JavaScript - Deployment: Maven, Docker, Artifact Registry (GCP), CloudRun (GCP) - Version managment: Git, GitHub ## Development Operating Procedure 1. Demands List 2. Pull Request 3. Code Review ## Software Architecture - Clean Architecture - Entity: User domain - Repository: DB, User repository - UseCase: LM UseCase, User UseCase - Presentation: MVC (Web) - MVC - Controller: User Controller, Web Controller ## Design Pattern - SOLID Principle - Clean Code: Modularization, Low Coupling - Singleton Pattern - Simple Factory Pattern ## LM Selection Algorithm - Segment query in random - Offset the weight of all models when one of models less than 0 ## Presentation Layer - Restful API - Web API to notify controller - Use authenticated @RequestBody to improve security ## Bottlenecks ### Previous Plan - Too optimistic about development progress - Overestimating the original question ### Current plan - Each language models were not so easy to connect - Assigning tasks to members is not simple ## Outcome ![截圖 2024-06-04 下午5.12.46](https://hackmd.io/_uploads/By529Un4C.png) ## Future Plan - Complete the deployment - Support more language models, e.g. Llama, HuggingFace - Add cache with Redis ## Contributions - 蘇東毅 - Architecture - App deployment - Entity - Repository - Data Storage - LM UseCase - LM Selection Algorithm - User UseCase - 林彥丞 - Web View - Web Controller - User Controller - Entity - 胡智涵 - LM UseCase(因為不會接被取代了) - LM Selection Algorithm(因為寫太爛被取代了) --- - Project: Voxel Game Engine - [專題簡介與說明](https://drive.google.com/file/d/1w2W77wWWWq_VItWChmKtGGCWteoKECpN/view?usp=sharing)   ```mermaid gantt dateFormat YYYY-MM-DD title Java Final Project 時間排程(體素引擎) axisFormat %m-%d 計畫書繳交: 2024-04, 2024-04-17 Voxel Render System: 2024-04-15, 2024-05-31 User Story: 2024-04-17, 2024-04-21 Be familiar with LWJGL: 2024-04-15, 2024-04-30 Terrain Design: 2024-04-15, 2024-05-10 Physical Engine: 2024-04-15, 2024-05-10 Chunk System: 2024-04-15, 2024-05-10 Game Object System: 2024-04-15, 2024-05-10 Coordinate System: 2024-04-20, 2024-05-16 Collision: 2024-04-20, 2024-05-16 Shader System: 2024-04-20, 2024-05-16 Integration & Optimization: 2024-05-10, 2024-06-01 Adjust & Optimization: 2024-05-20, 2024-06-05 Demo: 2024-05-31, 2024-06-05 ``` ## Tech/Tools Stack ### Model - LangChain4j - Deeplearning4j ### Backend - SpringBoot ### Other - GitHub ## 會議紀錄 ::: spoiler 4/8 ### Tech Stack - SpringBoot - LangChain4j - Deeplearning4j - Flutter ### 分工 - Model: 蘇東毅 - SpringBoot: 林彥丞、湖致函 - Flutter: 蘇東毅、林彥丞、湖致函 ### 結論 - LLM 主導 LLMs，或是 ranking - 目標是使用者在多次問答中，同一個模型重複錯誤的地方能被其他模型修正。 - DB: 使用者歷史紀錄、評分、偏好(prompt) - 前端: Flutter，有空的人去做 ::: ::: spoiler 4/9 ### Tech Stack - DB: SQLite - Tool: GitHub, Figma ::: :::spoiler 4/14 ### 新的提案 - 整個砍掉改新的題目 - 基於大型語言模型輔助駕駛並以視覺化模擬呈現 - LWJGL (OpenGL) ### 功能 - Game server - Gamer engine - AI logic ::: :::spoiler 4/15 ### 分工 - Game Integration: 胡致函、林彥丞 - Voxel Game engine: 胡致函 - Game server: 蘇東毅、林彥丞 - AI logic: 蘇東毅 ::: :::spoiler 4/16 ### 新的提案 - Voxel Game engine (基於LWJGL的方塊像素遊戲引擎) - Demo呈現 : minecraft like (landscape、physical engine) ### 分工 - Game Engine - Terrain (地形)：蘇東毅、林彥丞 - Optimization：胡智涵 - Culling (渲染優化) - Octree (空間優化) - Physical Engine：蘇東毅 - Shader Design：蘇東毅、林彥丞 - 材質反光 - Collision：胡智涵 - Axis-Aligned Bounding Box (碰撞檢測) - Systems - Coordinate System：胡智涵 - 方便處理Voxel座標跟Modify - Chunk System：胡智涵 - 切割成固定大小的Chunk儲存 - Game Object System (遊戲物件管理)：林彥丞 - Object File Management (儲存遊戲物件) - Format (統一物件格式) - Texture System (材質)：蘇東毅 - Voxel Render System：胡智涵、林彥丞 - Vertices 壓縮 (空間壓縮) ::: :::spoiler 4/20 ### Scrum - sprint: 10 day (Have a review meeting after each sprint) - Standup meeting: once a week ### Sprint 0: 4/21 ~ 4/30 1. Familiar with LWJGL 2. 平面地圖：蘇東毅、林彥丞、胡智涵 3. 可移動的第一人稱視角：蘇東毅、林彥丞、胡智涵 4. Project set up: ASD ### Maven Building - groupId: final4j.poateto - artifactId: ${NAME}Demo ```xml=  <groupId>final4j.poateto</groupId> <artifactId>AtetoDemo</artifactId> <version>1.0-SNAPSHOT</version> ``` ::: ## Project Abstract As a tool for developing Voxel games, we are going to develop a game engine with LWJGL (OpenGL). This game engine is able to develop and optimize drawing voxel. ### Terrain Generator Generate Voxel terrain efficiently. Due to the large land area, violently generating a large number of Voxels will be very inefficient. We need to find an algorithm to efficiently generate, save and render the terrain. ### Optimization Different from traditional OpenGL objects, a Voxel object will use more Verticies than a general modeling object, so it is necessary to optimize the rendering and storage methods of Verticies, such as Octree (to store connected Voxels together, you can put constants Optimized from 4 to 8/7), Occlusion Culling (after Voxels are stacked, only the surface Voxels need to be rendered). ### Physical Engine Simulate the physical movement of Voxel. ### Collision Since Voxels are small squares, you can use Axis-Aligned Bounding Box (AABB collision box detection) to achieve a good collision effect. However, too many collision boxes may require using Group slimiar to Octree to optimize. ### Voxel Renderer If you directly use the traditional method to render cubes one by one (passing in 8 endpoints and 36 indicies), it will use too much GPU bandwidth, so Voxel Renderer will use Bypass one vertices and one in order to render Voxel efficiently. The colors are then uniformly calculated by the Shader (plus 7 vectors to calculate the Verticies) and then rendered. ### Coordinate & Chunk System Storing the entire scene information in one file may cause the loading speed to be too slow. Therefore, cutting the entire scene into fixed-size Chunks and storing them dynamically can reduce RAM usage and rendering pressure. Coupling with the use of the coordinate system is convenient to mark the position of each Voxel Game Object and store it in Chunk information. ### Voxel Game Object Contains model information of Voxel objects, as well as coordinate positions, physical quantities, etc., in a unified format to facilitate game engine management. ### Texture Loader Unlike ordinary Texture drawing, a single Voxel will only have a single color. Therefore, a Texture doesn’t store a picture but the color information of an entire Voxel. ### Effect System Such as Particle System (particle special effects like flame burning, Voxel water flow), light and shadow changes (Drop Shadow on Voxel), reflection (Voxel material reflection, water surface reflection). ## User Interface Analysis ### Gameplay Demonstration - Rendering screen demonstration - Object interactice demonstration - Performence demonstration ## User Story ### STAR Principle The story should follow STAR principle: - S (Situation) - T (Task) - A (Action): The story should mention that user use this product to do something. - R (Result): The story should explain what problem user solve with this product. ### User Story - Example: Tom is a Developer, and he is asked to develop a 3D project in very short time, then he use Voxel Game Engine, the engine make him build voxel models in high performence. Finally, he success to develop the project by the deadline. ## Referece - [LWJQL (OpenGL)](https://lwjglgamedev.gitbooks.io/3d-game-development-with-lwjgl/content/) - [LangChain4j](https://github.com/langchain4j/langchain4j) - [DeepLearning4j (DL4J)](https://github.com/deeplearning4j/deeplearning4j) - [SpringBoot Initializr](https://github.com/spring-io/start.spring.io)  {%hackmd /@Ateto/Style %}