###### tags: `DX Innovation Challenge` # <center> KiKi Product Documentation </center> --- ## 1.0 General Information ### 1.1 Purpose The purpose of this document is to provide information to the DX-Innovation judges to demonstrate the workflow, features and details of the technique used in the project. The document also includes the future development value of the project. ### 1.2 Scope This part of the document outlines the functional, performance, security and other parts of the KiKi service. The KiKi service will: - Record the voice in meetings, speeches and conversations among a group. - Analyze voice files according to user requirements, evaluate them according to a criteria, and provide feedback. - Have professional evaluator manually analyze voice files and provide evaluation and feedback. The scope of this work includes the initial idea of KiKi. References to future development considerations are included in this proposal for information purposes only. ### 1.3 Project Reference Key documents supporting this proposal are listed below as reference: - The design procedure and CVCA of KiKi on [Miro](https://miro.com/app/board/o9J_ly_4Q6E=/) - Presentation slides on [Google Document](https://docs.google.com/presentation/d/1vdEb6raiPahiwhYfIq6LQyw-gwZLpr6oWKL_Y_7SJGQ/edit) - Group work notes on [HackMD](https://hackmd.io/@eagles/H1SWNt7fF) - 背景とエビデンス on [HackMD](https://hackmd.io/0tu1wS88SkmvjaqG9f6j8w?both) - Japanese version of this document on [HackMD](https://hackmd.io/tUb_hdhvSd2Y5xx52NHtTQ) ## 2.0 Summary of the Ideal System When using the KiKi service, each user will be able to upload their recorded voice file onto the server for analysis. ![](https://i.imgur.com/zRsak1E.png) As the above diagram states, user's data would be sent onto a database for batch processing. Batch processing would filter, aggregate and other wise prepare the data for analysis. The data would be analyzed by AI and a human evaluator, and the user would receive their personalized evaluation and feedback. Real-time voice messages would be processed in the same way. ![](https://i.imgur.com/Ia5FOeO.png) ### 2.1 AI algorithms To convert the speech to text, the system will use an open-source project called [DeepSpeech](https://github.com/mozilla/DeepSpeech). After converting the speech to text, the text will be evaluated using AI or neural network. Related research: - [Computerized paper evaluation using neural network](https://ieeexplore.ieee.org/document/8286032) - [Academic Paper Evaluation Software](https://www.researchgate.net/publication/297988426_Academic_Paper_Evaluation_Software) - [Algorithm research of spoken English assessment based on fuzzy measure and speech recognition technology](https://www.inderscience.com/info/inarticle.php?artid=105631) ### 2.2 Proof of Concept ![](https://i.imgur.com/NJXC7NF.png) To demonstrate the feasibility, a simple PoC(Proof of Concept) is introduced here. The participants of PoC are: user representatives, business leaders, project leaders, technical architects, test engineers, business managers, etc. #### 2.2.1 Mock PoC (1-2 months) During this phase, the testers perform information gathering. The information gathered includes, but is not limited to, the company's requirements for the product. #### 2.2.2 PoC - Step 1 (1-3 months) In this phase, the developer develops the project based on the requirements gathered and the pre-plan. The development includes the writing of AI algorithms, the training and correction of AI using training samples, the writing of test sets, etc... #### 2.2.3 PoC - Step 2 (1-3 months) During this phase, testers and user representatives will test the system functionality as well as provide feedback. The developers will optimize and modify the system based on their feedback and the issues that arise during testing. #### 2.2.4 PoC - Step 3 (1-3 months) In this phase, the developer will summarize and evaluate this PoC and conduct planning for the next PoC. ## 3.0 Functional Requirements and Impacts ### 3.1 Summary of Functions KiKi Service requires a technology based solution for a wide data collection and analyzing system whose primary functions include: - Data Accessibility - User Accounts - Data Integrity - Data Analysis - Administration Panel - System Security #### 3.1.1 Functional Requirements In order to accomplish the above articulated need, KiKi Service requires a technology based solution for a wide data collection and analyzing system that includes the following base functionality: - Data Accessibility - User's data, evaluation and feedback is available via a searchable website（if the Group setting is set to public) - Group's and Users' Analysis and Evaluation details can be downloaded in PDF format - Profile data is available for direct system-to-system access via an HTTP API (read only) - System will track the date and time of last API data pull - User Accounts - User accounts are password protected - Password reminders and resets are handled by the website - Data Integrity - When the uploaded voice file is corrupted, the system will prompt the user to re-upload - Data Analysis - The following characteristics of the user's uploaded voice files are analyzed by the AI as well as by human analysts: - Tone of voice - Speed of speech - Loudness - Time takes for a person to state his/her statement - Use of filler words - Percentage of statements in the discussion - Proper word choosing - The AI and the human evaluator will give scores of the performances of each user based on a criteria that will be developed in the future. - The AI and the human evaluator will provide feedback and advice based on users' scores and performances. - Administration Panel - Administrators can create, edit, and disable user accounts - Administrators allow teams to create shared team folders - System Security - Public will not have access to user's data - API provides read-only data access - Consideration will be given to potentially hiding of email address from public view to avoid collection by spam senders - Voice data will only be available to the user who uploaded it ### 3.2 Summary of Impacts #### 3.2.1 KiKi Service's Organizational Impacts The KiKi Service will require an initial team of developers and testers. The project will require administrative, project management, and training duties. To support the ongoing needs of this service, the KiKi will require an oversight committee to oversee and manage the system and the users' data to ensure ongoing reliability and integrity of the system. #### 3.2.2 KiKi Service's Operational Impacts We anticipate an eventual time savings for people who need the evaluation results of others' conversational ability after more functions of AI be developed in the future. Users will interact with the system in real-time via the web and they will be expected to acquire and maintain a secure and reliable internet connection adequate to dacilitate their data be successfully uploaded to the system. All user information, guidance and FAQ's will be provided by the services' developers and maintained on the KiKi Service's webpage. #### 3.2.3 KiKi Service's Social Impacts The 協調能力偏差値, as the output of KiKi's evaluation, will be combined with the 偏差値 system that's currently be used in the country to provide a more complete evaluation of a student's ability. ## 4.0 Performance Requirements ### 4.1 Specific Performance Requirements #### 4.1.1 Accuracy and Validity The system will employ numerous data quality assurance techniques, including but not limited to: - Record data completeness requirements - Record sample voice data for each person in a meeting when there're more than two participants - Record basic information of the theme/topic of the conversation #### 4.1.2 Timing and Capacity The system is intended to be available online 24 hours per day, 365 days per year with the exception of scheduled and pre-notified system maintenance downtimes, if needed. Data will be available for use after being pending in quete for evaluation by the AI and a human evaluator. The developing team will ensure that system resources are adequate for timely response times and overall software functionality. The service provider will review ISP/hosting provider options and once the initial development, it will be tested on different devices including but not limited to: - PC (Windows, OsX, Linux) - Smartphone and tablets(Android, ios) #### 4.1.3 Failure Contigencies The system is non-critical. Temporary inaccessibility, even up to several days, will not create a substantial burden on any user. The host site for the system will be chosen so as to include data backup capabilities and protocols. A selected party will maintain a copy of the code on a secure network, which has daily backup protocols. It is expected that with the use of an IPS/hosting provider that downtime will be minimal or non-existent. ## 5.0 Additional System Requirements ### 5.1 System Description The proposed KiKi Service will consist of a web-based, centralized database where all users' data would be saved in. Generally, all users will provide direct input to the system and outputs(reports) will also be generated directly from the system. However, to ensure growth ability, flexibility is also required for both input and output modes. Participating testers will provide input (i.e., voice file of their meetings or conversations) and the service providers will provide system administration and support for report generation. ### 5.2 Systems Integration An additional desired functionality of the KiKi Service is to accept and analyse video files. ### 5.3 Customization and Flexibility The desirability for our customers/users to continually update and improve the system is a given. ### 5.4 Sustainability and Open Source Plans The system will be developed under the leadership of a desired party using industry standard web development tools and practices. The party will commit to develop the initial application as described in this document, and to provide support and development services up to 5 hours per month on an ongoing basis. The developing party may pursue a transfer of maintenance responsibilities at any time. The party using the KiKi Service will perform at least an annual review of how the system is working and whether responsibilities need to be shifted or changed in any way. The developing team may, as its decision, or as contracted by the party using the service in exchange for appropriate remuneration, provide additional support or development services beyond this commitment. Source code will be stored on GitHub or in another mutually agreed repository. ### 5.5 System Documentation There is anticipated need for an end-user guide which will be provided by the developing party. The developing party will provide such documentation as necessary or as requested by the party using the service for technical requirements, including but not necessarily limited to documentation of the API. ### 5.6 Rights to Code and Data / Data Ownership The primary developing party shall retain all right and ownership in the software product including but not limited to source code, including right to license the product (but not the data) to any third party. The developing party will grant to the party using the service a perpetual, worldwide, royalty-free, non-exclusive, non-transferable license to the software product and derivative works, without the right to sub-license, for the using party and its agents to use the software product for its own purposes. This shall include no more than one production instance at any time, with unlimited backup, development and test copies permitted to maintain, improve and test the software as necessary. The party using this service shall retain all right and ownership in its data. The primary developing party shall receive a perpetual, royalty-free license to the data strictly for the purposes of maintaining, improving and supporting service-using parties' installation of the software product.