[toc]
# CI/CD Pipleline Design & Deployment
## Choice of CI/CD framework
Over the past few years, **DevOps** have become a crucial part of the software life cycle. We can find a range of tools to support the CI/CD pipeline. Gitlab CI/CD, Jenkins and GitHub Actions outstandingly stand among them.
GitLab CI/CD is an all-in-one platform for software development that provides integrated tools for version control, project management, and CI/CD.
GitHub Actions is a CI/CD platform integrated with GitHub that allows developers to automate, test and deploy their workflow using containers.[https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions] In terms of ease of use, GitHub Actions is considered to be the easiest to use, as it has a user-friendly interface and requires less setup and configuration compared to GitLab CI/CD and Jenkins.
***Jenkins*** is a free, open-source, self-contained Java-based automation server that offers plugins to support building, deploying, and automating any project.[https://www.jenkins.io/]
When determine which to use, our group do not hesitate to choose GitLab CI/CD, due to its integration with other features of GitLab platform and GitLab is strong enough for various CI/CD requirement.
## Design of CI/CD pipeline
<center style="font-size:12px;">Figure 1: The Overall Design of CI/CD Pipeline</center><br/>
For the design of CI/CD pipeline, we will describe it in the following different stages:
1. Version control system(VCS) and code repository. Version control system will be used to track and manage changes to the source code and help with teamwork.
For the choice of VCS and code repository, we will use Git and Gitlab, which are currently the most popular and shcool provides easy access to use them.
2. Build. Build tools facilitate a wide variety of build automation tasks, including:
① Compiling: Compiling source code into machine code.
② Dependency management: Identifying and patching necessary third party integrations.
③ Automated tests: Executing tests and reporting failure.
Here we will choose ***Maven***, which is a popular open-source build tool developed by the Apache Group. ***Maven*** provides a centralized repository for managing project dependencies, making it easier to manage and resolve conflicts between different versions of libraries.[https://maven.apache.org/] Besides, it provides automating tasks such as testing and deployment which facilitates the CI/CD pipeline.
3. Static code analysis. Static code analysis, is a method of computer program debugging that is done by examining the code without executing the program. By using static code analysis, we can help out team find errors before we run the code and alert us about any other issues, such as a lack of inline documentation, bad coding standards, security issues, performance issues, and so on.
Here we choose ***SpotBugs*** to do static code analysis. ***SpotBugs*** is a open-source tool for static analysis of Java programs, it can configured with ***Maven*** build as plugin which is handy and easy to use.[https://spotbugs.github.io/]
4. Test. Testing is a critical component of the CI/CD pipeline as it helps ensure the quality and stability of the software being delivered. Testing helps catch and fix bugs, improve software functionality and performance, and reduces the risk of introducing new problems into production. By automating testing and integrating it into the CI/CD pipeline, our team can identify and resolve issues early in the development process, speeding up the delivery of new features and improvements, and increase confidence in the code being deployed.
Here we will choose ***TestNG*** as out test framework to execute automated test. The specific reson for use of them will be described in section Choice of Testing Framework.
5. Deploy. Once the code changes have successfully passed through the previous pipeline stages, they can be deployed. Our application will be deployed on two environments: test environment and production environment.
Testing in a separate environment before deploying to production is important because it provides a controlled and isolated environment to assess our application. This helps identify and resolve issues that may not have been detected in development or earlier stages of testing and and reduces the risk of impacting production systems and users with unanticipated issues.
Here we choose ***Docker*** image to do deployment. Docker Container deployment is a method for quickly building and releasing applications.[https://www.docker.com/] Docker containers can be easily moved between different environments, including local development, testing, and production, without any compatibility issues. Besides, Docker images can be updated and maintained easily, reducing the effort and time required for application updates and bug fixes.
6. Health check. This is the final stage of CI/CD pipeline, which is use Linux Curl to check the health status of our application. After Curl is called, the application returns current health status information so the CI/CD pipeline knows if the application is running successfully after deployment.
## Deployment of CI/CD pipeline
The actual deployment can be viewed at https://git.ecdf.ed.ac.uk/psd2223/Chang_Fisher/-/pipelines
<center style="font-size:12px;">Figure 2: The Gitlab CI/CD pipeline dashboard </center><br/>
### Gitlab CI/CD Runner
GitLab Runner is an application that works with GitLab CI/CD to run jobs in a pipeline.
To be able to use Gitlab CI/CD, our group rented a server from a cloud service provider to set up Gitlab Runner.
<center style="font-size:12px;">Figure 3: The Gitlab CI/CD Runner </center><br/>
### Gitlab CI/CD Variables
Here we configure some changeable and sensitive data in Gitlab CI/CD Variables, so that they can be safely stored and maintained.
<center style="font-size:12px;">Figure 4: The Gitlab CI/CD Variables</center><br/>
### Gitlab CI/CD Configuration
The following is the Gitlab CI/CD configuration file：
```
image: docker
stages:
- package # Source code packaging stage
- quality # static code analysis stage
- test # Unit Test and Static code analyze stage
- build_push # Docker Image Build and Push stage
- deploy # Deployment stage
- health_check # Health Check Stage
variables:
MAVEN_DIR: "./.m2/repository"
PROJECT: "Chang-Fisher"
cache:
paths:
- ./.m2/repository
- /cache
mvn_build_job: # job name
image: maven:3.6.2-jdk-14 # The build image used for this stage
stage: package # Associated stage name
script:
- mvn package -B -DskipTests -Dmaven.repo.local=$MAVEN_DIR # Execute the script
- echo $MAVEN_DIR
- pwd
- cp target/Chang_Fisher-1.0-SNAPSHOT.jar /cache
static code analysis:
stage: quality
image: maven:3.6.2-jdk-14
allow_failure: true
script:
- mvn package -B -Dmaven.repo.local=$MAVEN_DIR
- mvn spotbugs:check -Dmaven.repo.local=$MAVEN_DIR
unit_test_job:
image: maven:3.6.2-jdk-14 # The build image used for this stage
stage: test
script:
- mvn test -Dmaven.repo.local=$MAVEN_DIR
artifacts:
when: always
reports:
junit:
- target/surefire-reports/TEST-*.xml
build_push_job:
image: docker
stage: build_push
script:
- pwd
- cp /cache/Chang_Fisher-1.0-SNAPSHOT.jar Chang_Fisher-1.0-SNAPSHOT.jar
- docker build -t registry-vpc.cn-shanghai.aliyuncs.com/huangzihe/chang-fisher:latest .
- docker images |grep chang-fisher
- docker login --username=$DOCKER_USERNAME --password=$DOCKER_PASSWORD registry-vpc.cn-shanghai.aliyuncs.com
- docker push registry-vpc.cn-shanghai.aliyuncs.com/huangzihe/chang-fisher:latest
deploy_test_job:
image: ictu/sshpass
stage: deploy
only: [dev]
script:
- sshpass -p $SERVER_PASSWORD scp -r -o StrictHostKeyChecking=no deploy/deploy.sh root@$TEST_SERVER_IP:/root/GitHub_Action
- sshpass -p $SERVER_PASSWORD ssh root@$TEST_SERVER_IP 'cd /root/GitHub_Action;sh deploy.sh'
deploy_prod_job:
image: ictu/sshpass
stage: deploy
only: [master]
script:
- sshpass -p $SERVER_PASSWORD scp -r -o StrictHostKeyChecking=no deploy/deploy.sh root@$PROD_SERVER_IP:/root/GitHub_Action
- sshpass -p $SERVER_PASSWORD ssh root@$PROD_SERVER_IP 'cd /root/GitHub_Action;sh deploy.sh'
test_env_health_check_job:
image: alpine/curl
only: [dev]
variables:
GIT_STRATEGY: none
stage: health_check
script: curl 101.133.146.88:33827/actuator/health | grep "{\"status\":\"UP\"}" || ( echo Helth Check Failed Actuator=$ACTUATOR_HEALTH_URL && exit 2; )
prod_env_health_check_job:
image: alpine/curl
only: [master]
variables:
GIT_STRATEGY: none
stage: health_check
script: curl http://$PROD_SERVER_ID:33827/health | grep "{\"status\":\"UP\"}" || ( echo Helth Check Failed Actuator=$ACTUATOR_HEALTH_URL && exit 2; )
```
## Basic Test of CI/CD Pipeline
### Build Stage
In this stage, the main task is to compile the source code using ***Maven*** and put the generated jar packages into the Gitlab CI/CD cache to prepare for the subsequent Docker image push.
When we triggered the pipeline, we can read logs on Gitlab CI/CD website, and job was successfully done.
<center style="font-size:12px;">Figure 5: The Test of Build Stage</center><br/>
### Quality Stage
The static code analysis will be done by ***SpotBugs*** tool in this stage. To do basic test, we write a infinite recursive loop in the code on purpose.
```
public void bug() {
bug();
}
```
Then, similarly, we checked logs on Gitlab CI/CD website and found that the tool detected the error, printed the log and failed the task.
<center style="font-size:12px;">Figure 6: The Log of Quality Stage</center><br/>
### Test Stage
To do basic test in this stage with ***TestNG***, still, we wrote a test which can not pass on purpose, like:
```
@Test
public void test() {
Assert.assertEquals(2, 3);
}
```
Then, after the job of Test Stage finished, we can find that the whole pipeline was failed and test report was shown on the website.
<center style="font-size:12px;">Figure 7: The Test Report </center><br/>
<center style="font-size:12px;">Figure 8: The Test of Test Stage</center><br/>
### The whole pipeline
By fixing the above bugs and test that can not pass, we can execute all jobs in the CI/CD pipeline and find that the CI/CD pipeline is passed. And the execution time of pipeline is ideal, less than 2 minutes.
<center style="font-size:12px;">Figure 9: Successful execution of pipeline</center><br/>
# Test Plan and Prototype
Our test plan is divided into unit test, integration test and system test. Unit test only tests every single and separate function. Integration test involves multi-end connectivity. System test is the final functional test and stability test. All tests will be applied with static data. Finally, prepare the corresponding test data in the database.
### 1. Unit test
| UID | Case Title | Description | Priority | Test Step | Expected Result | Actual Result | Status | Failure Management | Failure Communication |
|-------|----------------------------------|-------------------------------------------------------------------------------------------------------|----------|---------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|---------------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------|
| FT001 | email address input | test the input box of email address | high | 1. input a correct email address 2. check the display | the input can be seen | the input can be seen | Pass | Check the frontend components first, and then check the corresponding backend functions | contact frontend developers |
| FT002 | password input | test the input box of password | high | 1. input a correct password 2. check the input | the input can be seen | the input can be seen | Pass | Check the frontend components first, and then check the corresponding backend functions | contact frontend developers |
| FT003 | password display | check the password is shown in mask | high | 1. input a correct password 2. check the display | password shows in mask ('\*\*\*') | password shows in mask ('\*\*\*') | Pass | Check the frontend components for displaying password | contact frontend developers |
| FT004 | email address identification | test whether the system can recognize true email addresses | high | 1. input an invalid email address (without '@') 2. input a valid email address (with '@' in-between strings) | 1. input 1 passes 2. input 2 rejects | both input 1 and 2 pass | Fail | Check the algorithms in frontend for examing a valid email address | contact frontend developers |
| FT005 | password length judgment | test whether the system can limit text between 6~20 characters | high | 1. input an invalid password (less than 6 or more than 20 charecters) 2. input a valid password (between 6~20 charecters) | 1. input 1 passes 2. input 3 rejects | 1. input 1 passes 2. input 3 rejects | Pass | Check the algorithms for calculating password length | contact frontend developers |
| FT006 | login status display | test whether the frontend can display user's login status | low | 1. use an example account to login | the account name (email address) is displayed at the top-right corner of main page | the account name (email address) is displayed at the top-right corner of main page | Pass | Check the frontend components first, and then check the corresponding backend funtions | contact frontend developers |
| FT007 | logout test | test the logout function | medium | 1. use an example account to login 2. logout this account | users need to re-login | users need to re-login | Pass | Check the frontend for problems, and then check the backend algorithms. | contact frontend developers |
| FT008 | search box length limitation | test whether the system can limit text within 20 words | high | 1. input nothing 2. input more than 20 words | 1. no response 2. show a red warning and do nothing | 1. no response 2. show a red warning and do nothing | Pass | Check the frontend algorithms for calculating the text length in search box | contact frontend developers |
| FT009 | accurate result match | test whether the system can make an accurate result matching (case sensitive) | high | 1. input example text 'Alan' | only matchs 'Alan' but not 'alan' or 'alana' | Match 'Alan' and 'alan' but not 'alana' | Fail | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT010 | fuzzy matching | test whether the system can make an fuzzy result matching (similar results permitted) | medium | 1. input example text 'Alan' | matchs 'Alan', 'alan' and 'alana' | under developing | Under developing | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT011 | publish year filtration | test whether the system can make filtrations by the publish year of papers | medium | 1. search a random keyword 2. use filter to make filtration by publishing years | results only from specified years | results only from specified years | Pass | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT012 | type filtration | test whether the system can make filtrations by the type of papers | low | 1. search a random keyword 2. use filter to make filtration by types | results only from specified types | under developing | Under developing | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT013 | topic filtration | test whether the system can make filtrations by the topic of papers | low | 1. search a random keyword 2. use filter to make filtration by topics | results only from specified topics | under developing | Under developing | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT014 | sort by name | test whether the system can sort results by papers' name | high | 1. search a random keyword 2. sort the results by names | results sorted from A to Z | results sorted from A to Z | Pass | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT015 | sort by correlation | test whether the system can sort results by correlations of papers | medium | 1. search a random keyword 2. sort the results by correlations | results sorted from the closest correlation to the furthest correlation | under developing | Under developing | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT016 | sort by citation | test whether the system can sort results by numbers of citations | medium | 1. search a random keyword 2. sort the results by numbers of citations | results sorted from the highest numbers of citations to the lowest ones | under developing | Under developing | Check the frontend components, check the corresponding backend functions and check the correctness of database language | contact frontend developers |
| FT017 | page response when no results | test whether the system can react out the right tips when no results found | high | 1. search an item not existing | a tip will be shown telling that results no found | a tip will be shown telling that results no found | Pass | Check the frontend components, check the corresponding backend functions | contact frontend developers |
| FT018 | history checking | test whether users can check the last 5 histories | high | 1. search some texts 2. click the search box | searching history is shown below the box | searching history is shown below the box | Pass | Check the frontend components, check the corresponding backend functions, check the correctness of database language, and check whether histories are correctly stored in database | contact frontend developers |
| FT019 | recommendation by history | test whether the system can generate recommended paper according to searching history | medium | 1. search some texts 2. click the recommendation button | other related papers are shown | under developing | Under developing | Check the backend functions, check the history storage systems, and check whether the NLP network is working properly | contact backend developers |
| FT020 | language choice | test whether the users can choose their preferred languages | low | 1. log into the system 2. choose another language | 1. default language is English 2. all words changed into the other language | Most texts changed into the other language, but some icons and images cannot | Fail | Check the corresponding frontend support components | contact frontend developers |
| FT021 | cookie acception | test whether the system can show a cookie alert | low | 1. new user logs into the system 2. logout 3. login again | a warning pop-window jumps out to remind users the cookies we use only at the first time | under developing | Under developing | Check the frontend components | contact frontend developers |
| FT022 | frontend request: login | test whether the system can send login request to backend | high | 1. send a login http request | send success | send success | Pass | Check the functions of corresponding frontend components | contact frontend developers |
| FT023 | frontend request: logout | test whether the system can send logout request to backend | high | 1. send a logout http request | send success | send success | Pass | Check the functions of corresponding frontend components | contact frontend developers |
| FT024 | frontend request: register | test whether the system can send register request to backend | high | 1. send a register http request with email address and password | send success | send success | Pass | Check the functions of corresponding frontend components | contact frontend developers |
| FT025 | frontend request: search | test whether the system can send searching request to backend | high | 1. send a search http request with keywords | send success | send success | Pass | Check the functions of corresponding frontend components | contact frontend developers |
| BT001 | backend processing: login (y) | test whether the backend can pass login requests from frontend (account information exists) | high | 1. backend receives login requests from frontend and responses | login approved | login approved | Pass | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
| BT002 | backend processing: login (n) | test whether the backend can reject login requests from frontend (account information does not exist) | high | 1. backend receives login requests from frontend and responses | login rejected | under developing | Under developing | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
| BT003 | backend processing: register (y) | test whether the backend can pass register requests from frontend (new account) | high | 1. backend receives register requests from frontend and responses | register approved | register approved | Pass | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
| BT004 | backend processing: register (n) | test whether the backend can reject register requests from frontend (existed account) | high | 1. backend receives register requests from frontend and responses | register rejected | register rejected | Pass | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
| BT005 | backend processing: search (y) | test whether the backend can process searching requests from frontend with results (results > 0) | high | 1. backend receives search requests from frontend and responses | results found, return |results found, return | Pass | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
| BT006 | backend processing: search (n) | test whether the backend can process searching requests from frontend with 0 results | high | 1. backend receives search requests from frontend and responses | results no found, return | results no found, return | Pass | Check the corresponding backend and database functions, and check the response functions | contact backend developers |
### 2. Integration test
| UID | Case Title | Description | Priority | Test Step | Expected Result | Actual Result | Status | Failure Management | Failure Communication |
|-------|--------------------------------------|----------------------------------------------------------------------------------------------------------|----------|------------------------------------------------------------------------------------|------------------------------------------------------------|---------------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------|
| IN001 | result finding | test whether the system can find expected results | high | 1. input 'Alan' 2. input 'Bob' | 1. result found 2. result no found | 1. result found 2. result no found | Pass | Check correctness of database functions | contact backend developers |
| IN002 | history recording | test whether the system can record the histories searched by uesrs | high | 1. input some text in search box and search 2. check whether this search is stored | searching history is transferred back into database | searching history is transferred back into database | Pass | Check the correctness of the functions of history recording system, and check the database | contact databse developers |
| IN003 | main page display | test whether the system can display all results correctly after searching | high | 1. log into the system | all components display and display in the right position | all components display but some of them display in the wrong position | Fail | Check the components in the frontend, and correct the wrong positions | contact frontend developers |
| IN004 | result page display | test whether the system can display every result in a sub-page | high | 1. log into the system 2. search some keywords | all components display and display in the right position | all components display and display in the right position | Pass | Check the components in the frontend, and correct the wrong positions | contact frontend developers |
| IN005 | frontend request | test whether the system can send http request to backend | high | 1. send a test http request | send success | send success | Pass | Check the components in the frontend, and correct the wrong positions | contact frontend developers |
| IN006 | backend processing | test whether the backend can process requests from frontend | high | 1. backend receives test http requests from frontend and responses | print 'success' at backend | print 'success' at backend | Pass | Check and correct the funtions of the corresponding components | contact backend developers |
| IN007 | database response | test whether the database can be connected by backend | high | 1. database receives test query from backend and returns | query ok, return data | query ok, return data | Pass | Check the corresponding backend functions | contact databse developers |
| IN008 | database response: NLP | test whether the database can write data in from the NLP Python file | high | 1. execute NLP file | NLP file write data into database | under developing | Under developing | Check the corresponding backend functions and the functions for communicating with database, and also check the correctness of the use of database language | contact databse developers |
| IN009 | NLP processing | test whether the system can call NLP Python file to execute according to the given keywords from backend | high | 1. backend send keywords 2. NLP executed 3. NLP wirte results into database | database get expected results | under developing | Under developing | Check the links and communications between Python and database | contact backend developers |
| IN010 | database response: login(y) | test whether the database can find matched account | high | 1. database receives login query from backend and returns | query ok, return data | query ok, return data | Pass | Check the correctness of the corresponding database language | contact databse developers |
| IN011 | database response: login(n) | test whether the database can return correct data when no matchs | high | 1. database receives login query from backend and returns | query fail, return null | under developing | Under developing | Check the correctness of the corresponding database language | contact databse developers |
| IN012 | database response: register(y) | test whether the database can keep new account data when no matchs | high | 1. database receives register query from backend and returns | query ok, return data, record this account | query ok, return data, record this account | Pass | Check the correctness of the corresponding database language | contact databse developers |
| IN013 | database response: register(n) | test whether the database can return correct data when account already exists | high | 1. database receives register query from backend and returns | query fail, return null | query fail, return null | Pass | Check the correctness of the corresponding database language | contact databse developers |
| IN014 | database response: search(y) | test whether the database can return correct data from searching request | high | 1. database receives search query from backend and returns | query ok, return data, record this search | query ok, return data, record this search | Pass | Check the correctness of the corresponding database language, check the related data in database | contact databse developers |
| IN015 | database response: search(n) | test whether the database can return correct data from searching request when there is no results | high | 1. database receives search query from backend and returns | query fail, return null, record this search | query fail, return null, record this search | Pass | Check the correctness of the corresponding database language, check the related data in database | contact databse developers |
| IN016 | database response: search history(y) | test whether the database can return users' histories if possible | high | 1. database receives history query from backend and returns | query ok, return last 5 data | query ok, return last 5 data | Pass | Check the correctness of the corresponding database language, check the history stored in database | contact databse developers |
| IN017 | database response: search history(n) | test whether the database can return correct information when there is no histories | high | 1. database receives history query from backend and returns | query fail, return null | query fail, return null | Pass | Check the correctness of the corresponding database language, check the history stored in database | contact databse developers |
### 3. System test
| UID | Case Title | Description | Priority | Test Step | Expected Result | Actual Result | Status | Failure Management | Failure Communication |
|-------|------------------------------------------|---------------------------------------------------------------------------------------------------------------------|----------|-------------------------------------------------------------------------------------------------------------|--------------------------------------------|---------------|--------|--------------------|----------------------------|
| SY001 | concurrency test (<=3) | test whether the system can have a good performance when the concurrency is low | high | 1. 1 user logs into the system and does searching 2. 3 users log into the system and do searchings | system should have very good response | for developed functions, system can response but with slow rate | Fail |Optimize the algorithms to have a better performance during higher load | contact backend developers |
| SY002 | concurrency test (>3 && <=10) | test whether the system can have a good performance when the concurrency is high | low | 1. 10 users log into the system and do searchings | system should have generally good response | under developing | Under developing |Optimize the algorithms to have a better performance during higher load | contact backend developers |
| SY003 | system function: login | test whether the system can have the full function of login | high | 1. use a test account to login | login successfully | login successfully | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY004 | system function: logout | test whether the system can have the full function of logout | high | 1. logout the system | logout successfully | login successfully | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY005 | system function: register | test whether the system can have the full function of register | high | 1. input email and password to register an account | register successfully | register successfully | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY006 | system function: searching | test whether the system can have the full function of searching | high | 1. log into the system 2. search some keywords | results shown | results can shown but the fuzzy search function is under developing | Under developing |Check the organization, stability and usability of each corresponding function or component| contact backend developers |
| SY007 | system function: history checking | test whether the system can have the full function of history checking | high | 1. log into the system 2. search some keywords 3. click the search box | histories display | histories display | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY008 | system function: sorting by name | test whether the system can have the full function of sorting by name | high | 1. log into the system 2. search some keywords 3. sort results by the sorting button | sort successfully | sort successfully | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY009 | system function: sorting by correlation | test whether the system can have the full function of sorting by correlation | medium | 1. log into the system 2. search some keywords 3. sort results by the sorting button | sort successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY010 | system function: sorting by citation | test whether the system can have the full function of sorting by citation | medium | 1. log into the system 2. search some keywords 3. sort results by the sorting button | sort successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY011 | system function: filtering by year | test whether the system can have the full function of filtering by publishing year | medium | 1. log into the system 2. search some keywords 3. filter results by the filtering button | filter successfully | filter successfully | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY012 | system function: filtering by type | test whether the system can have the full function of filtering by paper type | low | 1. log into the system 2. search some keywords 3. filter results by the filtering button | filter successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY013 | system function: filtering by topic | test whether the system can have the full function of filtering by paper topic | low | 1. log into the system 2. search some keywords 3. filter results by the filtering button | filter successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY014 | system function: show in figures (1) | test whether results can be shown in histograms | high | 1. log into the system 2. search some keywords 3. click the visulazation button 4. check in hisograms | visualize successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY015 | system function: show in figures (2) | test whether results can be shown in relation schema | medium | 1. log into the system 2. search some keywords 3. click the visulazation button 4. check in relation schema | visualize successfully | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY016 | system function: check details | test whether the system can have the full function of clicking into the page of one result and displaying correctly | medium | 1. log into the system 2. search some keywords 3. click the paper | enter detail page | enter detail page | Pass |Check the organization, stability and usability of each corresponding function or component | contact backend developers |
| SY017 | system function: recommendation creating | test whether the system can have the full function of making recommendation from searching results | medium | 1. log into the system 2. search some keywords 3. click the recommendation button | recommendations can be seen | under developing | Under developing |Check the organization, stability and usability of each corresponding function or component, and check the function of NLP network | contact backend developers |
### 4. Non-machine-based testing
We also plan and have done some non-mamchine-based testing. For our user acceptance testing and usability testing, we have invited some volunteers to help our testing. Our team has gained some general feedbacks based on their feelings. We have learned their opinions and summarized them as follows.
a. The UI can be further adjusted.
b. Poor stability of core functions such as search function.
c. Poor concurrency stability and slow response.
d. The chart is messy.
e. Recommendations are not accurate enough.
The above are areas that can be further improved and will be gradually solved in the future development.
The form below is the prototype of the questionnaire that we are going to collect some further feedbacks from the volunteers. The case or function indicates the functions that our system can perform, and the score is given by the volunteers to the acceptance of the corresponding function. The score ranges from 0 to 10, with 5 being acceptable, 0 being totally unacceptable and 10 being excellent. The volunteers can also leave their text comments for further description.
|ID|Case (Function)|Score|Comments|
|:---:|:---:|:---:|:---:|
|1|Login|||
|2|Logout|||
|3|Register|||
|4|Search|||
|5|Filter|||
|6|Recommendation|||
# Risk Register and Mangement Approach
| ID | Risk | Probability | Impact | Priority | Method | Strategy | Person |
| :--: | :----------------------------------------------------------: | :---------: | :----------: | :------: | :-------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
| 1 | The plan ignores essential tasks | Possible | Catastrophic | Critical | Avoid | All members make enough understanding and analysis with the project, and keep discussion with users | All |
| 2 | The plan is optimal, but unrealistic | Probable | Moderate | High | Avoid, Eliminate, Assume, Control | Try to find and delete the unrealistic part, or temporarily keep and modify it | Haopu Wang |
| 3 | Software size is larger than estimated | Probable | Severe | Critical | Assume, Control | Assume the software size is larger than we estimated, and design a more strict plan for it | Haopu Wang |
| 4 | Excessive schedule pressure causing reduced productivity | Possible | Severe | High | Avoid, Control | Try to make a reasonable plan and strictly work follow it | All |
| 5 | Delays in one task lead to a chain reaction of related tasks | Possible | Severe | High | Avoid, Control | All members try to finish their own work on time | All |
| 6 | Need to work in unfamiliar areas and spend more time than expected on design and implementation | Possible | Severe | High | Avoid, Eliminate, Assume, Control | If possible, try to use familiar languages or tools to develop for the whole project. Otherwise allocate a part of time for learning | All |
| 7 | The project lacks a cohesive top leader | Probable | Severe | Critical | Avoid | Choose a leader in our group | All |
| 8 | Too poorly planned to accommodate the desired pace of development | Possible | Severe | High | Control | Try to make a more thoughtful and strict plan | Haopu Wang |
| 9 | Development facilities are not sufficient to meet demand (e.g. training complex NLP model needs GPUs, but we may not have suitable hardware) | Possible | Catastrophic | Critical | Avoid, Eliminate, Buy, Control | Try to use the existing model or train a small model which does not require complex training step, or find usable GPUs for training | Zixian Ni, Yanbing Luo |
| 10 | Cannot find suitable dataset for training NLP model | Possible | Catastrophic | Critical | Avoid, Eliminate, Buy | Try to use the existing model, and find datasets from various resources whether it is large or small | Zixian Ni, Yanbing Luo |
| 11 | Project plans are abandoned due to pressure, leading to chaotic and inefficient development | Possible | Severe | High | Avoid, Control | All members try to follow the time plan and finish their own work on time | All |
| 12 | Development tools are not as effective as expected and developers need time to create a working environment or switch to a new tool | Possible | Severe | High | Avoid, Buy, Control | Try to find more information from internet about development tools to reduce the gap between the ideal and actual. And also find some alternative tools | Zihe Huang |
| 13 | The choice of development tools is not based on technical requirements and does not provide the performance required by the plan | Possible | Catastrophic | Critical | Avoid, Buy | Try to read and learn more about tools and the requirements of project to choose the most suitable one. | Zihe Huang |
| 14 | Learning the new development tools was longer and more informative than expected | Probable | Severe | High | Eliminate, Assume, Control | If possible, try to use development tools which at least one member of group is familiar with. Otherwise, allocate a part of time for learning when making plan | All for learning and choosing their familiar tools. Haopu Wang for making plan. |
| 15 | User requirements need to be modified during development | Possible | Severe | High | Avoid, Control | Consider as comprehensive as possible to make the planned requirements need less or no change when developing | Zihe Huang |
| 16 | Poorly defined requirements, where further definition would extend the scope of the project | Probable | Moderate | Medium | Avoid, Control | Try to define every requirements more clear and delete some requirements which may extend the scope too much | Yanbing Luo |
| 17 | Add additional requirements | Probable | Moderate | Medium | Avoid, Eliminate, Assume | Assume there are requirements will be added in the future when making plan. If there are too much requirements, try to delete some of them | Haopu Wang for making plan. Yanbing Luo for deleting function |
| 18 | Vaguely defined parts of the product take more time than expected | Probable | Severe | High | Avoid, Assume, Control | Try to analyse the project more comprehensive to reduce the vague understanding of project. And also making a more thoughtful plan | Zihe Huang |
| 19 | Modules with more errors require more testing, design and implementation work than expected | Probable | Moderate | Medium | Eliminate, Assume, Control | Delete some non-necessary modules that may have too many errors. Or try to write the code and design the test and time plan more carefully | Yanbing Luo, Zixian Ni |
| 20 | Improving a product of unacceptably low quality requires more testing, design and implementation work than expected | Probable | Moderate | Medium | Assume, Control | Allocate more time for quality testing. For important parts, let two members monitor and check each other's work to ensure quality | All |
| 21 | The software structure is not well designed and needs to be redesigned and reimplemented | Probable | Severe | High | Avoid, Buy, Control | Try to understand the project more comprehensive and search some other's software structure to learn how to design good softwares. | Zixian Ni, Yanbing Luo |
| 22 | Development of additional unneeded features lengthened the programme schedule | Probable | Moderate | Medium | Eliminate, Control | Without affecting the whole project, allocate part of time for additional features. Or just delete some of them. | Zihe Huang |
| 23 | Compatibility with existing various systems, requiring more events than expected for testing, design and implementation work | Certain | Moderate | Medium | Eliminate, Assume, Control | For a website, it is certain to encounter problems with compatibility with different systems computers. We can add more tests when developing to ensure the compatibility. Or just ignore the problem if don't have enough time. | Haopu Wang for designing tests, all for writing the code more carefully. |
| 24 | Project team members are not fully engaged in the project and therefore do not achieve the required level of product performance | Probable | Severe | High | Control | Let two members monitor and check each other's work, and set a strict deadline for every tasks. | All |
| 25 | Project team member is ill / has an emergency and cannot work for a period of time | Possible | Severe | High | Assume, Control | Assume someone will have an emergency. All team members keep track of everyone's work, and make enough comments when writing codes. | All |
| 26 | Conflict in group discussions leads to slow decisions and inefficient work | Possible | Severe | High | Avoid, Control | All group members maintain a good attitude during discussions and communicate any problems in time | All |
| 27 | Lack of necessary regulation increases errors and duplication of work | Probable | Severe | High | Avoid, Control | Let two members monitor and check each other's work and set a strict deadline for every tasks. | All |
| 28 | Some people need more time to adapt to unfamiliar programming languages and development tools | Probable | Severe | High | Avoid, Eliminate, Assume | If possible, try to allocate the tasks to members more reasonable and everyone can write code with their familiar languages. Otherwise pre-allocate enough time for learning. | All for choosing their suitable task, Zihe Huang for allocating time. |
| 29 | Mismatch of skills of persons assigned to the task | Possible | Severe | High | Avoid | Leader makes more discussion with members about their familiar tools and allocate the most suitable task | Zihe Huang |
| 30 | Some features cannot be implemented with existing code and libraries and developers will have to use new libraries or develop the required features themselves | Probable | Severe | Critical | Eliminate, Assume, Control | If the function is not too complex to write buy ourselves or is necessary, then allocate some time to write it. Or just delete the function. | Yanbing Luo, Zixian Ni |
| 31 | Low quality code and libraries, resulting in the need for additional testing, bug fixes or rework | Probable | Moderate | Medium | Eliminate, Buy, Assume | Try to find another libraries or existing code with high quality, or write the code by ourselves if possible and decide more tests. Otherwise delete the function. | Haopu Wang for designing tests, Zixian Ni for finding existing code |
| 32 | Separately developed modules cannot be integrated effectively and need to be redesigned or redone | Probable | Catastrophic | Critical | Avoid | Design the integration part carefully when designing the software structure. And all members track and discuss others' work in time. | Zihe Huang for designing software, all for tracking others' work |
| 33 | Inaccurate progress tracking, making it difficult to determine whether the project is behind schedule | Possible | Severe | High | Avoid, Control | All members try to track and discuss others' work in time. Every week has at least one meeting. | All |
| 34 | Inaccurate quality tracking, resulting in quality issues affecting progress not being known | Probable | Severe | High | Avoid, Control | Let two members monitor and check each other's work to ensure the quality | All |
| 35 | The computer is lost / broken, or other accident which leads to loss of work | Possible | Catastrophic | High | Avoid | Everyone upload their own work to gitlab at any time | All |
| 36 | The function of logging in and registering account function cannot pass the test | Possible | Moderate | Medium | Eliminate, Buy | Not core function, find some existing codes and make more tests, or delete the funtion | Yanbing Luo |
| 37 | The function of searching paper by typing some keywords cannot pass the test | Unlikely | Catastrophic | High | Avoid, Buy | Core function, relatively easy to implement, allocate enough time and find existing codes if cannot fix bugs | Zixian Ni |
| 38 | The function of filtering or sorting the search results cannot pass the test | Unlikely | Moderate | Low | Avoid, Eliminate, Buy | Not core function, if do not have enough time or cannot fix bugs, delete the function | Zihe Huang |
| 39 | The function of viewing the page of the details of a paper cannot pass the test | Unlikely | Catastrophic | High | Avoid, Buy | Core function, relatively easy to implement, allocate enough time and find existing codes if cannot fix bugs | Haopu Wang |
| 40 | The function of accessing the related data provided by recommendation and knowledge map cannot pass the test | Possible | Moderate | Low | Avoid, Eliminate, Buy | Not core function, if do not have enough time or cannot fix bugs, delete the function | Yanbing Luo |
| 41 | The function of generating recommendation based on search history of each user cannot pass the test | Probable | Moderate | Medium | Avoid, Eliminate, Buy | Not core function, if do not have enough time or cannot fix bugs, delete the function | Zixian Ni |
| 42 | The function of viewing recently search history cannot pass the test | Unlikely | Negligible | Trivial | Avoid, Eliminate, Buy | Low priority function, if do not have enough time or cannot fix bugs, delete the function | Zihe Huang |
| 43 | The function of switching the language of web page cannot pass the test | Unlikely | Negligible | Trivial | Avoid, Eliminate, Buy | Low priority function, if do not have enough time or cannot fix bugs, delete the function | Haopu Wang |
| 44 | The UI is not designed with good user-friendly and needs to be redesigned | Possible | Moderate | Low | Avoid, Buy | Design or use a existing good user-friendly UI before developing | Haopu Wang |
| 45 | The integration of different functions of front-end, back-end and database includes more errors or cannot pass the test | Possible | Catastrophic | High | Avoid, Buy | Core part of software, design how to integrate before and when developing. Learn from existing code about intefration | Zihe Huang for frontend, Zixian Ni for backend |
| 46 | Poor stability and access rate of website cannot pass the performance test | Possible | Severe | High | Avoid, Buy | Non-function requirement, write or use the code more carefully, and deploy the backend server as stability as possible | Zihe Huang, Zixian Ni |
| 47 | The software portability cannot pass the test (e.g. cannot work properly on different platforms or web browsers, or the components cannot be displayed in right position in different devices) | Probable | Moderate | Medium | Avoid, Buy, Eliminate | Consider more comprehensive when writing code or find existing framework with high portability. If do not have enough time, ignore the portability problem | Yanbing Luo, Haopu Wang |
| 48 | The accuracy of searching results cannot pass the test | Probable | Moderate | Medium | Buy, Control | Non-functional requirements, find existing code and improve it to make the results as accurate as possile | Haopu Wang |
| 49 | The NLP model of this project cannot be implemented because of dataset or other problems | Possible | Catastrophic | High | Avoid, Buy | Core function, allocate enough time and start to develop it as early as possible. Also find existing codes if cannot fix bugs | Yanbing Luo, Zixian Ni |
| 50 | The accuracy of NLP model analysis results is not high enough | Probable | Moderate | Medium | Buy, Eliminate | Non-functional requirement, try to find better training dataset and model from internet and make the results as accurate as possible. If too hard to improve, just keep the original version. | Yanbing Luo, Zixian Ni |
| 51 | The software is not security enough for users and authors of paper | Possible | Moderate | Medium | Avoid, Buy, Eliminate | Non-functional requirement, consider the security problem when developing to avoid the problem, and find secure systems from internet. If do not have enough time, just ignore it | Haopu Wang, Zihe Huang |
| 52 | The backend of software cannot be deployed to a remote server | Probable | Moderate | Medium | Buy, Eliminate | Deploy a stable server on our local computers, allocate time to find and buy reliable remote server and domain name from internet. If do not have enough time, just keep the local server | Zihe Huang, Zixian Ni |
# Choice of Testing Framework
## Java Backend Testing
In our project, we are going to apply unit testing to test our functions in the Java backend.
***JUnit*** [https://junit.org/junit5/] Framework like ***JUnit5*** is a unit testing framework for Java programming language. Unit testing will be used to measure whether the function of each of our units in the Java backend works correctly.
However, Unit testing can not ensure that all units work well when putting them together. We use Integration testing to test the communication between different parts of the whole backend Java system.
Another testing framework is called ***TestNG*** [https://testng.org/doc/index.html]. Both ***TestNG*** and ***JUnit*** use annotation, and they both can be used in unit test in Java language.
| |JUnit|TestNG|
|---|-----|------|
|Annotation|Y|Y|
|Unit testing|Y|Y|
|Integration testing|N|Y|
|End-to-end Testing|N|Y|
|Functional Testing|N|Y|
|Suite Test|Y|Y|
|Ignore Test|Y|Y|
|Exception Test|Y|Y|
|Timeout|Y|Y|
|Parameterized Test|Y|Y|
|Dependency Test|N|Y|
*Table: General differences between ***JUnit*** and ***TestNG****
Both ***JUnit*** and ***TestNG*** may have support for some test according to the table above, but generally, ***TestNG*** is more powerful compared to ***JUnit***. For example, in suite test, ***TestNG*** is able to use XML to bundle all tests at one place.
*An example of ***JUnit*** for suite test:*
```
package guru99.junit;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
@RunWith(Suite.class)
@Suite.SuiteClasses({
SuiteTest1.class,
SuiteTest2.class,
})
public class JunitTest {
// This class remains empty, it is used only as a holder for the above annotations
}
```
*An example of XML file used in ***TestNG*** for suite test:*
```
<?xml version = "1.0" encoding = "UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="My test suite">
<test name="testing">
<classes>
<class name="com.guru99.SuiteTest1" />
<class name="com.guru99.SuiteTest2" />
</classes>
</test>
</suite>
```
One of the differences is that ***JUnit*** is only able to be used in unit test, while ***TestNG*** can be used in integration test as well. This is because that ***TestNG*** has the dependences which can group the test cases. ***TestNG*** provides the supports of dependency test as well compared with ***JUnit***.
*An example of grouping tests in ***TestNG***:*
```
@Test(groups = { "Sanity", "Regression" })
public void test_method()
{
//Test implementation
}
```
The annotations in ***TestNG*** are more various, understandable and easier to use than in ***JUnit*** such as @ExpectedExceptions and @DataProvider. Using ***JUnit*** may require a certain amount of dependencies. ***TestNG*** also provides support for creating parallel tests.
|Description|JUnit|TestNG|
|---|---|---|
|Test annotation|@Test|@Test|
|Executes before the first test method is invoked in the current class|@BeforeClass|@BeforeClass|
|Executes before each test method|@Before|@BeforeMethod|
|Annotation to ignore a test|@ignore|@Test(enable=false)|
|Executes before all tests in the suite|N/A|@BeforeSuite|
|Executes before a test runs|N/A|@BeforeTest|
|Executes before the first test method is invoked that belongs to any of these groups is invoked|N/A|@BeforeGroups|
*Table: Some details about the differences in annotations*
If the test suite fails to run, ***JUnit*** has to rerun the entire test suite in the next time. When ***TestNG*** fails to run, it will create an XML file to describe the failed tests. With this file, the program will not repeat the successful test in the next time.
In summary, we decided to choose ***TestNG*** as our testing framework for our Java backend because it is more powerful and efficient according to the evaluation.
## Python Testing
We are also going to use a framework for Python programming language because we use Python to apply NLP to deal with jobs like the classification of papers.
***Unittest*** [https://docs.python.org/3/library/unittest.html] is one of the Python testing frameworks inspired by ***JUnit*** framework in Java. It does not need extra modules to be installed which makes it easier to be applied.
***Pytest*** [https://docs.pytest.org/en/7.2.x/] is another Python testing framework. It can be used to write various types of software tests, including unit testing and integration testing. It is an open-source framework with supports from a large community. ***Pytest*** supports simpler assert sentences instead of complex self.assert* functions. For ***Unittest***, we will have to import modules, create a class and define the testing functions within that class. But for ***Pytest***, we only have to define the testing function.
| |Unitest|Pytest|
|--------|-------|------|
|Built-in|Y|N|
|Unit testing|Y|Y|
|Integration testing|N|Y|
*Table: General differences between ***Unitest*** and ***Pytest****
In conclusion, we decided to choose ***Pytest*** for our Python testing framework due to its convenience and power.
Google Drive
Gist
Clipboard
Sign in with Wallet
