# IMT4886 Project 5: TaskEx

## Original Features

- Development of a standalone tool that executes based on external invocation (e.g., webhook), can perform open-ended configurable tasks/commands (generally associated with code)
- The tool should provide execution reports based on webhook integration
- Instances of this tool should be poolable or dockerized to support scalable execution
- Data structures should be human-readable and specified in general form to allow generic use with open-ended commands (and varying in-/outputs)
- Aggregate reports of multiple executions should be exportable in various formats (e.g., JSON, CSV).
- The tool should offer a simple UI that indicates execution progress.
- The preferred language for the tool is GoLang

## Features more in-depth

### Develop Standalone tool

### Execution based on webhooks

- Webhooks written in GoLang [webhooks](https://github.com/adnanh/webhook)
-

### Poolable and/or Dockerized to support scale
https://pkg.go.dev/github.com/docker/docker/client
https://willschenk.com/articles/2021/controlling_docker_in_golang/
https://www.loginradius.com/blog/engineering/build-push-docker-images-golang/

### Human readable data structure
- JSON
- CSV
- XML?
- YAML?
- LaTex/PDF?
### Simpe UI for execution progress

If going for GoLang we have some GUI libraries to pick from, list found here [here](https://golangr.com/gui/)
Of course it might be unessecery to go the extra step for a GUI if some simple terminal printed execution progress work.

But list them up here for order's sake:

#### Web GUI

All Web GUI are by their nature* cross-platform, as long as the platform can display a modern webpage

\* some use underlying backend that is not crossplatform, but that's just bad web-gui

- [go-astilectron](https://github.com/asticode/go-astilectron)
- Electron based Web GUI
- Tomas's Pick for Web GUI
- [go-sciter](https://github.com/sciter-sdk/go-sciter)
- Binding for [Sciter](https://sciter.com/)
- [gowd](https://github.com/dtylman/gowd)
- Node based [link](https://nwjs.io/)

#### Desktop GUI

- [Go-gtk](https://mattn.github.io/go-gtk/)
- Binds the [gtk](https://www.gtk.org/) package
- Which binds the GNOME
- For Linux (mostly)
- [gotk3](https://github.com/gotk3/gotk3)
- Similar to go-gtk except it binds GTK3 the newer version
- For Linux (Mostly)
- [qt](https://github.com/therecipe/qt)
- Go Bindings for [QT](https://www.qt.io/)

- [walk](https://github.com/lxn/walk)
- Binding for Natvie Windows Applications
- Windows ONLY
- [webview](https://github.com/webview/webview)
- Tiny Cross-platform UI
- Two-way JavaScript Bindings

### Writen in GoLang?

We Should know what our team langauge skill is regarding GoLang

#### Tomas' Programming Langauge knowledge

I am most familiar with C++ and Python. So related to this task of course Python would be my pick. However, I do have some GoLang experience, so at the cost of slighly slower development of feature. I am willing to try my hand on GoLang

#### Bhanu's Programming Langauge knowledge

I have had the most experience with Java, however, I have also done some development work with Python and Go Lang. I would be very much okay with performing the requirements with GoLang if that is the preferred language for the program but am open to other languages if required.

#### Even's Programming Langauge knowledge

Currently most familiar with Go. Javascript is also fine. I know Python well, but i have limited experience with large projects in Python and also limited experience with some of the popular packages we might use. If we use Python i will need to catch up.

#### Alf's Programming Langauge knowledge

Experienced with python where I do most of my programming and done some larger project (bachelor thesis). Ive also used some C and C# but there I more rusty, recently I've tried learning js.

#### Notable PL suggestions for this
- GoLang
- Product prefere use of this
- Find out why this is prefered
- Most of the team have some familiarity with it
- Relativly "new" language and "simple" to learn
- Go Routines
- Python
- "Old" and harded language with lots of support for just about everything (for better or worse)
- Very easy to get into (Google just about any easy to learn PL and you get python somewhere on the list)
- Most of the team have some familiarity with it
- Rust
- New Lanague
- Better for more Largescale/complex SW
- Fast
- [WebAssembly](https://en.wikipedia.org/wiki/WebAssembly)
- Best Pratice Language
- Very Great Error Handeling

### Other Features/Breakdown

### Linter
### Analytics
### Unittest in reop
### Unittest outside repo
### Repo written for other Langues
### Task1 Design it Modular/Cutomizable

#### Input YAML
- Things to run once
- things to run every commit/change/request
- things to run on tag
- things to run when deleting
- git repo
- Cross-Platformness?
- Run on docker container(s)
- optional docker contrainer parameter
- task you want to execute
- Staging of tasks
- Variables to task
- rules for the output
#### Output JSON

- Output to both Admin and User/ Even more modual output to multiple web API/hooks and Admin edcide where it goes

##### Structure all Data from git that we can get
- Git commit (Should we look back at its hitory? Optioan configureable?)
- Who committed
- Lines of code?
- files changed?
-
- Request
- Who requested
-

### Task2 Build the engine to pull, build and delete repos

https://github.com/go-git/go-git

#### Example of modular Only allow x amount of warnings
#### Repos/DOcker Maybe have a timeout/lifespan
#### Size of group vs contribution
#### Make rules customizable?
#### Idealy integrate it with the submission system

#### Web front-end
Use whatever tool you want
Prioritize the backend

## Acceptance criteria/Requirements

## Meetings
Thursday 11:30 Everyother week
Next couple of weeks everyweek.

# Sprint 1

## 1st meeting with PO's (16.09)

## 1st Planning meeting (19.09)

### Need Clarifications
- Clarification needed about the structure for tasks in **GitLab** *(Milestones-Epics-User Stories and Tasks)*
- Ask PO's about "configurable cmd execution on repositories" acceptance criteria (Specifically Run analyst)
- If we should default an all-in-one docker or multiple small on as a test enviorment
- Do we run milestone based on iterations or milestones based of complete features

### Decisions/Discussions
- Usertest will mostly be ignored and not focued on due to time constrains, and lifespan of project

### For next time

each person writes 3 user stories

- Add a user-stories to gitlab
- Git integration (might be slpit up to several)
- Docker integration
- User stories for input data/configurability
- User stories for output data, what data does the user want?
- Webhooks/Output feedback
- to get progress updates
- get result
- Webgui
- Define issue style
- Define coding style
- Define group roles (including rules for role changes)

example
- git integration (milestone)
- **As a Professor I want my students git repo to be cloned and run a set of commands on it, so that I can run unit tests *(User Story)***
- get newest version (task)
- clone if not existing in filesystem
- pull if exists in filesystem
- **As a Professor I want the repositories deleted after 2 weeks, so that my data/disk get cleaned up**
- delete after deadline end (task)

a milestone has one or more user stories.
a user story has one or more tasks.
tasks should be small and specific (https://www.atlassian.com/blog/productivity/how-to-write-smart-goals)

#### For Wednesday, 21 September
- Agree on the user stories
- Clarify/discuss what we want to talk to Chrisopher/Mariuz about *(Maybe send email?)*

#### User-story template

as a [description of user]
i want [functionality]
so that [benifit]

## 2nd meeting with PO's (22.09)
### Clearification from last time

### Get clearification

#### Friday pres.

- what has been done
- how did it go
- what went wrong
- improvements
- show burndown chart
- plan for next sprint
- sometimes show and tell

### Feedback on user stories

1 and 2 can be functional requirements

not take more than 10 seconds

4 - too large (more of an epic as of now). be more precise

7 - be more precise

## Team Meeting (22.06)

### Context
The team members sat down after the meeting with PO's to finalize behavior rules, git and coding sytle as well as role division.

### Major conclusions
- We set expected behavioral rules between each other.
- The git rules and standards that we want to follow will be pushed in the gitlab wiki of the project.
- It was decided we would have the product roles to be assigned to each other in a rotating basis.
- The rotating roles is because we are a small team and we will get to learn challenges of each role in turn
- The issue cycle was decided to be kept simple:
- 4 issue types

# Sprint 2
## 2nd Planning meeting (26.09)

- Role delegation
- As agreed we decided to assign roles every week separately
- Scrum Master
- Bhanu
- Time Keeper/Log Keeper
- Even
- Secretary
- Alf

### Need Clarifications
### Decisions/Discussions
### For next time
- How should JSON format look like?

## 3rd Planning meeting (28.09)
Scrum -> updates

Bhanu
Download code and tested branches
We should find a way sturucturing the code so we don’t have to change branches to test thing

Alf
Downloaded and testing the branches
Setup own

Even
- Docker code

Tomas
- CI pipeline - setup own laptop
- Make docker container able to run within docker

- Each time we do merge request update readme file with the requirements (write it for yourself just two years down the line) Just enough so you can run the code
- We go for GO version 1.19 for this project
- Write add to appendix - we did this version because XYZ…
- Chose branch
- We will ask the PO about changing branches
- The default branch would be the one we pull, or should we be able to pull a different branches?
- We look into this in sprint 3 or 4
- Create task
- Chose which branch to pull
- Credentials
- HTTPS get credentials
- SSH assume that the computer have the keys
- (Storing multiple )
- The end user case
- For a professor a SSH key would make sense as it can authenticate all
- Clarify with PO

Implement how to authenticate for GIT and not for gitlab spesific

- PO clarify, we need another repository for web GUI

Clarify How should the review mechanism should work in our team
- We should add comments and feedback

Resources that the code uses but are not strictly code should be in the resource file
- Docker
- Markdown
Config file separate - static for now -> configurable by the web gui
No use of Environment variables it should rather be read by config

Break down tasks in categories
- Research
- Documentation

Clarifying
- A merge request should mostly just do one thing

Create task
- Create read from yaml file
- Create read from json file

Planning poker - all estimate how long it will take
We deside 1 time unit (1 day of work 0.5 day of work)
You take the issue - everyone do an estimate how much time each task will take.
You take highest and lowest time estimate and check how much time it will take

Next sprint add an issue for supporting SSH and HTTPS

Keep the runner files (so we can have several main.go files, just structured under e.g. docker, testing)

We should probably implement test-cases down the lines - we have the CI pipeline

Add output to gitignore
### Need Clarifications
-PO down the line we would need another repository for a web GUI
-PO create wiki requirements page
### Decisions/Discussions
### For next time

## 3rd meeting with PO's (29.09)
Conserns docker container
- One requirement is good error reporting, chances are that we need to differenicense failed to build and failed to run (strong requirement, this one of top three requirement) Robustness errorhandling
- We need to classify which queries
- Identify the socket on which the service its mounted
- Eg. trading service (host trade stock telenor would be url, parameters appended, expected outcome, what content to receive) E.g. marius coruses - embedded script.
- External test, once check respons 200 ok 404 etc. (here is the issue )
- Docker compose (yaml file) is easier to use then port handeling (chose right architecture and bevare of ram usage)
- Be open to other code languages
- (Asynchronos reporting could be seen on in the future)
- Saving the results, to a obscured url guid
- Version 2, make a notification function e.g. mail when finished.
- Load perspective (students might have a peak load close to deadlines)
-
- Robust
- Reporting accessible
- Configurability
- Scalability

For first presentation
- Its fine to show a jason output ()
- Show subsensiate where you made the progress
- Dont edit the weights, its good for excercise learning estimates
- This should end up in the sprint (why was it to big, to many tasks, tasks and issues, did you overestimate, what strategies did you use)

Branch clarification
- Take the main/master branch or dev or submition branch
- Flexibility for the the upper years students

The one who request the repos should have the credentials to the repos
- Lowest common denominator (what autentications would work on all of those, all might support ssh)
- Which user is running this (service user, which ssh key for 3rd party users) Make an issue on exploring this.
What would be a solution
- we have a machine with a ssh key
- Could have make this as a requirement for students to add this

Making another repo or just create a subfolder for webgui
- Christopher could setup, but we should revisit this, should get feedback from Marius

Versioning the API

- Initial api use versioning, which could allow it to support new versions of a frontend/backend mix.

Wikipage requirements
- Document, and link to in general chat when we want feedback
- The POs can help prioritize

What are the requirements the PO want by the end of the product?

Running test (automating it):
- Test one which works
- One which fails at compile
- One which fails at runtime

Oral exam
Product
Report on the product and process (group report)

## 4th planning meeting (03.10.22)

Comment issues in the task
Later in the merge request

Git workflow clarification: Make a subbranch of development and merge request to development

Current functionality
* one function is mounting a folder to a docker
* one function is downloading a repo to a folder
-> We will need one function where we combine the two functionalities into one

Good reflection point
* Protected the branch, so you shouldn't be able to push to main or development, but GitLab was still set to allow owners to allow push.
* This was uncovered by Alf pushing mistakenly directly pushing to the development branch rather than making a subranch of development and then pushing to that

We will try to log our time usage to keep an overview -> toggl.com is the chosen tool
* We add issue number to our toggl track
* Add tags for meeting, code review, coding, sprint (and issue) etc.

## 5th Planning meeting (28.09)

- Alf
Mounte git -> OS.remove
Started working with chrono
Started on Issue \#13 and created own branch to implement on it
https://www.geeksforgeeks.org/how-to-delete-or-remove-a-file-in-golang/

- Even
Read to json file
Working on yaml? - (theres no built in yaml parser)

- Bhanu
Started on \#28, working on mounting
We might have a issue with merging everything

- Tomas
Made account on bitbucket, didn't manage to pull
blocked account on github

For next sprint -> Fix pipeline

## 4th meeting with PO's (06.10)
Make names customizable (go away from the course name, professor / student naming scheme)
Is it for the general public or only for academic use?
- The configurations sits on server only (if the professors give the test cases, the students would only fix the software to only solve the testcases).
- Make the naming scheme based of config file, and make the paths relative
- For (command based invocation) command line version, give status code and possible body of error
- For (url based invocation) would need to tink about how to guarante that the url (port)is , we need to retrieve the IP of the docker container to get the right container.

Any sample commands/ tests?
- Ask Marius, he made some tests 2 years ago, written in go
- Do a compare against the expected structure of the result against actualt structure of reply. 200 OK just tells that the request was okay,

(Having a consept as a result of a sprint is okay, as long as it is something concrete)

Assignment level (run one instance per assignment)
In an assignment we can duplicate it.

Document and make an guidence for the different types of repos's supported bitbucket, github, gitlab etc.

Trigger the product when subision

Ensure a clean shutdown of the container before you

Cleanup script, make a return code

config file, ensure that it also works for python,
# Sprint 3
## 6th Planning/Retro/Standup meeting (10.10) Mon - Sprint 2 end - Sprint 3 planning
- Bhanu
Worked on #13 run custom commands #28
- Tomas
Created merge request auth
- Alf
Made scheduler two examples one with chrono and one with go keyword-> pushed merge review

Sprint 3
- Cleanup of remaining sprint 2

For friday's presentation we should prepare a demo which includes:
- Presentation including
- Read from config file
- Pull a git repo / several repo's
- Run commands
- Get output
- Delete after x amount of time
- Show retrospective chart

Made a new discord channel for issues, ping and link to gitlab

Retrospective
- https://studntnu-my.sharepoint.com/:wb:/g/personal/bhanus_ntnu_no/EfGkhgnXbYJFnSacZMJDj1UBPTPXLLLFx2wJobh8QwWdQA?e=A17hFb
-

- Alf
- Bhanu
- Even
- Tomas
- Blank
- Scrum master
- Secretary
- Time
New Roles for sprint 3

### Clearification from last time

### Get clearification

## 7th planning meeting (12.10)

Plan for meeting:
- Standup
- Planning
- Start preperation for presentation on friday

Standup
- Alf
- Finished #13
- Looked at Bhanus work
- Work on feedback
- Bhanu
- Cleaned up code for #11 #28
- Run custom command on cloned repo
- mount and run commands
- To be done:
- read from file
- point to a test repo
- Issue:
- merging everything into dev
- Even
- Approved !5
- Created git analysis
- Will figure out which requirements can be set on the analysis result
- Tomas
- Merged in !5 (#39 #43)
- Started prototype (stitching together all parts)
- No issues, just some more figuring out

Planning
- Going through new issues
- Update README on merge to master
- Planning poker

Presentation
- Focus on sprint 2
- Time spent and relation to weight of issues
- Mention that we are in the middle of sprint 3
- Burndown chart
- Weight was assigned a few days into sprint
- Looks bad, but the remaining issues were in review and were moved to sprint 3 to finish review
- Showcase !5
- Very large task (like many others, propably too large)
- Lots of discussion

For PO meeting:
- should feedback be result of command or a result of the commands i.e. pass/fail + reason
- commands could be marked "hidden" to not return restult

## 4th meeting with PO's (13.10)

- Should have configuration to show or hide details of individual command
- Separate admin and children in config
- Use docker compose instead of docker
- make a schema

## 8th planning meeting (17.10)

Standup
- Alf
- New branch for #52
- Learning new go concepts
- Bhanu
- Merged !8
- Started #56
- Will look at linter
- Needs to figure out the right json format
- Blocked by
- Even
- git analysis
- Tomas
- havn't worked too much
- looked at docker compose
- might figure out
- might also just call underlying system
- Might do #55 is

Created sprint 4 in gitlab

## 9th planning meeting (19.10)

Plan for meeting:
- Standup

Standup
- Alf
- Working on branch
- How to remove git repo
- How
- Where
- Configured secrets file
- Bhanu
- Has loaded json
- Will sort commands to run in right order
- Even
- Git analysis
- Git analysis requirements
- Create invite to PO meeting 11:30 to 12:00
- Tomas
- Comment on !10
- Working on #55. Docker copy

## 5th PO meeting (20.10)

Git
- consider standard deviation
- look at different distribution/clustering metrics

Docker
- Make copying method configurable

Config files
- Make sure it is robust. There is always chance that custom commands are wrong. Make sure the users are sent where exactly there are errors and show there is an error.
- Have solid documentation to debug the commands that are wrong
- Tell users where there is error and if the commands are okay. If the config file is correct when a new config file is seen.
- Maybe parse the config file first and check that everything is okay. The file is validated first before it is executed. The commands need to be validated so that there is no down time because of random errors.
- A configuration that is same across all assignments and smaller configurations for each assignments
- Transparency of where you are at the code and what have been activated and what have not been can be done
- **Suggestions from Christopher** Make a documentation or visualization of how this would work so that there is clearer idea about this.
- Have default values

# Sprint 4
## 10th planning meeting (24.10)

Plan:
- Standup
- MR discussion
- Retro
- New roles
- Planning

### Standup

- Alf
- Working on deleting files and testing
- Relative vs absolute paths
- Bhanu
- Work on config
- Golang language problems
- Working through the comments on the MR
- Testing is diffucult / unsure
- Even
- Merged git analysis
- Code review
- Tomas
- Docker compose
- Code review
- Made new issues

### MR discussion

!13
- git.GetGit() => git.Get()
- function to check if a file exists

!14
- ContainerRun => CreateAndRunContainer

### Retro

retro is in [whiteboard](https://whiteboard.office.com/me/whiteboards/p/c3BvOmh0dHBzOi8vc3R1ZG50bnUtbXkuc2hhcmVwb2ludC5jb20vcGVyc29uYWwvYmhhbnVzX250bnVfbm8%3D/b!CnzhqdyBCk2OUn5DG_cxTKqm7B-HjUpHlM1_ejyVv6_eE8Juiw-STYeFAnhmV260/01QKYJMK7RUSDATV3NQJCZ2JU4MTBEHD2V)

### New roles

- Scrum Master
- Alf
- Time Keeper/Log Keeper
- Tomas
- Secretary
- Bhanu

### Planning

...

What do we want in the end of Sprint 4?
- A CI/CD pipeline setup
- Create a prototype V2 (a main file that does the whole process loop)

We will have 8 pm meetings from now on.
The sprint planning meeting should be tomorrow at 8 pm.
#
## 11th planning meeting (25.10) 20:00

Plan:
- [x] Standup
- [x] Planning poker
- [x] Write requirements

### Standup Updates
### Alf
- Been fixing merge comments
- Completed merge comments from last meeting
- Requirements and Tests have been created
- Fixed some paths with absolutes
- Ready to be reviewed and hopefully merged

### Bhanu
- Will look into test cases for #56

### Even
- Looking into merge request and approved some of them

### Tomas
- Docker compose can work now
- Use yaml to open two different containers now
- Still testing more, does work for a test case for now
- Need some feedback from Christopher about repository to be used for docker compose use to build container
- Have started working on the docker hierarchy task and started by pushing into a branch

### Planning Poker

#### Git checkout #51
- Change in configuration to cloning or checkout to the correct branch
- If no branch mentioned in config then just pulls the main branch
- Most of the code is already in the git library
- https://git.gvk.idi.ntnu.no/course/imt4886/imt4886-2022/projects/taskex/-/blob/ft_protoype/main.go#L115
- Bhanu should probably do this. Will also help in wiki creation
- Work hours weight - 8

#### Start on a webhook prototype #53
- Even should probably do this
- Should probably do some database lookup (mongodb or firebase)
- Every interval of maybe 5 seconds to do some webhook task
- For now -ls in container and return the values
- Review will take a bit longer
- Team split on planning poker so decided on average of 17 work hours weight

#### Create CI Pipeline YAML file #54
- Maybe do something simple as build and execute the main.go file
- Or run a test case in the CI pipeline
- Should be short but will have to learn about CI pipeline
- Will also have some discussions
- Simple main file can be created for the testing and learning portion
- Agreed by everyone for weight of 8

#### Create out lifetime loop (main file) #58
- Tomas will continue working on this
- The docker hierarchy task and this task could be combined
- This function takes in the webhook config and runs the webhook config
- Create a go channel that will wait for the webhook config
- Will also depend on issue #62 related to ensuring docker hierarchy
- The code from #62 will be used for this task
- Average was taken for 15

#### Ensure docker hierarchy #62
- Average was taken for weigth of 8
- Discussion and inclusion of outcomes for #58

#### Config/docer due date #60
- The due date will be in the admin config
- Cron job will look for the due date is reached
- Professor should set the due date (due date for this is after sometime of the deadline to account for student's who submit things late)
- Whenever due date is reached the docker and webhook is cleaned and removed
- There should not be too many discussions here
- Agreed by everyone in the second meet at weight of 5

#### Create Requirements wiki page #20
- This will be used for smaller tasks
- Can be active throughout the duration of the project
- Non Functional requirement up for discussion
- Define how much time it takes from submitting webhook to getting a result or some form of response
- 20 seconds could be good
- Can we define how much time it would take to get everything done
- A performance metric should be set
- No more than 10 seconds to run something
- How many docker containers should we be able to run simultaneously
- How many webhook requests we can manage simultaneously
- How many git repos should we have stored at the same time
- Fucntional Requirements
- Feedback from system to admin and users
- Validate config file on startup
- Validate all config files used for any of the tasks
- Check if there are wrong mount types or other details
- Check regex of types of project name
- Should there also be an authenticator?
- For now the person running the computer is the admin
- If we expose the admin loading and configuration to another website but we are not doing that at the moment.
- The one using the terminal is the admin at the moment.

- Bhanu takes ownership for requirements in the wiki

### Notes
- There is an optional requirement for project name, and the webhook requires this to align with the right admin config. The unique project name links the admin config with the webhook, so the project name should be a compulsory requirement and make sure it is unique.
- Should there also be an update config?
- What if the next thing to be run is the only thing that needs to be updated and does not affect the other containers that are running with existing configs?

### Outcomes
- Meeting on 20:00 Wednesday 26 October 2022 for Slide prepration
- Get feedback from Christopher for the slides
- Const should be used instead of global variable with snake case with capital letters and an underscore
- NEED to get Christopher's feedback for requirements
- Should start working on documentation for report from next week

## 6th PO meeting (27.10)
*Attended by: Alf, Christopher, Even, Tomas, Bhanu*
Started with review of the slides to be presented for Christopher

- Feedback from Christopher for slides
- Can focus on the reflection part of the presentation
- Reflection could be that some tasks were over estimated and some were under estimated
- The bulb at the end of slide 6 in presentation slide of sprint 3 indicates tasks were underestimated.
- These kinds of input might be helpful to clarify that we understand how the burndown chart is created and what it means

- Feedback from Christopher about Requirements
- C : Try not to limit the time for execution so that there is restrictions for more complex tasks
- Our current restriction is more like how much time it takes to give some response
- C : As an instructor, I want to be able to design and add the commands that I want to run
- C : Have an ability to log who has run what | "logging feature"
- This will give more insight about how much the users are utilizing the system
- Gives the greater insight about how the users use the system

- Feedback to students would have some web ui
- High level overview response
- Some more details
- add some visual output like a traffic light
- red, yellow and green lights to notify about status of repo that was sent
- red if no response
- Yellow if there is response but some issues and so on
- Specify input, output, runtime environment
- Inform or notify that something is completed by sending a mail with a link (that is hashed; sufficiently unique)
- The link will give detail about execution status and other details
- Specify what kind of results are exepcted by lecturers and students
- As students see some statistics
- As lecturers they see student visible statistics and more detailed statistics and output

- Determine how the user interacts with the system
- Do you want to send individual notification or aggregated
- Students might require immediate information about information from the system
- Ideally instructor gets aggregated one which has multiple details about student responses and get it in the form like a dashboard that the instructor is able to refresh and get details more easily

- Feedback about graphs and system diagram
- Think about the various tasks and steps to just breakdown how it works
- The overall diagram and system design diagram will be helpful for the overall report

- Feedback on git analysis
- Make sure there are some analysis that is only visible for instructor only and that is for students and instructors
- Maybe we have a hidden key for some of the git analysis details to have teachers also get some access to it

### Seminar 5 Feedback
- Start working on the documentation for the final report

## 11th planning meeting (31.10) 20:00

### Tomas Update
- Continuing with docker compose.
- Fixing issues and exploring
- Working on setting the working directory to simulate 'cd' command
- Will continue with docker compose
- Working on running custom commands on docker to see if code is hidden or detached
- Some changes on file remove branch
- Things have been merged from branches into the development and master branch
- Question from Even: Should merge into master be rebasing? **weakness to add to report**
-

### Bhanu Update
- Working on issue #51 with git checkout for branch
- Have the git able to clone different branch based on webhook config file's branch parameter
- Issue with checking out to different branch if repository already exists

### Even Update
- Worked on merge request
- Working on the webhook
- We are storing everything in database and using websockets
- Many working parts that need to be brought together
- The final outcome is taking shape and will have some outcome by Wednesday for this
- Some tests have been done but overall tests need to be done soon
- The concept of stages of commands is also being worked on
- Stages will have multiple commands that will be run to check together

### Alf Update
- Merge fixes on ft_remove_repo
- New branch ft_Ci_pipeline (CI_pipeline not supported by regex)
- Added git runner (needs more config)
- Made the CI pipeline run something, still needs a main.go and some more fixees

## Discussions
- Do we have exit code? Can docker return exit codes?
- Can be used to determine the result of a stage
- Database demo by Even
- When the user wants us to checkout to another branch from main branch, should the repository be recloned with separate branch or just checkout to different branch after pulling?
- Do we only download the specific branch single branch?

## 12th planning meeting (2.11) 20:00

Plan
- [x] Standup
- [x] Demo
- [x] Plan questions for PO

### Tomas Update

### Bhanu Update
- Working on #51
- Clones, pulls and checkouts the branches as per task
- Will work on test cases for this
- Suggestion from Tomas:
- Use public repository so no authentication required
- Test for success and failure such as pulling branches that does not exist

### Even Update
- Created !19 for #53
- Need cleanup and work on till final deliverables

### Alf Update
- CI Pipeline #56
- Runs all tests, builds repo
- Added own desktop as a runner
- Docker image builds every time
- Config docker due data #60

## Demo by Even
Run the following commands
- docker-compose up
- make createdb
- make migrateup
- go .\main.go

- Now the socket should be ready to be connected

- The localhost:8080/project endpoint is ready for POST command
- The input should be similar like a config file
- MAYBE HAVE THE REQUIRED INPUT JSON HERE
-

## 6th PO meeting (3.11)

PO feedback from Christopher:
- Default setting should be there for beginner students who do not know about different branches or different features
- A form variant instead of JSON data structure could be there for the submission
- Maybe also look into CSS for beautifying the outcome
- Can you possibly assign weights on different elements
- Add weights to different sections or task
- Calculate score out of 100% , how many things went well or not
- The completion scores could also be added for next students
- This could be future additional features that could be added

- Focus on integration for the next phase
- Try to have all features under one umbrella, integrate the independent activitis into single feature
- After the integration is done, have one sample that the lectures can run and build
- Simple example and complex example that highlight the code that has been done

Q: How do you synchronize the front end and back end to show the outcome and have the data
Note down all that has been done for future reports and document

- Suggestion, the prototype can be continued to be worked on for the coming semester

- What is sensible deployment for larger usage when this goes to products?
- Experiment with different peak load
- Operational decisions rather than developmental tests

- Should there be first and secondary deadline

# Sprint 5
## 13th planning meeting (07.11)

Plan:
- [x] Standup
- [x] Demo of result of pair programming 3.11
- [x] Discussion
- issues need to be closed or moved today
- plan for merge request
- [x] Retro
- [x] Sprint planning
- writing report
- [x] new roles
- [x] DB setup

### Standup
#### Bhanu
- git (integration) test

#### Alf Update
- CI Pipeline
- Issue wih CI Pipeline integration due to testartifact folder needs to be fixed
- Schedule remove on due date
#### Tomas
- Even and Tomas spent time mixed front end and back end to create a demo

#### Even
- Worked on demo with Tomas
- Issue with demo is that it only takes the last line of the command output

### Demo
Everyone got it running

### Discussion
- Pushed sprint 4 due date to wednesday to finish the reviews of the tasks from sprint 4

### Retrospective
- Done with retrospective
- Test case was helpful which was surprising
- Task too big so might think about breaking things down in next sprint
- Need to start working on documentations and report creation
- Next stage should start in documentation, cleaning up and making code efficient
- Time spent on the project will decrease due to exams and submissions of other subjects coming near

- Scrum Master
- Even
- Time Keeper/Log Keeper
- Alf
- Secretary
- Tomas
- Backup
- Bhanu

### Sprint Planning

#### Task form last sprint
- 1st order of busniess i do get branches merges
- Bhanu and will look at merge req 22
- Tomas will look at even's brach
- Try to be donbe befreo wedneday

#### Issues for Sprint 5
##### Issue 16 Make the go plugin
##### Issue 15 Fix the cecent commands(Sprint 5)
- Weight: 4
- Ässigned: Even
##### Issue 12 Unique ID for repo (Sprint 5)
- Weight: 3
- Assigned: Alf
##### Issue 21 Requires certain files (Nice to have)
##### Issue 69 Make Plugin work with the db submission (Sprint 5)
- Weight 5
- Assigned: Bhanu
##### Issue 64 Remove docker(Nice to have)
##### Issue 70 Deactivated Project (Sprint 5)
- Weight: 4
- Assigned: Even
##### Issue 59 Unload load CLI (Sprint 5)
- Weight: 8
- Assigned: Tomas
##### Issue 68 Fix CI Pipeline bug (Sprint 5)
- Weight: 3
- Assigned: Even

####

## 14th planning meeting (09.11) 21:00

Plan:
- [x] Standup
- [x] Issues with MRs
- [x] Close issues
- [x] Presentation friday
- [x] Prepare questions for PO
- [x] DB discussion
- [x] Things that needs to be done before fridays demo
- [x] Give out report writing task

### Standup

#### Tomas

- Merged branches
- Started on Terminal loading of config
- did code review
#### Bhanu

- Did refactor and added more testing
- Will work on making git pull only the artifcats for the brach
#### Even

- Did some code review
#### Alf

- started on bug issue to Make CI pipeline work
- But had to remove a test
- Code review
- Continues work on scheduler from last sprint
### Issues with MR

- Mergred Docker hierarchy
- Merge conflict with every
- Issue 19 Okay, but need to info asbout make
- Issue 20 Has merge conflict but other than that looks good
- Issue 23 Is a draft but is ver y soon ready for review
- Issue 24 Got commets but onces those are resolved it looks good

Closed Sprint 4

### Presentation
Bhanu has created some slides for the presentation

### Final demo
on the 25th of November

### Final Delivery
1st of December

### Question for PO/Aida

1. What's the deal with the "exam" on the 21st
2. Cadidate nummber or stud number
3. Time schedule on friday. When is the retros?

### DB discussion
Tomas and Even will look at how to create multiple Proejct in DB

### Report
Write a summery of the sprint you were scrum master for

Make grapsh and figures for relvent stuff you worked on

## PO Metting (10.11) 18:00

- [ ] Demo

### Demo

On live trying false, it worked

Test / get it working detached

Document how it works / cmds work regarding hidden and detached

Get the git analysis module integrated

Document limitation

### Project document
Unsure-what is inconmpleted- and work that need to be done

Ideas and solution to our current probles

## **CHRISTOPHER feedback:**

Requirements could be that
Students should be able to run this independently maybe (Asynchonisable with notifications (such as email))

RTesource management to remove all things spawned and created

Feedback must be clear and self explanatory
Inherently dockerized and not run in actual machine
Students should be able to run it repeatedly
Clarifying which requirements have been completed and not completed
Prioritization of commands can be done
Meaningful integration of front and backend

Students should clearly know that the server is running or that there is an issue in the server

System should be language agnostic. Should be able to run any programming language.

Document future ideas for future iterations of the product to be actual use ready in real courses

Test cases to check if it supports multiple instances 10-12 instances (Maybe something for the future)

You can justify that you have large weight because this was something that needed to be done to have a working prototype. Frame this as a learning opportunity and learning from doing. Focus on reflectiveness of this.

Probably frame that there is delay in response for discussion in reviews.

## 15th planning meeting (14.11) 20:00

Plan:
- [x] Standup

### Standup

#### Tomas
Worked on other projects mostly

#### Bhanu
Unsure about what is holding up merge request nr 20
Looked into proto type v2 brach
Meet with Even tomorrow to look into plugin (Pair Programming)
Asked for 2MACS to get sample reports

Want to fill boilerplate stuff for the project reprort

#### Alf
Worked on merge request 64,
review comments made some tests
fixed bugs (related to files)
Fixed the CI pipe line issue

#### Even
Worked on merge req 19 (should be ready)
Improvment branch for Proto_Type_v2

Will be support for DataBase integration with other modules/parts

### Project Plan

Write a paragraph by

## 16th planning meeting (16.11) 20:00

Plan:
- [x] Standup
- [x] Demo of issues
- [x] Intro to logging

### Standup

#### Bhanu
- Working on the unwarpping plugin command to the Database (w/ help of Even)
- Unsure about how use stage with regards commands order
- Looked into former reports for the project

#### Alf
- Looked into formal requirements
- Had some Pair Programming with even to learn a bit abou the database

#### Even
- Mostly been Support for everybody else
- Made add and delete project function
- Created a merge request

#### Tomas
- Pair Programming with Even
- Continued our Terminal interface

## PO Metting (17.11) 11:35

- [x] Tomas demo
- [x] Bhanu concern

### Terminal demo
No specific feedback

### Plug in commands
Mixing them for both stages and commands
Also write down/document the usecases

### Report feedback
Use visualization
USe the report to fill out the wiki

Process
- Documentation
-

Requirement must be divided into must have, could have, good to have

Process - SCRUM and Agile process must be clearly described

Features and architecture could be clearly described

Implementation could have some CODE, UI and other snippets
Show how they are done and explain around it

Future Work
Showcase features that are working, that are wonky and that you do not have yet

Add another section
Evaluation of prototype &

Evaluation of work or process (Reflection of group work)

Talk about validity and reliability
- If you add different inputs and where the system can be used and will it give reliable values

Write the report in active form (aka "we" did ..., as opposed to passive where you avoid using "we/i/us")

## 17th Planning meeting (21.11) 20:00

Plan
- [x] Standup
- [x] Merge requests
- [x] Writing
- [ ] When to prepare presentation for friday?

### Standup
#### Alf
- Worked on formal requirments for users
- Had problems setting up sqlc on windows
- Did reviewing and QA
-

#### Bhanu
- Plugin integration with merge request
- Looked into the report writing and started on the introduction

#### Tomas
- Added a terminal interface
- Added a new logger
- Create a merge request with this

#### Even
- Mostly been helping others
- Denial of late submission
- Update the logger
- Plan to integrate git analisys

# Coding Rules

https://github.com/golang/go/wiki/CodeReviewComments

## Git rules

# Feedback after Seminar 3
_Date: 14 October 2022_
## From Mariusz Nowostawski
A comment for task execution group: it is important to have "end-to-end" tests that go from a teacher through student submissions to testing the actual submissions. So having a simple assignment that demo the entire system would be desirable at the end. Not in the next sprint or the next one, but, ultimately it would be good to have demo like that.

Want a demo (sprint 4? meeting with PO to define)
- Test with a trivaial repo e.g. with one sourcefile git repo which shows the process
- Show the perspective from the different parts
- student
- teacher
- system architect

## From Christopher Frantz
I guess we will build up toward a full system

## Preparation for oral exam
- Explain what you did in the project.
- PO Chrisopher said:
- Know what everyone else did
- Issues they had, and what they struggeled with

- Why do we not have functional and non-functional requirements
- Because we did not have specified requirements
- This was more a prototype

- Usecase diagram
- Web interface can create projects and add repos to them by API calls
- Risk mitigation
- CI Pipeline
- Issue: Making the CI Pipeline run within docker (docker in docker)
- Was not implemented for a long time
- When there was tests the CI Pipeline was applied
- Web sockets
- Allowed push based functionality (such that the users wont need to refresh the site to get feedback)
- Web hooks were considered but we chose web sockets because it was simpler and faster for our use case
- Websockets are more appropriate to push new information to front end

- Database
- Issue: More and more fields were stored into the database

- Initially we considered using JSON file
- But we chose DB because JSON would be very simple and reading from DB seemed more robust. Database would have better intigrity and is considered standard.
- When there were changes, there were so many issues because we did not have adequate

- Git and Docker
- functionality used through go library, because this gave use a more structured and predictable manner
- Git analysis
- Analyzed git logs using go-git
- Future work
- We worked mainly on backend, as this project would explore and be a demo of a taskex system. So frontend
- Tests
- Test the DB and web interface with mocking

- Testing
- Whitebox testing: Know how the functionality works
- Blackbox testing: You know your input and expected output, but not the inner workings of the code

## Simple overview who made what
- Even
- Database
- schema
- code
- Web interface
- Web server
- WebSockets server
- Git
- pull/clone (first part)
- analysis
- requirements
-
- pair programming
- Alf
- CI pipeline
- Scheduled removal
- Tomas
- CI pipeline setup
- Docker and docker compose
- Cmd execution in Docker
- Terminal Interface
- SSH support for git
- Prototype v1
- Bhanu
- Plugin
- Git checkout / branch switch