https://uppsala.instructure.com/courses/96483


# 1.Set up
Installation stuff to windows
## Install the windows subsystem and linus bash

https://learn.microsoft.com/en-us/windows/wsl/install
Open the windows terminal and write `wsl --install`
It take some time to install, after installation, restart computer.
The terminal with a penguin logo pop up. You need to set up the new user name and password with it.

Your default installed linux distros (remember this term) is Utanbu. You can use `wsl --list --verbose` or `wsl -l -v` in windows terminal to check.
To check what distributions are avaialable: `wsl --list --online` or `wsl -l -o`
To enter the distro: `wsl -d Ubuntu`
To install additional Linux distributions after the initial install, you may also use the command: `wsl --install -d <Distribution Name>.`

More detailed information regarding: if you forgot your passowrd, set default distribution and set default verison pof a distribution can be dound at this page. https://learn.microsoft.com/en-us/windows/wsl/troubleshooting#installation-issues

### How to do termination linux distro windows
Terminate one linus distro
Terminate all linus distro
https://pureinfotech.com/shutdown-linux-distro-wsl/
https://learn.microsoft.com/en-us/windows/wsl/install

### How to change between linus
ctrl+D to exit from windows to linus subsystem
use `wsl -d Ubuntu` to enter your subsystem

## intall git
Use `git --version `to check ther veriosn of git in both windows and linux system

##
change your directory anc clone a github in your linux

## Intall conda and mamba
https://uppsala.instructure.com/courses/96483/pages/pre-course-setup?module_item_id=1027795

# 1.Git
https://education.github.com/git-cheat-sheet-education.pdf
https://rogerdudler.github.io/git-guide/
https://docs.github.com/en/get-started/getting-started-with-git/set-up-git
https://book.git-scm.com/docs
## 1.1 creating repository
Build a new folder and open the git bash in it
`git init`.
Run `git status` to check for the status of your current non-git repository.

If you copy some files from a git repositotty and run `git status` again, it will show that there are new files in the directory

## 1.2 committing changes

### going to staging area (per file) and going to repository (all together)
`git add Dockerfile Snakefile`
files go to the staging areas

`git status`

`git add config.yml environment.yml`
files go to the staging areas

`git status`

`git commit -m "Add initial files"`
Files are commited
-m options add commit message, describing what the commit contains. If you run git commit without the -m flag, Git will open the default terminal text editor (which can be configured with the core.editor variable) where you can write a longer commit message and body.

`git status`
It shows nothing to commit again.

Change smth in the `environment.yml` and `config.yml` file and run `git status` again.

Run `git diff environment.yml`, it will show you what is changed.

Run `git add environment.yml config.yml` to make the files go to staging area

### go back from staging area to working copy

Unstage the staged changes.
`git reset HEAD environment.yml`

`git status`
Note that unstaged changes will show as red, while staged changes will show as green. The commited changes to git tepositroy will not show up here.

`git commit -m "Change to ST398 for alignment"`
Only staged changes will be commited

`git status`

### Delete a final comitted file
`rm Dockerfile`: This is to delete the file.

`git add Dockerfile`: This is to stage the deletion
`git status`
`git commit -m "Remove Dockerfile"`: This is to commit the deletion
`git status`

(`git rm Dockerfile` can do `rm Dockerfile` and `git add Dockerfile` in one step)

### View history
`git log`

## 1.3 ignoreing files
### Making directory and files

Make directory
`mkdir -p results/multiqc`

Make files
`touch results/multiqc/multiqc_general_stats.txt`
`touch results/supplementary.html`
`touch log.tmp`

### Make .gitignore
`touch .gitignore`
Add .gitignore and paste this in it
Note that for the specifed directories and files, their changes and status (unstaged or staged) will be ignored.

Add some other data
`mkdir data`
`touch data/huge.fastq.gz`
`touch data/metadata.txt`
Specifed in the .gitignore

### adding .gitignore and stuff to staging area and commit them
`git add .gitignore`
`git commit -m "Add .gitignore file"`
`git add data/metadata.txt`
`git commit -m "Add metadata file"`

The folder and files specified to be ignored in .gitignore will not show up as changes(red), staging (green) and commited finally (not shown)

## 1.4 git branch

### Basics
See the current branch
`git branch`

Make a new branch (will copy what is finally commited)
`git branch test_alignment`
`git branch`

Move to the branched
`git checkout test_alignment`
Now the folder and file in your repository are in that branch. Everything you change will be on that branch only.

Probably happens when there is a conflict>
When you switch back `git chekout main`, it seems you need to at least add and commit the changes in your branch first (if you newly changed something in your file and folder).
Similarly, when you switch back `git chekout trimming`, it seems you need to at least add and commit the changes in your main first(if you newly changed something in your file and folder).

When you switch to a branch, all the files and folder will be mireculously updated to what is in that branch

You can create and checkout a new branch in one line with `git checkout -b branch_name`.

### Add some change to new branches

To get a visual view of your branches and commits history you can use the command:
`git log --graph --all --oneline`

To see the difference between branches (what is already added and commited):

`git diff main` (You need to run this after you checkout to the trimming branch)
The green text are what is in the trimming branch and the red is what is in the main.

`git diff trimming`(You need to run this after you checkout to the main branch)
The green text are what is in the main branch and the red is what is in the trimming.

We can also add the `git diff main --color-words` to display the difference on a word-per-word basis rather than line-per-line.

### Merge the branch
Checkout to the branch you want to merge into:
`git checkout main`

To merge (You need to commit first before merging; You need to change it to the main directory to merge. The merge function only work on specified branch--> current branch):
`git merge test_alignment`

Run `git log --graph --all --oneline` again to see how the merge commit brings back the changes made in `test_alignment` to `main`.

### Delete the branch
`git branch -d test_alignment`
Note that you need to get out from that repository (checkout to another repostory) first, before deleting it, otherwise there might be an error.

Run `git log --graph --all --oneline` again. Only main branch is there.

### Create a branch from another branch

## 1.5 Tags
### Add one tag
`git tag "submission1"`

List all the tag
`git tag`

Add annotation for your tag
`git tag -a submission1 -m "Annotation for tag submission1"`

List the first 10 lines of each tag's annotation
`git tag -n10`

### Add another tag and use it switch versions
Make some changes in one of the files and then add a new tag for the version after the change using:
`git tag "publication"`

Move to the orignal version
`git checkout submission1`

Move back
`git checkout main`

Open the file after running each `git checkout `and you can see that the file under `git checkout main` is changed

You can also see the difference between tags in the same way as for branches and commits using e.g. `git diff <tag1> <tag2>`.
`git diff submission1 publication`

At this point could run `git log --oneline --decorate` to get a condensed commit history

## 1.6 Working remotely
### Create a new repository from R studio terminal
After you create a new repository on
`git remote add origin [e-mail]:user_name/repository_name.git`

`git remote -v`
This will show you what remote location is connected to your local Git clone. The short name of the default remote is usually “origin” by convention.

`git push origin main`
The push command sends our local history of the `main` branch to the same branch on the remote (`origin`). Our Git repository is now stored on GitHub!

Note that all the history are also pushed.

`git status`

You always need to specify `git push origin main` by default, but you can circumvent this by telling Git that you always want to push to `origin/main` when you’re on your local `main `branch. To do this, use the command `git branch --set-upstream-to origin/main`.

`git status`

### Create a read me file and commit
Build a readme.md file in your git_tutorial folder (on your local computer)
This can be done by: open a text editor--> save the file as "All files", and name it with .md in the end.

Run the following lines on git
`git add README.md`
`git commit -m "Add README.md"`
`git push origin main`

### Working with repository on Git
Clone it on yout local computer
First, create a different directory (e.g. git_tutorial_new) in a separate location that is not already tracked by Git and `cd` into it (open gitbash in that directory). Now we can download the repository we just uploaded using the following:

`git clone [e-mail]:user_name/repository_name.git .`
The dot in the end is very important!This will put the clone into the current directory, instead of creating a new directory with the same name as the remote repository. You will see that all your files are here, identical to the original `git_tutorial` repository!

`git remote -v`
This will show you what remote location is connected to your local Git clone.

Change things in you second repository, add, commit and push the change to the remote/github repository. Now the remote repository is the same as your second repository, but not the first repository.

### Remote branches
Make a new branch named `trimming` in your first repository,
change another file in your first repository, add, commit.

Then push the `trimming` branch into remote.
`git push origin trimming`

We now have two branches both locally and remotely: `main` and `trimming`.

####################################################################

### git pull
https://stackoverflow.com/questions/1709177/pull-a-certain-branch-from-the-remote-server
Go back to yout first local repository. You want to update it.
`cd` back into the first local repository (e.g. git_tutorial) and run the `git pull` command.

**Note that sometimes before you `git pull` in your first repository, you need to add and commit unstaged changes in the first repository.**

**Note if there is a conflict in some places between your first and the remote repository, after your `git pull`, you need to fix that issue in your file, add, commit and push.**

**Note that the git pull will not automatically changes your folder and file if you local is different from remote. If your local is ahead of the pulled remote, You need to `git push` after git pull.**

`git status`

Another command is `git fetch`, which will download remote changes without merging them. This can be useful when you want to see if there are any remote changes that you may want to merge, without actually doing it, such as in a collaborative setting. In fact, `git pull` in its default mode is just a shorthand for git fetch followed by `git merge FETCH_HEAD` (where `FETCH_HEAD` points to the tip of the branch that was just fetched).

## How to pull from a remote branch to a local main/branch

Sometimes there are issues:
Imagine senario, You have local_branch and remote_branch. You push the local_branch to remote_branch. When you pull it still works

Then you change smth in the local_branch, when you pull, it does not work anymore, fetch also does not work.

## Remove remote branch
`git push origin --delete trimming`

## sharing tags
`git push --tags`


## 1.7 Conflicts
Imagine scenario, there is local_main1 and local_main2, remote_version1.
Local_main1 is based on remote_version1 and made some changes and pushed to generate remote_version2.
Local_main2 is based on remote_version1 and is not aware of the change, Local_main2 then make changes at the same place but changes differently.
When Local_main2 pushed it to remote, it will conflict with remote_version2, an error will occur.

In this case, download the changes but without merging them directly
`git fetch`
The `fetch` command is very similar to `pull` in that it downloads remote changes that are not present locally, but differs in that it doesn’t try to `merge` them locally; `pull` both downloads and merges (unless there’s a conflict, in which case it will tell you so and raise an error like the one above). You can thus skip `fetch` and just do `pull` straight away, if you prefer and there is no error

If the error occurs
Run this to see the difference, when you are in local_main
`git diff origin/main` (this indicate the remote main)

If you then run `git merge`, will lead to a conflict.

What you should do is then: Resolve the conflict, add, commit and push.


## 1.8 Extra
https://uppsala.instructure.com/courses/96483/pages/git-9-extra-material?module_item_id=1027808
### Amending commmits
To do this, the prerequisite is that you have make a first commit but has not pushed your commit. You then have made you second change

`git add <file>`
`git commit --amend`

This will make your second commit into your first commit.

### Rebasing
This will make the history a bit messy


Then


### Rebasing as clean up
`git rebase -i HEAD~4` Search back 4 commits
Don't use it so far, it is a bit messy

**In your text editor**
Switch to edit mode by pressing `I` to be able to edit this file.

Press `esc` to exit edit mode and type `:wq` to save the file. Note: If you made changes to the file that you do not want to save, type `:q!` to force quit

Note if it gives error at some point, in youtr git terminal, use `git rebase --abort` to abort the change.

### The reflog
`git reflog`
It shows all the changes yo make in the repository.
The commits are ordered with the most recent one at the bottom.

### Decorating your prompt
https://uppsala.instructure.com/courses/96483/pages/git-9-extra-material?module_item_id=1027808
### Bash aliases for git and petty logs

# 2. conda
Conda is a package and environment manager. As a package manager it enables you to install a wide range of software and tools using one simple command: `conda install`. As an environment manager it allows you to create and manage multiple different environments, each with their own set of packages

## 2.1 The basics
In the Linux subsystem, create a project
`mamba create -n project_a -c bioconda fastqc`
This will create an environment called `project_a`, containing FastQC from the Bioconda channel

`mamba activate project_a` activate the environment:
`mamba info --envs` To see all your environments you can run:

`mamba list` To see the installed packages and their versions in the active environment, run:

`mamba env export --from-history > environment.yml` To save the installed packages to a file, run:

`mamba deactivate` Then only the base environment is activated

### Adding more packages and enviroment
After project_a is activated
`mamba install -c bioconda multiqc` To install another package

`mamba search -c bioconda multiqc` Run the following to see what versions are available:

`mamba install -c bioconda multiqc=1.13` Now try to install a different version of MultiQC, e.g.

Make a new environment
`mamba create -n project_old -c bioconda bbmap=37.10`
`mamba activate project_old`
`bbmap.sh --version`

### Removing packages and environment
`mamba remove multiqc` To remove an installed package from the active environment

`mamba env remove -n project_old` To remove an environment

`mamba clean -a` To remove unnecessary files with the command

## 2.2 projects
Create a yml file in the directory

`mamba env create -n project_mrsa -f environment.yml`

Note that instead of the `-n` flag you can use the `-p` flag to set the full path to where the Conda environment should be installed.

`mamba activate project_mrsa`

`bash code/run_qc.sh`

`ls -Rlh` To check out directory content

### keeping track of dependencies

There is one command that can make this easier:` mamba env export`. This allows you to export a list of the packages you’ve already installed, including their specific versions, meaning you can easily add them after the fact to your environment file. If you use the `--no-builds` flag, you’ll get a list of the packages minus their OS-specific build specifications, which is more useful for making the environment portable across systems.

## 2.3 extra
https://uppsala.instructure.com/courses/96483/pages/conda-4-extra-material?module_item_id=1027812
### conda configuration

### managing ython versions

### decorating prompt

### bash aliases for conda

### rolling back to an earlier version of environment

# 3. Snakemake
depriortized in learning
https://uppsala.instructure.com/courses/96483/modules
# 4. Nextflow
https://uppsala.instructure.com/courses/96483/modules
# 5. quarto

# 6. jupyter
depriortized in learning
https://uppsala.instructure.com/courses/96483/modules
# 7. contianers

# CI/CD

# Steal the way how scilifelab built their "sophisticated" website!
This is the template
https://github.com/NBISweden/workshop-reproducible-research/tree/main