# GRChombo - docker tutorial

### Set up Docker
An alternative guide on youtube for the first few steps: https://www.youtube.com/watch?v=6xOZxA_Oz-I&t=11s&ab_channel=GRChombo

* Install docker from https://www.docker.com/get-started if you have not already done so
* Create a new folder

mkdir GRChombo_docker 
cd GRChombo_docker

* Get the GRChombo docker container

sudo docker pull grchombo/grchombo 

> Docker might require sudo (Root premissions). Alternativley use "sudo usermod -aG docker $USER" and restart shell
and enter it

sudo docker run -v "$(pwd):/settings" -it grchombo/grchombo 

For Windows powershell:

docker run -v /d/path/to/my/folder:/GRChombo/Examples/BinaryBH/hdf5 -it grchombo/grchombo

>Please note that changes get lost when you exit the container. Best open two terminals, one to work within the docker container and one for outside.

### 1. Running GRChombo

1.1. After entering your contrainer you show see these three folders

BinaryBH  KerrBH  ScalarField

1.2 Enter the BinaryBH folder, and you can run the executable

./Main_BinaryBH3d_ch.Linux.64.mpicxx.gfortran.DEBUG.OPT.MPI.OPENMPCC.ex params_very_cheap.txt

> :fire: This will make your computer sweat a bit :fire:

1.3 Kill the job after a few seconds. There should be enough data for your to visualize parts of the insprial

1.4 Move your data out of the current container

mv hdf5 /settings

> At the beginning, we defined /settings and mounted it to the local folder you started your docker environment in

### 2. Visualizing outputs using Python
2.1. Open a new terminal and go to the folder where you opened your docker. Your should see the folder

cd hdf5    

> If you have issues try to change permissions ```sudo chmod 777 hdf5```
2.2. We have python scripts to render the output files of GRChombo. Clone the repository from https://github.com/ThomasHelfer/quick_plot.

git clone https://github.com/ThomasHelfer/quick_plot.git

> This code is based on the open-source library https://yt-project.org/doc/ . This library has many fantastic tools to play around with GRChombo data.

2.3. Go to the script

cd quick_plot

2.4. Open a new virtual environment

virtualenv dev 
source dev/bin/activate

> Always use a virtual environment for python

2.5. Install required

pip install yt h5py matplotlib

2.6. Run the code

python parallel_pictures.py

You should expect
2.6b. If you have ffmpeg, you can make a movie from the frames available

ffmpeg -r 1/5 -start_number 0 -i chi/BinaryBH_000%03d.3d.hdf5_Slice_z_chi.png -c:v libx264 -r 30 -pix_fmt yuv420p out.mp4

#### Visualisation Challenge:
Use and download the GUI software https://visit-dav.github.io/visit-website/releases/ and visualize the data interactively.

### 3. Changing GRChombo

In this example, we implement a topological domain wall on a Black Hole. This example is only here to show how to modify the code and should not be used as an example for any science since Hamiltonian constraints are not fulfilled. For reference, we implement domain walls as described in "Cosmic Strings and other topological defects." by Vilenkin and Shellard in equations (3.1.2) and (3.1.3). All needed equations can be found in this tutorial.

3.1. Go to the ScalarField example

cd GRChombo/Examples/ScalarField

3.2. Open InitialScalarData.hpp with your favorite text editor (which should be vim) and change line 47 to

> Vim tip: Type ":set number" to see line numbers in vim
to the profile of a domain wall

data_t phi = eta * tanh( pow(lambda/2.0, 0.5)* eta * x );

and add before that all recommended values

data_t lambda = 10000;
data_t eta = 0.01;

and also get the x - coordinate from the coordinate class

data_t x = coords.x;

> NOTE: We use the template data_t instead of double type since that allows us to speed our code up using vector registers.

to check if all your changes worked, compile the code using

make all -j

3.3. For domain walls, we do also have to change the potential of the scalar field, which can be found in the file Potential.hpp.
Here we do have to change line 33

V_of_phi = 0.5 * pow(m_params.scalar_mass * vars.phi, 2.0);

and line 37

dVdphi = pow(m_params.scalar_mass, 2.0) * vars.phi;

to reflect the new potential mexican hat potential to

V_of_phi = lambda / 4.0 * pow((vars.phi * vars.phi - eta*eta) , 2 );
dVdphi = lambda * vars.phi * (vars.phi * vars.phi - eta*eta);

Also define lambda and eta as in the file before. Check if everything works by compiling

make all -j

3.4. We change some of the parameters in params.txt to be able to run in with few resources

N_full = 64
L_full = 32

this reduces the physical size of the box

max_level = 0

> Try to use m_level 1 if you feel your computer can handle it

which allows the code to use no levels of refinement, which makes our code fast to evaluate

hi_boundary = 0 0 0
lo_boundary = 0 0 0

we change the boundaries to be static, simplifying our problem

stop_time = 20

to make sure our simulation does only go on for a short amount of time

kerr_spin = 0.7

which increases the spin of the black hole, making the dynamics of the problem more visually interesting. Lastly, we add

kerr_center = 16 16 16

which sets the position (x y z) of the Black Hole. In this case, in the center of the grid.
> NOTE: We choose all of these parameters to make it feasible to run in a short timeframe. Play around and see the effect

3.5 Run the code and mv the data out of the container

./Main_ScalarField3d_ch.Linux.64.mpicxx.gfortran.DEBUG.OPT.MPI.OPENMPCC.ex params.txt

3.6 Modify the plotting script, change the variable you are plotting in line 37 in parallel_pictures.py

variable_names = ["phi"]

and remove line 43

center[2] = 0

> We delete this line because one simulation uses reflection symmetry and ones does not. Center positions the slice, if we delete this line it defaults to the box center.

3.6 We recommend checking out the outputs and compare with https://youtu.be/Uf4gyWxhzlU

#### Challenge:

* Make the scalar field massive by changing its potential and send a wavepackage through.

#### Coding Challenge:

* Make the code read the parameter lambda and eta from the params file, instead of hard-coding it

#### Hard Challenge:

* What happens if the black hole shoots through the wall?
> Never done this. Curious to see what happens there.