# PC987 Day 3

## Recap Day 2

- git clone
- runing bash scripts/commands
- status of a process: running, waiting, zombie
- moving a process in background, foreground, suspend
- kill a process by PID
- cron job

# Day 3

## Running Python scripts for file management

- copy, move, delete files
- get a list of files in a folder
- handle folders: make folder, listing file inside

### Example: automate updating webpage

Scenario: You have a folder containing some source files for your personal webpage. When you update your webpage, you need to compile and **copy** into the publishing directory.

task:

- manual prepare example of static webpage: mkdocs
- for copying and listing of the directories and files, we use `shutil` and `pathlib`

#### Codes for the tasks

- In Anaconda Powershell

cd Desktop\Day3
pip install mkdocs
mkdocs new myhomepage
cd myhomepage
mkdocs build
mkdocs serve

the last commands `mkdocs serve` will create a temporary web server at `localhost:8000` (127.0.0.1:8000). Open a browser to view your home page. To stop the server, press `Ctrl-C`.

Now we want to do this in the Ubuntu Server

- configure your Ubuntu Server for port-forwarding: forward guest port 80 -> to localhost port 5555
- start the server
- ssh into the server

$ sudo apt install apache2
$ sudo systemctl status apache2  # check status, press 'q'
$ sudo a2enmod userdir# enable user to have their own webpage
$ sudo systemctl restart apache2
$ mkdir public_html   # this is in your home directory

Open your browser and open the page at 127.0.0.1:5555 to see the message from apache2.

Create a file `index.html` in `public_html` as your webpage. You can view this page at http://127.0.0.1:5555/~dummy/

Now, activate your python virtual environment and install mkdocs.

$ mkdocs new myhomepage
$ cd myhomepage
$ mkdocs build   # your compiled website is at 'site'
$ cp -r site/* ~/public_html/    # copy files

Back to local windows machine. Start `spyder` and create a file `deploy_web.py`, to automate the copying of files.

Here is the listing of the deploy_web.py

# FILE: deploy_web.py
import pathlib
import shutil
import subprocess
import os

# objective: copy all files in homepage/site to public_html
src = pathlib.Path('myhomepage')
#src = pathlib.Path('website')
dst = pathlib.Path('public_html')

assert src.exists(), "Make sure your mkdocs folder is called myhomepage"

site_path = src.joinpath('site')
print(site_path)
try:
    os.chdir(src.name)
    cwd = pathlib.Path('.')
    cmd = 'mkdocs build'
    subprocess.run(cmd, shell=True)
    os.chdir(str(cwd.absolute().parent))
    shutil.copytree(site_path, dst, dirs_exist_ok=True)
except:
    raise Exception("copytree failed")

print("All done!")

Then, we need to copy our python scripts to the server. To make life easier, we write a python code to automate the copying of all python files to the server. Let's call the script as `scp2remote.py` and the listing is below:

# FILE: scp2remote.py
import subprocess
import pathlib

cmd1= 'ssh -p 2233 dummy@127.0.0.1 "mkdir py_files" '
cmd2 = 'scp -P 2233 -pr "{fname}" dummy@127.0.0.1:/home/dummy/py_files/'

subprocess.run(cmd1, shell=True)
pyfiles = pathlib.Path('.').rglob('*.py')
for fname in pyfiles:
    print(cmd2.format(fname=fname))
    subprocess.run(cmd2.format(fname=fname), shell=True)
    
# You can also use rsync to do this

With this scripts now you can edit your code locally and later transfer to server when you want to use the scripts in the server.

Finally, you can run the `deploy_web.py` every time you update your `docs/index.md`, and no need to go through build-then-copy process.

Similar idea you can use to push your source files to git repository. Just add the following code in your local git repository. Whenever you want to push your files to remote server, just run the script

# FILE: git2repo.py
import subprocess
import sys

try:
    comment = sys.argv[1]
except:
    raise Exception("Usage: python git2repo.py <commit comments>")
    
cmd1 = 'git add -A'
cmd2 = f'git commit -m "{comment}"'
cmd3 = 'git push origin master'
subprocess.run(cmd1, shell=True)
subprocess.run(cmd2, shell=True)
subprocess.run(cmd3, shell=True)

If you have setup key-based git authentication, you will have the entire push process in one single command, not even asking your password.