Try   HackMD

ARCHIVE Welcome to Linux Shell Scripting 2022

tags: Linux Shell Scripting

This is a hybrid course.

Remote SSH to a native Linux

Day 1

Introduction

  • Welcome to the HackMD for this course, this is like "google doc" but optimized for sharing bits of code.
  • Everyone can write here, this is the place where you can write your questions / comments / etc
  • This is much better than the zoom chat because we can have threads and then save this document for your reuse
  • Test it here below!
  • Write your question like this
    • an answer by someone who knows
    • A comment with a bit of code pasted
    • here longer code:
    ​​​​#!/bin/bash
​​​​whoami
​​​​hostname
​​​​str="hello"
​​​​echo $str
  • another question here
    • another answer
  • ..
  • Blah
  • Seems to work
  • .
  • …
  • Howdy!
  • …
  • testing

Presemo ice-breakers

Quoting, substitutions, aliases

Materials: https://aaltoscicomp.github.io/linux-shell/quoting-substitution-aliases/

  • I entered bash in my command line bash, but still I am getting zsh in response to echo $SHELL.
    • Could you try /bin/bash ? and then enter
    • yes, but the same result!
      • On which machine are you doing this? Some shared ssh server? (e.g. at Aalto kosh)
    • no my local machine Ubuntu 18.04
      • Try exec bash Does it help?
    • no:(
      • try chsh -s /bin/bash and then open a new terminal (it changes your default shell to bash)
    • Connection timed out at /opt/bin/chsh line 67, <DATA> line 755.
      - chsh will change the default shell, so if you're running on e.g. OSX, changing it might not be a good idea. On Aalto linux machines bash is recommended. You can also run SHELL=bash bash if you want to change the $SHELL-variable for this process only. Simo
  • …
  • With cp ~/bin/doinit ., I get No such file or directory. Should i create that myself?
  • What did the $(command) do?
    • Command substitution, compare the variable today when running today=date versus today=$(date). I don't wanna spoil the answer on presemo :D
    • Does it work like this: the bash executes the command inside the parenthesis and then uses the output in place of the $(command)?
      • Pretty much yes. Simo
  • Can you turn on 'edit' in presemo?
    • I will remind the instructor during the break
    • I misclicked the wrong answ =)
  • What is -s doing in $(hostname -s).txt?
  • Is it still possible to get the 1 credit if I have to leave at 14:15 on mondays?
    • We don't track presence so you can leave if you have other overlapping events. Ivan (the instructor) will tell later about the homeworks needed. I will send an email reminder about credits+homeworks (enrico)
    • Homeworks (=special assingments) are coming, yes, thet I expect at 75% of tutorials to be done. One can watch the youtube vides later though.
  • Are warnings also default to STDERR?
    • Usually they are sent to STDERR, but it depends on the tool you use sometimes.
    • Bash commands have warnings to STDERR, but I cannot find a reference for this, so I will have to google better (enrico)
    • That depends, depends on the command. In general yes, all non-standard ouput is going to the STDERR (filedescriptor 2). But depends on the programmer.

See you at 13:45

Exercise 2.1 20mins, at least first 4 bullet points
https://aaltoscicomp.github.io/linux-shell/quoting-substitution-aliases/#exercise-2-1

  • Credits (1 ECTS or even 2 if you are very motivated)

    • Tell Enrico that you want the credit using whatever media you prefer (direct zoom message, email, etc)
    • We don't track attendance but we expect that you are watching this / watch the videos later.
    • There will be a "special assignment" for those who want the credit (1 ECTS = about 27 hours of work minus 12 hours of lecture time so about 15h)

  • the comparison: is just the file names or it looks into the file content to comapre if they are the same?

    • Are you referring to the diff example? Diff compares the content of txt files. In the example diff <(ls dir1) <(ls dir2) the part (ls dir1) is like a file so that ls dir1 > fileA.txt. So it would be like comparing two txt files diff fileA.txt fileB.txt created with the commands surrounded by ( )

  • Why does echo $({1..5}) not work?

    • inside $( ) you want to have a command, but {1..5} is not a command. The dollar+roundbrackets does command substitution
    • $( ) expects a command, while {1..5} gives you a string like 1 2 3 4 5; echo {1..5} works as it shoud

  • "Use the example in the text above to send du -hs * .[!.]* | sort -h output to yourself." What do you mean to yourself? To STDOUT?

    • I guess "send the output as an email to yourself", Ivan will confirm.
    • I meant 'mail …', but yes, incorrect phrasing.

  • When trying to execute u -hs .[!.]* * | sort - I get error message cannot access .[!.]*: No such file or directory

    • +1
    • it means that in the folder where you are there are no files or subfolders with name starting with dot ".". In your home folder you usually have multiple ".filename" files (hidden files/folders). In the folder where you get this error, try creating an hidden file for example with touch .hiddenfile.txt and re-run the command again and you will see that it won't give the error anymore.
    • alternatively to the answer above you can 2>/dev/null, overall yes, du can't find any file that starts with a dot (hidden files) and complains

  • What does the end of the mailing example mean: ; mv listing listing.$(date +"%Y-%m-%d-%H-%M");? I assume that in the beginning the command creates temporary file named listing and then it is used later in the command?

    • yes, it moves (=renames) the file name 'listing' to 'listing.2022-03-21-13-53' or alike; good for keeping logs etc

  • I feel that I am repeating myself in the diff task, namely using stat -c "%y" filename twice, is there anywya to avoid this?

  • Testing mail gets stuck asking for cc for some reason, I tried running mail -s "this is the subject" "my address"

    • yes, depending on where you run it, you might actually hit some security block that stops any emails from terminal
      • Im on kosh via ssh
        • I think kosh has mail from terminal blocked. Or did you manage to send it?
          • No, I get cannot send message: Process exited with a non-zero status
            • Yeah we cannot change that. Maybe we need to re-think the exercise.

  • I'm stuck in trying to test whether a directory exists, what commands should I use? echo $(path) and redirect it to /dev/null ? Is echo ls $(path) > /dev/null && echo exists || echo "doesn't exist" close ?

  • Can't mail myself, my system does not allow it:

Command 'mail' not found, but can be installed with:
apt install mailutils
Please ask your administrator.

  • ^^^ Most likely you are on a remote server, for example at Aalto kosh they blocked use of "mail" from the terminal. This change happened quite recently, maybe we need to re-think this exercise :)

  • It would be good to see the exercise solution also here: https://users.aalto.fi/~degtyai1/shell/bash_history.2022-03-21

  • I get this:
    stat: illegal option – c
    usage: stat [-FlLnqrsx] [-f format] [-t timefmt] [file …]

    • Most likely you are on OSX (Mac) and the "stat" command on Mac is not identical to the one in Linux.

Variables, functions, environment

https://aaltoscicomp.github.io/linux-shell/variables-functions-environments/

  • In the var=26_file.ext; echo ${var%.[a-z][a-z][a-z]} why three of the [a-z]'s?

    • each [a-z] is a placeholder, it says here is a letter, any of a, b, c,
      • But 1 or 2 does not just take away 1 or 2 of the letters

  • ..

  • ..

  • Will the recordings be in youtube before next class? or at teh end.

    • Yes enrico will upload to a google drive between us, edited recordings will go to youtube later.

  • Next class is on Friday right? (I thought I heard you say Wednesday)

    • Friday indeed.

Feedback

Write anything that comes to your mind! What was good? What can be improved?

-In addition to "It's ok, i can manage", I felt that it was a bit fast though. If I missed something, it moved forward quickly. Luckily there will be the recordings and the scripts available.

  • How can I get credits?

Day 2

Any question from last session?

  • How can I get credits?
    • You will need to complete a special assignment to obtain 1 ECTS credit (2 credits if you are very motivated). Please email enrico.glerean@aalto.fi to obtain the special assignment.
  • Is it only for Aalto students or others as well?
    • For Aalto students: if you have a student number we can register the credit for you, most likely it will look as "Individual Courses in Computer Science"
    • For other universities: we write a certificate where it explicitly says that this is equivalent to 1 ECTS, and your supervisor or course coordinator will register it to your system. Please note that not all universities/supervisors might accept this, usually this is fine for Doctoral students, but non-Aalto master students should better confirm with their responsible person.
      • Cool thanks :)
  • I recive this error while running spaceusage:
    du: cannot access '.[!.]*2': No such file or directory
  • ..
  • …Can we search from a database find details of some identifiers and divert them into a text file using Bash?
  • ..

Recordings which might have your voice or username are only shared internally with the participants. If we upload to youtube, then no voices / names / etc will appear there.

Functions

https://aaltoscicomp.github.io/linux-shell/variables-functions-environments/#functions

  • question you can write hre…

    • answer

  • We can see the screen.

  • Is this link still valid today? "https://users.aalto.fi/~degtyai1/shell/"

    • yes, you see the history of today for example == the commands that Ivan is typing

  • Why is it source functions instead of source ~/bin/functions

    • source looks for the specified file on the path and by default the ~/bin/ folder is on the path
    • do we need to go to the ~/bin directory and create function or it could be made in home directory?

  • I am still getting the same output as the presenter, meaning host part has not been recognized even I added the echo. (m3:domain: command not found)

    • can you post the me() command from your functions file? And: Did you source functions again? Otherwise your modified function will not get loaded
    • Thanks, I did not source it again:)

  • What does me stands for?

  • You did not source function this time. Was that intentional?

    • I assume not.
    • Nope, one has to source the file time every time you make changes to a function

What should the output be for filepath function? If I add a file as argument I just get that filename back.

  • I recive this error while running spaceusage:
    du: cannot access '.[!.]*2': No such file or directory

    • Most likely you do not have any file/folder starting with ".". So it is not an error, just the fact that there are no files/folders with name ".something".
    • There is also an extra "2" which should not be close to the "star" otherwise it tries to find a file named ".something2", see below
    • make sure your syntax is correct du -hs ${1:-.}/* ${1:-.}/.[!.]* 2>/dev/null | sort -h

  • In zoom we cannot hear what you talk about in the in-person room, so if the questions are interesting please repeat them. Thanks :)

    • Good point, yes.

  • what is mean by path/to/
    -filebasename path/to/archive.tar.gz what should i put in instead of path/to/

    • The location or folder where the tar.gz is located. For example, if your tar.gz is in a folder Downloads in your home folder:
      ~/Downloads/archive.tar.gz (The path can be absolute, such as ~/Downloads/archive.tar.gz or relative: ../DOwnloads/archive.tar.gz)

  • Okay, pretty clear, thanks. One more question, is there any special function that works like grep to find the files' location?

    • There are a few options.
      • If you know the name of the file, you can try if locate command is installed in your computer. It is very efficient.
      • If you know the file is in a specific smallish sub folder (a small search space) because find can be resource-hungry.
      • If you want to know where a command file is, eg. what is ssh command? which ssh would work
        • Okay, noted them all. Thanks!

  • why word variable put into "{}"? cannot be only in form of $word?

    • sorry was confusing parenthesis and braces.
    • mine is working without this ${} as well, what is the problem?
      • ok, so, the {} braces in this instance (i.e. calling a variable in braces) is only for separation. e.g. if you want to have ${word}ing to refer to the environment variable word and add ing instead of $wording where you would refer to the environment variable wording

  • why at end of each line &&\ has been added, mine is working without this!

    • \ as the last character of a line is line continuation. Bash then interprets that whole paragraph as a single line
    • && is… I'll give an example: AA && BB. Execute AA, and if AA is succesfull, then execute BB.

  • Are we (non-aalto ones) also supposed to have triton account for the course?

    • No triton access provided. Any linux will do.

  • In the fork-bomb function, why the functions output has to be piped to itself? Would not it be sufficient just call the function and send it to background like this: { bomb $; }

    • I imagine the key is to do this recursively, so that no bomb can ever return.

  • Why would we want to kill the system? Why would that be needed? If there is overload. I might have missed some explanation, but still seems unclear to me.

    • I think it is mainly a funny example, but sometimes you need to stress test systems.

  • whatever file i type it says exists:
    $ [[ -filesadfasdfasd ]] && echo file exists
    file exists

    • Are you missing a space or having too many?
      [[ -file- ]] double checked it, seems okay
    • It should be [[ -f filename ]]

  • Some links to learn these regex more would be great.

  • recieving this error even tho i created the file with nano: chmod: cannot access 'if-test.sh: No such file or directory

    • Could you paste the whole command line, please? It really looks like you have a typo in the filename or forgot to save the file in nano or something similar.

  • I got this:
    Can't add archive to itself(base)

    • Could you paste the command you are giving, please?
      • ./tarit.v1.sh ../../linux_shell_scripting_spring_2022
    • And also your script…please?
Click to expand the script 
#!/bin/bash
# usage: tarit.sh <dirname>
d=$1
# if dirname is given, we archive it
if [[ -d $d ]]; then
  tar caf $(basename $d).$(date +%Y-%m-%d).tar.gz $d
elif [[ -z $d ]]; then
  tar caf $(basename $(pwd)).$(date +%Y-%m-%d).tar.gz .
else
  echo $d does not exist
fi
  • Hmm. It is not obvious what's the problem. If I needed to debug this, I would add to the script a line that prints what it is planning to do:
Click to expand the script 
#!/bin/bash
# usage: tarit.sh <dirname>
d=$1
# if dirname is given, we archive it
if [[ -d $d ]]; then
  echo tar caf $(basename $d).$(date +%Y-%m-%d).tar.gz $d
  tar caf $(basename $d).$(date +%Y-%m-%d).tar.gz $d
elif [[ -z $d ]]; then
  echo tar caf $(basename $(pwd)).$(date +%Y-%m-%d).tar.gz .
  tar caf $(basename $(pwd)).$(date +%Y-%m-%d).tar.gz .
else
  echo $d does not exist
fi

  • Or, replace the shebang:
    #!/bin/bash -x that does effectively the same thing. Bash will write out what it does.

  • please could you also show the script for this (again) if bin was not part of path variable?

    • it worked without doing that. So forget this question.

  • I followed all steps but i am still getting "cx: command not found"

    • inmy case (=lecturer) I have created demospace/bin directory and have added the directory to $PATH. That is the way to get 'cx' or any other executable placed in that directory to work anywhere. But one can still any executable from anywhere by 'chmod +x executable; ./executable'. With havin it in bin and 'bin' in PATH it makes
    • easier
      -Thank you

Day 3

​​​​## Getting started
  • The question was if the course pace is too fast, how exactly?
    • May be more explanations please
    • The tasks take a lot of googling and testing if you don't already know this stuff, so more time for tasks please

ex. 2.3

Loops

https://aaltoscicomp.github.io/linux-shell/loops/

  • questions here

  • …

  • .

  • if you use more exit then the same number 1 applies?

    • exit returns the exit status value of the last command
    • but you can choose any number 0..255 as a parameter. (0 is success, others are error codes whose definition should be documented.)

  • Why are the results different when using
    - n=10; ((n++)) then echo $n and
    - n=10; echo $((n++)) ?

  • Could you do the scripts on demospace again? I think you are on triton now?

    • Thanks

  • The parameters or looping variable contents in the for loops: in your examples they have been "horizontal vectors", space separated. When or how should one use new-line separated lists?

    • Newline delimiter also works. Good point. I must confirm by testing, but all the space, tab, newline etc delimiters should work within the loop items list.

Excercise 2.4

Please ask for assistance! :-)

  • Q1
  • Q2

This is the end of the document, WRITE ABOVE THIS LINE ^^

HackMD can feel slow if more than 100 participants are editing at the same time: If you do not need to write, please switch to "view mode" by clicking the eye icon on top left

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Last changed by 
 
AaltoSciComp
0
209

Read more

Kickstart-2025-{2,3} (archive)

Programming langages you use? (poll, add o to vote):

Jun 5, 2025
Kickstart-2025-1 (archive)

To watch: https://www.twitch.tv/coderefinery

Jun 5, 2025
LLM Examples

materials:

Jun 5, 2025
2025 summer kickstart - triton tutorial notes

This is the notes from the

Jun 5, 2025
Read more from AaltoSciComp
Published on HackMD