owned this note   changed 4 years ago  
Published Linked with GitHub

Welcome to Linux Shell Basics 2021

tags: Linux Shell Basics

This is a hybrid course.

For the remote SSH to a native Linux

Day 3

Going over exercises 1.3

https://aaltoscicomp.github.io/linux-shell/file-archive-transfer/#exercise-1-3

  • Why does the command find ~ -type f -perm /r+w,r+r needs the / character?

    • Check the find manual page for the -perm option. There are different prefixes that mean different things: "all bits set", "any bits set", …
      • In this instance, the / makes find list all finds that have "at least read and write permissions", without it would not list those that have additional flags/bits set.
      • find perm {mode} is exact, find -perm -{mode} is "all", find -perm /{mode} is any. I would never remember this.

  • Running find ~ -type f -perm /o+w results in find: -perm: /o+w: illegal mode string

    • what OS are you on?
      • I'm on Unix (MacOS).
    • There are slight differences between the utilities included in different unices. It is possible that these symbolic mode strings don't exist on others. Linux (with GNU tools) is usually one of the most modern)

  • Cannot access Ivan's commands history. Says Access denied by the owner of the requested resource.

    • Image Not Showing Possible Reasons
      • The image file may be corrupted
      • The server hosting the image is unavailable
      • The image path is incorrect
      • The image format is not supported
      Learn More →

Command line utilities

https://aaltoscicomp.github.io/linux-shell/cli_utiltities/

  • What are you favorite unix utilities?

  • What is the difference between less and cat for checking the contents of a file?

    • cat prints straight to stdout, a one-shot thing
    • less is an interactive pager that lets you scroll up and down, search within, limit lines shown, etc. less is what we call a pager: shows files in pages.

  • Can you sort the contents of the file?
    When you cat

    • sort: sort filename is the same as cat filename | sort
    • I mean contents of the file inside the file
    • That does it. It writes it to standard output, which you then need to save back.
      • warning: cat filename | ... > filename will overwrite and remove contents from the file, since it opens it for output before it reads it in!
      • sponge (non-default, I think in package moreutils) can make this better
  • Can you also use 'tr' to replace the spaces in a file name?
    • In a filename, that's interesting. For filenames I would use the rename utility, it lets you replace expressions in filenames.
    • When renaming files, I also often use vidir (non-standard, package: moreutils). It opens an editor with filenames so you you can apply an editor's search-and-replace or other tools to do renames.
    • you need to put filename into a variable and then use either tr or a ${var/ /} construct to rename file with mv. We go through such cases in the Part 2.
  • I wonder how I capture the name of the file in a loop and save it somewhere? ie. unzip -j 1st_specimen.zip target.png and I want to rename the target.png to 1st_specimen_target.png automatically after decompressing it? I dont know how to pipe this…
    • You would probably need a script for this, as the output file is not something that goes to the std:out.
      • actually, unzip has an option to extract to the stdout (-p). So what you would do is: unzip -j 1st_specimen.zip target.png > 1st_specimen_target.png

Break until xx:08

  • How does one access other disks, like USB-sticks?

    • This is what I do (perhaps old-fashioned, but I do almost everything from the shell and want full control): plung in the USB drive. pmount /dev/sde (pmount is another tool to install that mounts removable media, /dev/sde is a sample name for the disk. it automatically mounts it at /media/sde). Then access via /media/sde/.
    • There are also things that automatically mount it.
    • Or I access it via the file browser, I browse to it, open it, then right click and "Open in terminal". This gives me a much longer, harder to use path.
    • What about others?
      • I commonly don't need to do anything on a usb besides file transfers, and those I will do via the GUI (but then I haven't yet gotten huge files via usb, where e.g. copying by rsync would be a better option (due to checksums etc)

  • du -hs * .[!.] | sort -h
    du: cannot access ‘.[!.]’: No such file or directory

    • I remember this expressions is for hidden file?
    • Yes… there are no hidden files there +1
    • there is a missing star at the end, it should be
      du -sh * .[!.]* it will list the space taken by normal files/folders (*) and files/folders that start with a dot (hidden files)

Pipelines and grep

https://aaltoscicomp.github.io/linux-shell/pipelines-grep/

Break and exercise until xx:31

Exercises 1.5: https://aaltoscicomp.github.io/linux-shell/pipelines-grep/#exercise-1-5

  • make a pipe that counts number of files/directories (including dot files) in your directory
    • ls -1A | wc -l
  • grep directories out of ls -l
    • ls -l | grep -v ^d
  • grep all but blank lines of the ‘man cut | grep …’
    • man cut | grep -v '^$'
  • Using pipes and commands echo/tr/uniq, find doubled words out of My Do Do list: Find a a Doubled Word.
    • echo 'My Do Do list: Find a a Doubled Word' | tr -s ' ' '\n' | uniq | tr -s '\n' ' '; echo
  • If you are on a multiuser system, count unique logged in users. Tip: w or users gives you a list of all currently login users, many of them have several sessions open. Commands to discover: cut / sort / wc
    • w -h | cut -d ' ' -f1 | sort -u | wc -l
  • (*) Play with the commands grep, cut: find at least two ways to extract IP addresses out of /etc/hosts. Tip: grep has -o option, thus one can build a regular expression that will grab exactly what you need.

  • can we show this in a tree format?
    root and branches

    • there is a linux tool called tree but it might not be installed.

  • Will this Q&A material also be available later? I think it has been very insightful.

    • Yes, we generally archive it. In this case we'll probably leave it on hackmd read-only.

  • .
    Thank you!

  • I think this never got answered (was in the day 1 hackmd)
    I started mathematica from bash today and when I was done I tried to shut it down from the bash as well. When I started it from terminal the cursor moved to an empty line which implicated I was not able to type any commands. So I killed it with ^C and the the cursor moved to the next line and I was able to type commands again but the mathematica was still up and running? Also tried to send it to background and kill it there, the terminal said it was terminated but it still was running?

  • I have never used mathematica
    Another question about this, why is it not possible to type commands after starting a program? The cursor is just blinking on an empty line. Is the idea to send it to the background if one wants to continue working in the same terminal?

    • Correct. For example I could start matlab with matlab & and continue using that terminal for other things. interestingly, if error messages come up, they might also show in the terminal window (it depends on the sofware thoughghghghghghghghghghghgh)

  • .

Feedback

One good thing

  • Thanks for the course! I believe I will be able to use Bash for simple operations without extra googling. As for non-linux user, the two and third lessons were quite intense, but I apprreciate we are provided with enough materials and the records.
  • The topics of the course were very punctual, with punctual demonstration of how the commands work and what it is possible to do with very simple inputs. The course also provided the opportunity to practice in both a basic level user as well as gave more "challenging tasks". It was also greatly appreciated that these tasks were solved and explained by the demonstrator and that not only did he explain the outcome of each command but also gave a little bit of background of what it contained or its "definition". In a more informal level, I want to thank you for these course, it was really useful and mind-blowing. If part 2 is also open, I am definitely attending.
    Image Not Showing Possible Reasons
    • The image file may be corrupted
    • The server hosting the image is unavailable
    • The image path is incorrect
    • The image format is not supported
    Learn More →
  • . Thanks a lot for this nice introductory course! I got started using Bash shell and looking forward to other courses. The efficient way to ask questions and giving answers via hackmd was smart, also quiz by Presemo was very good! I also liked very much going through the exercises and solving the probles once everyone is done with the exercise. Having two people one for delivering the lectures and one to take care of the other practiclity was also another strength!
  • .
  • Thanks for the course! Especially this last time was very good, all the stuff seemed important for future use (especially piping)

One thing to be improved

  • .
  • At least for me, it helps to remember the commands if I know what they are abbreviations of. So you might try to use them more often (e.g. not always saying just du, but 'disc usage')
  • .
  • Your explanations are good, but it feels like it's a list of different comments without a common narrative, and therefore a bit hard to actually understand and remember all of it. Maybe it would be good to prepare a repository with example files prior to the course and then share it with the participants, such that everyone can actually execute the exact same commands and see what happens
  • It might be good to add some section about why Linux/shells/command lines work like this. We now know how to use it, but I still feel like I'm missing the background or the underlying reasons why that's the case. Might be a subject for an entirely separate course though :)
  • I also agree with the previous comment that a prior material would be a good idea for upcoming courses. Not necessarily on the historical aspect but I think it is easier to execute multiple (an perhaps more complex) commands only after you have understood what is actually happening and what do they stand for (as stated in the abbreviations comment).

**
Thanks for the course! Lot of information in three days and excellent course materials to keep practising by own**

This is the end of the document, WRITE ABOVE THIS LINE ^^

HackMD can feel slow if more than 100 participants are editing at the same time: If you do not need to write, please switch to "view mode" by clicking the eye icon on top left

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Select a repo