Lab Block 1.1. VMs with Vagrant.

--- tags: BigData-MS-2019, BigData-BS-2019, BigData-MS-2020, BigData-BS-2020 title: Lab Block 1.1. VMs with Vagrant. --- # Lab Block 1: VMs with Vagrant This tutorial guides you through creating your first Vagrant project. Vagrant is a virtual machine management software. It allows us to create instructions for automatic configuration and deployment of virtual machines at scale. The configuration of virtual machines is easy to understand. We start with this topic and later will move further to learn how containers can replace virtual machines. We will do the following: - Spin up a generic Ubuntu VM - Install Apache server - Perform port forwarding - Learn how to create a multi-machine environment - Connect multiple virtual machines with VPN ## Install Vagrant Download and install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) Download and install [Vagrant ](http://downloads.vagrantup.com) ::: success Linux Users: versions in the default repository of Ubunty 20.04 work without issues ::: Vagrant uses VirtualBox as a standard hypervisor. At least one hypervisor provider required to run a VM.   ## Setup up a Vagrant project from a repository: Go to https://app.vagrantup.com/boxes/search and select an image that you are going to spin up. We recommend choosing **ubuntu/bionic64**. You will see that you can configure your VM using only a single command ```bash vagrant init ubuntu/bionic64 ```   ![](https://i.imgur.com/Ho3D22e.png =350x) Create a where you want to store your VM and run configuration. Alternatively, you can pre-download a box image and use it to init your VM. ```bash vagrant init my-box /path/to/my-box.box ``` This will create `Vagrantfile` that contains configuration for your virtual machine(s) instance. You can view this file using text editor. ## Creating a VM Your VM is ready, and you can now ask Vagrant to start your VM: ``` vagrant up ``` The VM is now running in VirtualBox. You can ssh into it (no password required) as follows: ``` vagrant ssh ``` Further, you can log out by typing `exit` or pressing `Ctrl+D`. Even though you closed your ssh session, the VM is still working in the background. You can verify this by opening VirtualBox window. ![](https://i.imgur.com/JJIm94I.png =300x) To terminate or suspend your VM you can type `vagrant halt` or `vagrant suspend` correspondingly. If you want to delete your VM instance, type `vagrant destroy`. ## Synced Directory Note that on the new VM, `/vagrant` is a shared directory linked with the init directory on your host machine. On your host machine try :::info Bash required to execute the following command. On Windows, try using Git Bash, or look up how to perform equivalent operations using Windows Terminal. ::: ```bash echo "Hello World" > hello_world.txt ``` Then log into your VM and verify the presence of the file ```bash vagrant ssh vagrant@vagrant-ubuntu-bionic-64:~$ cd /vagrant/ vagrant@vagrant-ubuntu-bionic-64:/vagrant$ ls hello_world.txt Vagrantfile vagrant@vagrant-ubuntu-bionic-64:/vagrant$ cat hello_world.txt Hello World vagrant@vagrant-ubuntu-bionic-64:/vagrant$ ``` ## Vagrantfile The configuration of your VM is done with `Vagrantfile`. When you were setting up your VM, this file was automatically added to your directory. Take a look at the content. There are several useful configs: 1. `config.vm.box` specifies your VM image 2. `config.vm.provider` specifies your hypervisor and additional related settings. The default hypervisor is VirtualBox 3. `config.vm.network` defines network settings for your host to see your VM 4. `config.vn.synced_folder` allows to change default shared folder path 5. `config.vm.provision` specifies additional configuration ## Synced Directory To change the default shared directory path, simply modify the Vagrantfile ```ruby Vagrant.configure("2") do |config| # other config here config.vm.synced_folder "src/", "/srv/website" end ``` ## Provisioning Provision section of the Vagrantfile allows you to install additional software on your VM during its deployment. Let us create a simple example. Create `bootstrap.sh` with the following content in your init directory ```bash #!/usr/bin/env bash apt-get update apt-get install -y apache2 if ! [ -L /var/www ]; then rm -rf /var/www ln -fs /vagrant /var/www fi ``` As you can see, it installs `apache2` from the repository and swaps `/vagrant` for its server directory. To execute this script during deployment, modify Vagrantfile ```ruby Vagrant.configure("2") do |config| config.vm.box = "my-box" config.vm.provision :shell, path: "bootstrap.sh" end ``` Provisioning happens at certain points during the lifetime of your Vagrant environment: - On the first `vagrant up` that creates the environment, provisioning is run. If the environment was already created and the up is just resuming a machine or booting it up, they will not run unless the `--provision` flag is explicitly provided. - When `vagrant provision` is used on a running environment. - When `vagrant reload --provision` is called. The `--provision` flag must be present to force provisioning. You can also bring up your environment and explicitly *not* run provisioners by specifying `--no-provision`. ## Networking Vagrant allows to set up port forwarding to your VM. This will enable you to access a port on your machine, but actually, have all the network traffic forwarded to a specific port on the guest machine. Modify Vagrantfile according to the following ```ruby Vagrant.configure("2") do |config| config.vm.box = "my-box" config.vm.provision :shell, path: "bootstrap.sh" config.vm.network :forwarded_port, guest: 80, host: 4567 end ``` You need to perform `vagrant reload` after changing networking settings. Since you modified the default server directory, you need to create a directory `html` in your shared folder, containing a simple html file `index.html`. ``` Hello World! ``` Once the machine is running again, load `http://127.0.0.1:4567` in your browser. You should see a web page that is being served from the virtual machine that was automatically setup by Vagrant. ## Multi-Machine Vagrant can define and control multiple guest machines in one Vagrantfile. This is known as a "multi-machine" environment. These machines are generally able to work together or are somehow associated with each other. Here are some use-cases people are using multi-machine environments for today: - Accurately modeling a multi-server production topology, such as separating a web and database server. - Modeling a distributed system and how they interact with each other. - Testing an interface, such as an API to a service component. - Disaster-case testing: machines dying, network partitions, slow networks, inconsistent world views, etc. One of the simplest ways to create a multi-machine environment is ```ruby BOX_IMAGE = "ubuntu/bionic64" Vagrant.configure("2") do |config| config.vm.define "master" do |subconfig| subconfig.vm.box = BOX_IMAGE end config.vm.define "node1" do |subconfig| subconfig.vm.box = BOX_IMAGE # you can use different image here if you need end config.vm.define "node2" do |subconfig| subconfig.vm.box = BOX_IMAGE end end ```  To ssh into any of these, you will need to type ssh command with a name ```bash vagrant ssh master ``` To destroy all VMs, type ```bash vagrant destroy -f ``` ## Multi-Machine Networking To allow machines to communicate with each other, specify additional networking parameters. First, each VM needs a unique hostname. By default, each of the VMs has the same hostname (`vagrant`). Change this with ```ruby subconfig.vm.hostname = "a.host.name" ``` Next, we need a way of getting the IP address for a hostname. For this, we’ll use DNS – or mDNS to be more precise. On Ubuntu, mDNS is provided by Avahi. To install Avahi on each node, we’ll use Vagrant’s [provisioning feature](https://www.vagrantup.com/docs/provisioning/). Before the last `end` in the `Vagrantfile`, we’ll add this code block: ```ruby config.vm.provision "shell", inline: <<-SHELL apt-get update && apt-get install -y avahi-daemon libnss-mdns SHELL ``` This will call `apt-get install -y avahi-daemon libnss-mdns` on every VM. Last, we need to connect the VMs through a [private network](https://www.vagrantup.com/docs/networking/private_network.html). For each VM, we need to add a config like this (where each VM will have a different IP address): ```ruby subconfig.vm.network :private_network, ip: "10.0.0.10" ``` You can now call `vagrant up` and then ssh into any of the VMs: ```bash vagrant ssh my-box1 ``` From there you can ping any other VM by using their hostname (plus `.local` at the end): ```bash ping VM_addr.local ``` ## Multiple Provisioners Multiple `config.vm.provision` methods can be used to define multiple provisioners. These provisioners will be run in the order they're defined. This is useful for a variety of reasons, but most commonly it is used so that a shell script can bootstrap some of the systems so that another provisioner can take over later. ## Extra Find how to bridge a virtual machine into your host's network, so that you VMs are accessible from the rest of the world. ## Additional Features For the information about additional features visit documentation web site - [Push](https://www.vagrantup.com/docs/push/) - [Plugins](https://www.vagrantup.com/docs/plugins/) - [Providers](https://www.vagrantup.com/docs/providers/) - [Triggers](https://www.vagrantup.com/docs/triggers/) - [Other](https://www.vagrantup.com/docs/other/) # Troubleshooting ## `Vagrant` does not see `VirtualBox` even though you have installed it There were issues in compatibility between older vagrant and latest virtual box. Try installing newest versions from the web site instead of using packages from the repository of your linux distribution ## Avahi daemon does not work (local domain name is not recognized) Some reported an issue with `avahi`: the domain names cannot be resolved. [Possible solution](https://askubuntu.com/questions/837982/how-to-configure-local-dns-lookup-in-ubuntu-16-10/838395#838395) provided by Muhammad. ## VBoxManage cannot be used due to missing kernel modules The issue is with signing the kernel modules on the system protected by Secure Boot. Generally, disabling Secure Boot should solve the problem. Sometimes it does not work due to outdated bios version. Virtualbox version after 6.0.8 should work out of the box, but for some manufacturers the issue remain. The basic steps for solving this issue - Disable Secure Boot (read [this](https://stegard.net/2016/10/virtualbox-secure-boot-ubuntu-fail/) for more information) - Try rebuilding kernel modules (Hint how to do this is provided in `VBoxManage --version` error message) - Make sure your BIOS/UEFI is up to date ## Virtual machine does not boot after `vagrant halt` or `vagrant reload` This issue is rare. Usually it can be resolved by using another box image, e.g. `generic/ubuntu1804`. If you use a different image, make sure that folder syncing between host folder and `/vagrant` on the guest system is configured properly. If not, edit the setting `config.vm.synced_folder` in `Vagrantfile`. # Self-check Questions Useful resourse: [Vagrant Documentation](https://www.vagrantup.com/docs/cli/) 1. What will happen if you will start the vagrant machine without `$vagrant init`. Why? 2. To bring up the Vagrant virtual environment, you can use `$vagrant up`. What will happen after the command? 3. How do you access (login into) virtual machine created with vagrant? 4. What is a BOX in Vagrant? 5. What is Provider in Vagrant? 6. What is Provisioner in Vagrant? 7. What is Vagrantfile? 8. What are Synced Folders in Vagrant? 9. What is Multi-Machine environment in Vagrant? 10. How do you define multiple machines in Vagrant? 11. What does `vagrant push` do? 12. What does `vagrant box list` do? 13. What does `vagrant box outdated` do? 14. What does `vagrant box prune` do? 15. What does `vagrant box remove` do? 16. What does `vagrant box repackage` do? 17. What does `vagrant box update` do? 18. What does `vagrant connect` do? 19. What does `vagrant destroy` do?