Vagrant: Difference between revisions
(41 intermediate revisions by the same user not shown) | |||
Line 2: | Line 2: | ||
= Resources = | = Resources = | ||
What is vagrant | |||
* Vagrant is an open source tool used for creating a portable virtual environment. | |||
* Using Vagrant, developers and sysadmins can create any virtual environment instantly. | |||
== Download == | == Download == | ||
[https://www.vagrantup.com/downloads.html Download] link for the latest version or use '''sudo apt install virtualbox vagrant'''. | |||
Use '''vagrant -v''' to see the vagrant version currently installed in your machine. | Use '''vagrant -v''' to see the vagrant version currently installed in your machine. | ||
== Documentation == | == Documentation, medias == | ||
https://docs.vagrantup.com/v2/ | * https://docs.vagrantup.com/v2/ | ||
* [http://www.linuxjournal.com/content/vagrant-simplified Vagrant Simplified] from LinuxJournal. | |||
== Books == | == Books == | ||
Line 15: | Line 21: | ||
* Pro Vagrant (2015) | * Pro Vagrant (2015) | ||
* Vagrant: Up and Running (2013) | * Vagrant: Up and Running (2013) | ||
== Windows == | |||
* http://www.sitepoint.com/getting-started-vagrant-windows/ | |||
* http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-windows-7-and-8 | |||
On Windows OS w/o admin privilege, I need to use Admin/PM to open CMD. Otherwise, 'vagrant up' will give a permission error. | |||
Also, since Windows does not have the '''ssh''' command, it is better to borrow the command from the [https://git-scm.com/download/ Git package]. After installing it, open '''Git Bash''' from the start menu (''Use Admin/PM''). Then cd to the vagrant project directory. Continue to run '''vagrant up''' or '''vagrant ssh''' as you want. | |||
Too bad. [https://github.com/mitchellh/vagrant/issues/6861 On Windows 10], we will get an error with the current version of vagrant. | |||
<pre> | |||
URL: ["https://atlas.hashicorp.com/ubuntu/precise32"] | |||
Error: | |||
</pre> | |||
Need to downgrade vagrant to [https://releases.hashicorp.com/vagrant/1.7.4/ 1.7.4]. | |||
= [https://docs.vagrantup.com/v2/cli/index.html Some commands] = | = [https://docs.vagrantup.com/v2/cli/index.html Some commands] = | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
vagrant -v # get the vagrant version | |||
vagrant up | vagrant up | ||
vagrant halt # gracefully shut down | vagrant halt # gracefully shut down | ||
vagrant reload # restarts vagrant machine, loads new Vagrantfile configuration | |||
vagrant global-status # output status of all vagrant machines (not boxes) | |||
# If a machine is up before, it is shown as 'running' in the output | |||
# EVEN IT IS NOT RUNNING now. For example, the following machine | |||
# is currently off now. | |||
# id name provider state directory | |||
# ------------------------------------------------------------------------ | |||
# a48f6f3 default virtualbox running /mnt/bigdisk/vagrant/trusty64-gui | |||
# BUT THE VIRTUALBOX shows it is off now. | |||
vagrant suspend | vagrant suspend | ||
vagrant status # displays the currently initialized vagrant environments | |||
vagrant resume | vagrant resume | ||
vagrant destroy # stops the running machine Vagrant is managing and destroys all resources | vagrant destroy # stops the running machine Vagrant is managing and destroys all resources | ||
vagrant destroy a48f6f3 # Use id | |||
vagrant box SUBCOMMANDS | vagrant box SUBCOMMANDS | ||
Line 30: | Line 64: | ||
vagrant ssh | vagrant ssh | ||
vagrant ssh-config # You can set any of the values in the Vagrantfile if you need to modify it | |||
vagrant package # This packages a currently running VirtualBox environment into a re-usable box. | vagrant package # This packages a currently running VirtualBox environment into a re-usable box. | ||
Line 36: | Line 71: | ||
= Examples = | = Examples = | ||
== Create Ubuntu machines == | |||
<syntaxhighlight lang='bash'> | |||
# Method 1: 14.04 (Trusty Tahr) | |||
mkdir -p ~/Vagrant/trusty64 | |||
cd ~/Vagrant/trusty64 | |||
## create a Vagrantfile by entering its content from https://app.vagrantup.com/ubuntu/boxes/trusty64 | |||
cat > Vagrantfile <<EOF | |||
Vagrant.configure("2") do |config| | |||
config.vm.box = "ubuntu/trusty64" | |||
end | |||
EOF | |||
vagrant up # a hidden directory .vagrant will be created. | |||
vagrant ssh # based on the hidden environment to log in | |||
vagrant halt # shutdown | |||
vagrant destroy # delete | |||
# Method 2: 16.04 (Xenial Xerus) | |||
mkdir ~/Vagrant/xenial64 | |||
vagrant box add ubuntu/xenial64 | |||
vagrant init ubuntu/xenial64 # Create <Vagrantfile> under the current directory | |||
vagrant up | |||
... | |||
# Method 3: 18.04 (Bionic Beaver) | |||
mkdir ~/Vagrant/bionic64 | |||
# Skip the "box" command | |||
vagrant init ubuntu/bionic64 # Create <Vagrantfile> under the current directory | |||
vagrant up | |||
... | |||
</syntaxhighlight> | |||
== R box == | |||
Not available! | |||
== A simple example == | == A simple example == | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
Line 110: | Line 179: | ||
</pre> | </pre> | ||
== Share | == Share folders/Synced folders == | ||
=== Default implementation from providers === | |||
Suppose we want to set up a web server to develop html files. We share a source folder that contains html documentation with the guest. | Suppose we want to set up a web server to develop html files. We share a source folder that contains html documentation with the guest. | ||
# In the working directory, create a new dir called vagrantsite. | # In the working directory, create a new dir called vagrantsite. | ||
Line 138: | Line 208: | ||
* A Vagrant machine executing on a remote hypervisor (eg a cloud service) will usually not have shared filesystems available. Vagrant provides methods to copy files from host ot guest using the '''rsync''' protocol. | * A Vagrant machine executing on a remote hypervisor (eg a cloud service) will usually not have shared filesystems available. Vagrant provides methods to copy files from host ot guest using the '''rsync''' protocol. | ||
== Share | === [https://docs.vagrantup.com/v2/synced-folders/nfs.html Share Network File Systems] === | ||
Sharing directories from the host machine will require the host system to export a directory for use by the guest machine. When using NFS, this means that Vagrant will add an entry to the native '''/etc/exports''' file to define the rule to export the specified directory to the guest box. | |||
Prior to starting, make sure that '''Network File Share Daemon (nfsd)''' is installed on your host machine. | |||
Notice the '''type''' option in the ''synced_folder'' line. | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
config.vm.define "web", primary: true do |web| | |||
web.vm.box = "ubuntu/trusty64" | |||
web.vm.network "forwarded_port", guest:80, host:8888 | |||
web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type: "nfs" | |||
web.vm.provision "shell", inline, "apt-get install -y nginx; ln -s /opt/vagrantsite /usr/share/nginx/html/vagrantsite" | |||
end | |||
end | |||
</pre> | |||
You can verify the shared folder is from NFS by typing ''vagrant ssh'' and executing the '''mount''' command. The ''mount'' command should contain an entry that shows ''type nfs'' for one partition. | |||
For Windows '''Server Message Block (SMB)''' protocol, modify the synced_folder to | |||
<pre> | |||
web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type:" smb" | |||
</pre> | |||
=== [https://docs.vagrantup.com/v2/synced-folders/rsync.html rsync] === | |||
'''rsync'' method is useful when | |||
* NFS or VirtualBox shared folders aren't available in the guest machine. | |||
* Processes that generate significant disk activity (I/O) on shared folders | |||
* Vagrant can be used to control vm in remote locations (even in remote data centers over the public internet) | |||
Pay attention to the synced_folder line. | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
config.vm.define "web", primary: true do |web| | |||
web.vm.box = "ubuntu/trusty64" | |||
web.vm.network "forwarded_port", guest:80, host:8888 | |||
web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type: "rsync" | |||
web.vm.provision "shell", inline, "apt-get install -y nginx; ln -s /opt/vagrantsite /usr/share/nginx/html/vagrantsite" | |||
end | |||
end | |||
</pre> | |||
When the files in the host machine are changed, we need to run '''vagrant rsync''' command on the host to synchronize the /opt/vagrantsite directory. | |||
== Customizing virtual machine settings in VirtualBox == | == Customizing virtual machine settings in VirtualBox == | ||
Line 177: | Line 289: | ||
We can change the default directory by modifying the '''VAGRANT_HOME''' variable. See https://docs.vagrantup.com/v2/other/environmental-variables.html. | We can change the default directory by modifying the '''VAGRANT_HOME''' variable. See https://docs.vagrantup.com/v2/other/environmental-variables.html. | ||
== Linux == | |||
One time use. | |||
<syntaxhighlight lang='bash'> | |||
export VAGRANT_HOME=my/new/path/goes/here/ | |||
</syntaxhighlight> | |||
== Windows == | |||
* https://harvsworld.com/2014/change-vagrant_home-directory-windows/ | |||
<syntaxhighlight lang='dos'> | |||
set VAGRANT_HOME=X:\PATH\TO\VAGRANT # one time only | |||
setx VAGRANT_HOME "X:/your/path" # permanent | |||
</syntaxhighlight> | |||
Double check | |||
<syntaxhighlight lang='dos'> | |||
set VAGRANT_HOME | |||
</syntaxhighlight> | |||
= Vagrant Share = | = Vagrant Share = | ||
Line 216: | Line 346: | ||
= Boxes = | = Boxes = | ||
https://app.vagrantup.com/boxes/search | |||
== Download a box w/ initializing an env == | == Download a box w/ initializing an env == | ||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
Line 239: | Line 371: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== [https://docs.vagrantup.com/v2/boxes/base.html Create a new box/ | == List running VirtualBox machines == | ||
Suppose we use vagrant to create a new box based on ubuntu/trusty64. We ssh to the box and install some software (eg r-base-core). We use vagrant halt to shut down the box. Now we can package this new box. | <syntaxhighlight lang='bash'> | ||
VBoxManage list runningvms | |||
</syntaxhighlight> | |||
== [https://docs.vagrantup.com/v2/boxes/base.html Create a new vagrant box/package from a VirtualBox machine] == | |||
The final note is saved under Google Drive!! | |||
The instruction at http://aruizca.com/steps-to-create-a-vagrant-base-box-with-ubuntu-14-04-desktop-gui-and-virtualbox/ works for me. Lots of information is given there like install vagrant public key, compact space, install OpenSSH server. Two things that are missing are | |||
* rm ~/.ssh/known_hosts | |||
* cat /dev/null > ~/.bash_history && history -c && exit. | |||
(Naive approach. See the above link at aruizca.com for more accurate steps) Suppose we use vagrant to create a new box based on ubuntu/trusty64. We ssh to the box and install some software (eg r-base-core). We use vagrant halt to shut down the box. Now we can package this new box. | |||
<syntaxhighlight lang='bash'> | <syntaxhighlight lang='bash'> | ||
Line 288: | Line 431: | ||
</pre> | </pre> | ||
Now we can run ''vagrant up'' and ''vagrant ssh'' commands. We could check R should be available in the virtual machine. | Now we can run ''vagrant up'' and ''vagrant ssh'' commands. We could check R should be available in the virtual machine. | ||
= Create a template for the virtual machine images: Packer = | |||
* https://www.packer.io/ | |||
* https://youtu.be/6-7WjA-hHvg?t=10m44s | |||
* [https://youtu.be/89AZapvTpmM Rolling Your Own Images With Packer] | |||
* [https://f1000research.com/articles/7-1656/ Orchestrating a community-developed computational workshop and accompanying training materials] by Sean Davis 2018 | |||
* Examples: | |||
** [https://github.com/chef/bento Bento] project | |||
** https://seandavi.github.io/post/2018/07/infrastructure-as-code-building-the-bioconductor-conference-ami-with-packer/ | |||
= Provisioning = | = Provisioning = | ||
== Running basic shell commands == | |||
Pay attention to the line containing '''config.vm.provision'''. | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
config.vm.box = "ubuntu/trusty64" | |||
config.vm.provision "shell", | |||
inline: "echo 'Knock Knock!' > /etc/motd" | |||
end | |||
</pre> | |||
Once the machine is booted, access the machine with ''vagrant ssh'' command. The login shell will display the message created with the inline shell script. | |||
== Executing shell scripts in a Vagrantfile == | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
$nginx_install = <<SCRIPT | |||
if [ ! -x /usr/sbin/nginx ]; then | |||
apt-get install -y nginx; | |||
fi | |||
# Default NGINX directory: /usr/share/nginx/html | |||
# Replace this with symbolic link to vagrant directory. | |||
if [ ! -L /usr/share/nginx/html ]; then | |||
rm -rf /usr/share/nginx/html | |||
ln -s /vagrant/html /usr/share/nginx/html | |||
fi | |||
SCRIPT | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
config.vm.box = "puppetlabs/ubuntu-14.04-32-nocm" | |||
config.vm.provision "shell", inline: $nginx_install | |||
config.vm.network "forwarded_port", guest:80, host:8080 | |||
end | |||
</pre> | |||
== External shell scripts == | |||
The working directory should contain both Vagrantfile and the shell script file. | |||
Pay attention to the line containing '''config.vm.provision'''. | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
config.vm.box = "ubuntu/trusty64" | |||
config.vm.provision "shell", path: "nginx-install.sh" | |||
config.vm.network "forwarded_port", guest:80, host:8080 | |||
end | |||
</pre> | |||
The script file <nginx-install.sh> looks like | |||
<syntaxhighlight lang='bash'> | |||
#!/bin/bash | |||
if [ ! -x /usr/sbin/nginx ]; then | |||
apt-get install -y nginx; | |||
fi | |||
# Default NGINX directory: /usr/share/nginx/html | |||
# Replace this with symbolic link to vagrant directory. | |||
if [ ! -L /usr/share/nginx/html ]; then | |||
rm -rf /usr/share/nginx/html | |||
ln -s /vagrant/html /usr/share/nginx/html | |||
fi | |||
</syntaxhighlight> | |||
Execute the ''vagrant up'' command and open the browser at http://localhost:8080 in the local machine. | |||
As a test of '''idempotency''', execute the ''vagrant provision'' command. You shall see this command will exit quickly as all conditions within the script should be satisfied. | |||
== Using management tools/languages: Puppet == | |||
* https://docs.vagrantup.com/v2/provisioning/puppet_apply.html | |||
* https://docs.vagrantup.com/v2/provisioning/puppet_agent.html | |||
Vagrant machines use the '''Puppet agent''' infrastructure to perform provisioning operations on a machine. Puppet agents can function in one of two ways. | |||
* By connecting to a '''Puppymaster''' to retrieve configuration information. A Puppetmaster is a server that is a centroalized location for systems to retrieve system configurations. | |||
* By executing a '''puppy apply''' command to interpret and apply configurations locally. This is often referred to as the ''masterless Puppet'' approach. | |||
=== Install Puppet === | |||
* [https://docs.puppetlabs.com/puppetserver/2.2/install_from_packages.html Puppet Server] | |||
* [https://docs.puppetlabs.com/puppet/latest/reference/install_linux.html Install Puppet Agent in Linux] | |||
* How to install Puppet master and client in Ubuntu 14.04 https://www.howtoforge.com/puppet-ubuntu-14.04 | |||
* http://www.bogotobogo.com/DevOps/Puppet/puppet_install_on_ubuntu_14_04_trusty.php | |||
* https://help.ubuntu.com/lts/serverguide/puppet.html | |||
* https://www.digitalocean.com/community/tutorials/how-to-install-puppet-to-manage-your-server-infrastructure | |||
=== Masterless approach === | |||
We apply Puppet configurations locally with some reusable code (a Puppet module) that is obtained from the Puppet Forget repository. | |||
* Vagrant box: it is necessary to use a Vagrant box that either has the Puppet agent installed or creates a ''bootstrapping'' script that configures packages repositories and installs the Puppet agent in the virtual machine. For simplicity, we can use a [https://vagrantcloud.com/puppetlabs/boxes/ubuntu-14.04-64-puppet box provided by Puppet Labs]. | |||
* Host machine: the host machine also needs to install the Puppet agent. Having the agent installed on the host will allow for management of Puppet modules and resources by the host for Vagrant guests to be provisioned. A common example is to use the ''puppet module'' utility to manage and use modules. | |||
The working directory has the following file structure | |||
<pre> | |||
|-- Vagrantfile | |||
|-- puppet | |||
|-- manifests | |||
|-- modules | |||
</pre> | |||
The Vagrantfile looks like (pay attention to the line containing '''config.vm.provision'''. The Puppet provisioner requires parameters to be set that define paths to our ''manifest'' and ''modules'' directories, as well as a manifest filename <site.pp> in this case). | |||
<pre> | |||
VAGRANTFILE_API_VERSION = "2" | |||
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | |||
# A Vagrant box with puppet pre-installed. | |||
config.vm.define "web", :primary => true do |web| | |||
web.vm.box = "ubuntu/trusty64" | |||
web.vm.hostname = "web" | |||
web.vm.network "forwarded_port", guest: 80, host: 8080 | |||
web.vm.provision "puppet" do |puppet| | |||
puppet.manifests_path = "puppet/manifests" | |||
puppet.manifest_file = "site.pp" | |||
puppet.module_path = "puppet/modules" | |||
end | |||
end | |||
end | |||
</pre> | |||
With the Puppet provisioner configured, we can write the Puppet code. | |||
'''Configure Puppet''' | |||
We use code downloaded from the Puppet Forget to install the Apache web server | |||
# Open Puppet Forget (http://forge.puppetlabs.com) | |||
# Search for 'apache' | |||
# Select the ''puppetlabs/apache'' module. There are 2 ways you can do this: | |||
#* If you have puppet installed on your host, install the module with the '''puppet module''' command: ''puppet module install --modulepath=puppet/modules puppetlabs-apache''. This approach will resolve dependencies and download them as well as the specified module itself. | |||
#* Download the tar.gz file and extract it into the modules directory. As this method does not resolve dependencies, this is something that need to be done manually. For this apache example, as we can see from the metadata.json file, the module dependencies are concat and stdlib. | |||
# With the module dependencies installed, we need to create a ''manifest'' file <site.pp> that will govern how resources and modules are used. | |||
<pre> | |||
node web { | |||
class{"apache": | |||
default_vhost => false, | |||
} | |||
apache::vhost{"default-host": | |||
docroot => "/var/www/html", | |||
docroot_owner => 'www-data', | |||
docroot_group => 'www-data', | |||
default_vhost => true, | |||
logroot => '/var/log/apache2', | |||
port => 80, | |||
} | |||
} | |||
</pre> | |||
This manifest file does 3 things: specifies an action for the ''web'' node, calls the ''apache'' class to install the Apache web server, and defines an ''apache::vhost'' type that will create a default virtual host for our web server. | |||
The working directory now has a structure | |||
<pre> | |||
|-- Vagrantfile | |||
|-- puppet | |||
|-- manifests -- site.pp | |||
|-- modules | |||
|-- apache -- lots of files and subdirectories | |||
|-- concat -- lots of files and subdirectories | |||
|-- stdlib -- lots of files and subdirectories | |||
</pre> | |||
Run the ''vagrant up'' command in the working directory. Open http://localhost:8080 in a web browser. | |||
= Networking = | = Networking = | ||
= Multi-Machine = | = Multi-Machine = | ||
Line 301: | Line 609: | ||
== VMware == | == VMware == | ||
== Docker == | == Docker == | ||
[[Docker#Vagrant_vs_Docker|Vagrant vs Docker]] | |||
== Hyper-V == | == Hyper-V == | ||
= Others = | |||
== Resize a hard disk == | |||
* https://gist.github.com/christopher-hopper/9755310 | |||
* http://askubuntu.com/questions/317338/how-can-i-increase-disk-size-on-a-vagrant-vm |
Latest revision as of 10:03, 17 October 2018
Official website https://www.vagrantup.com/
Resources
What is vagrant
- Vagrant is an open source tool used for creating a portable virtual environment.
- Using Vagrant, developers and sysadmins can create any virtual environment instantly.
Download
Download link for the latest version or use sudo apt install virtualbox vagrant.
Use vagrant -v to see the vagrant version currently installed in your machine.
Documentation, medias
- https://docs.vagrantup.com/v2/
- Vagrant Simplified from LinuxJournal.
Books
- Vagrant Virtual Development Environment Cookbook (2015)
- Creating Development Environments with Vagrant (2015, 2nd Ed)
- Pro Vagrant (2015)
- Vagrant: Up and Running (2013)
Windows
- http://www.sitepoint.com/getting-started-vagrant-windows/
- http://www.seascapewebdesign.com/blog/part-1-getting-started-vagrant-windows-7-and-8
On Windows OS w/o admin privilege, I need to use Admin/PM to open CMD. Otherwise, 'vagrant up' will give a permission error.
Also, since Windows does not have the ssh command, it is better to borrow the command from the Git package. After installing it, open Git Bash from the start menu (Use Admin/PM). Then cd to the vagrant project directory. Continue to run vagrant up or vagrant ssh as you want.
Too bad. On Windows 10, we will get an error with the current version of vagrant.
URL: ["https://atlas.hashicorp.com/ubuntu/precise32"] Error:
Need to downgrade vagrant to 1.7.4.
Some commands
vagrant -v # get the vagrant version vagrant up vagrant halt # gracefully shut down vagrant reload # restarts vagrant machine, loads new Vagrantfile configuration vagrant global-status # output status of all vagrant machines (not boxes) # If a machine is up before, it is shown as 'running' in the output # EVEN IT IS NOT RUNNING now. For example, the following machine # is currently off now. # id name provider state directory # ------------------------------------------------------------------------ # a48f6f3 default virtualbox running /mnt/bigdisk/vagrant/trusty64-gui # BUT THE VIRTUALBOX shows it is off now. vagrant suspend vagrant status # displays the currently initialized vagrant environments vagrant resume vagrant destroy # stops the running machine Vagrant is managing and destroys all resources vagrant destroy a48f6f3 # Use id vagrant box SUBCOMMANDS vagrant box remove ubuntu/trusty32 vagrant ssh vagrant ssh-config # You can set any of the values in the Vagrantfile if you need to modify it vagrant package # This packages a currently running VirtualBox environment into a re-usable box.
Examples
Create Ubuntu machines
# Method 1: 14.04 (Trusty Tahr) mkdir -p ~/Vagrant/trusty64 cd ~/Vagrant/trusty64 ## create a Vagrantfile by entering its content from https://app.vagrantup.com/ubuntu/boxes/trusty64 cat > Vagrantfile <<EOF Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" end EOF vagrant up # a hidden directory .vagrant will be created. vagrant ssh # based on the hidden environment to log in vagrant halt # shutdown vagrant destroy # delete # Method 2: 16.04 (Xenial Xerus) mkdir ~/Vagrant/xenial64 vagrant box add ubuntu/xenial64 vagrant init ubuntu/xenial64 # Create <Vagrantfile> under the current directory vagrant up ... # Method 3: 18.04 (Bionic Beaver) mkdir ~/Vagrant/bionic64 # Skip the "box" command vagrant init ubuntu/bionic64 # Create <Vagrantfile> under the current directory vagrant up ...
R box
Not available!
A simple example
$ mkdir precise32 $ cd precise32 $ vagrant init hashcorp/precise32 A `Vagrantfile` has been placed in this directory. You are now ready to `vagrant up` your first virtual environment! Please read the comments in the Vagrantfile as well as documentation on `vagrantup.com` for more information on using Vagrant. $ cat Vagrant VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "hashcorp/precise32" end $ vagrant up vm * The box 'hashcorp/precise32' could not be found.
The solution from stackoverflow works.
If we open VirtualBox GUI, we will see a new guest machine called precise32_default_XXXXXXXX is created and running though we do not see Ubuntu precise 32 desktop application in a new window.
A web server
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "web", primary: true do |web| web.vm.box = "ubuntu/trusty64" web.vm.network "forwarded_port", guest:80, host:8888 web.vm.provision "shell", inline: "apt-get install -y nginx" end end
Start the machine with the vagrant up command. Open a web browser on your host machine at http://localhost:8888. This will present the nginx Welcome page.
See my gist for the output.
A GUI example
Go to
- https://atlas.hashicorp.com/boxes/search. Type 'GUI' to search
- https://atlas.hashicorp.com/chad-thompson/boxes/ubuntu-trusty64-gui
Run vagrant init chad-thompson/ubuntu-trusty64-gui. Edit Vagrantfile.
Vagrant.configure(2) do |config| config.vm.box = "chad-thompson/ubuntu-trusty64-gui" config.vm.provider "virtualbox" do |vbox| vbox.gui = true end end
Then run vagrant up (1.5GB download).
Note that when I run df -h in the VM, it shows
Filesystem Size Used Avail Mounted on /dev/sda1 21G 3.1G 17G / /vagrant 1.8T 445G 1.4T /vagrant
The /vagrant partition contains the Vagrantfile.
We can also make the Vagrantfile a bit more general by including multiple providers in the same Vagrantfile.
Vagrant.configure(2) do |config| config.vm.box = "chad-thompson/ubuntu-trusty64-gui" config.vm.provider "virtualbox" do |vbox| vbox.gui = true end config.vm.provider "vmware_fusion" do |fusion| fusion.gui = true end end
Default implementation from providers
Suppose we want to set up a web server to develop html files. We share a source folder that contains html documentation with the guest.
- In the working directory, create a new dir called vagrantsite.
- In the vagrantsite dir, create a file named index.html.
- Create a new Vagrantfile in the working directory.
- Add a synced_folder directive to the web server configuration.
- Create an additional command to the web.vm.provision line to create a symbolic link from this directory to a directory in the nginx default web directory.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "web", primary: true do |web| web.vm.box = "ubuntu/trusty64" web.vm.network "forwarded_port", guest:80, host:8888 web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite" web.vm.provision "shell", inline, "apt-get install -y nginx; ln -s /opt/vagrantsite /usr/share/nginx/html/vagrantsite" end end
Now executing vagrant up command and open the page http://localhost:8888/vagrantsite/ in a web browser.
Note
- Vagrant shares the working directory of the Vagrantfile in the root vagrant directory by default. Executing ls /vagrant command should return this information.
- Sharing folders will require proper tools to be installed in the host machine. For VirtualBox, this means having the guest additions installed. For VMware, this means VMware tools installed.
Shared folders can also be implemented in different ways
- Using networked file systems such as NFS (Linux, OS X) and SMB (Windows). These options might offer significantly better file-sharing performance than shared folder functionality of the hypervisor; i.e. NFS is better than using the guest additions method in VirtualBox. See http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines.
- A Vagrant machine executing on a remote hypervisor (eg a cloud service) will usually not have shared filesystems available. Vagrant provides methods to copy files from host ot guest using the rsync protocol.
Sharing directories from the host machine will require the host system to export a directory for use by the guest machine. When using NFS, this means that Vagrant will add an entry to the native /etc/exports file to define the rule to export the specified directory to the guest box.
Prior to starting, make sure that Network File Share Daemon (nfsd) is installed on your host machine.
Notice the type option in the synced_folder line.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "web", primary: true do |web| web.vm.box = "ubuntu/trusty64" web.vm.network "forwarded_port", guest:80, host:8888 web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type: "nfs" web.vm.provision "shell", inline, "apt-get install -y nginx; ln -s /opt/vagrantsite /usr/share/nginx/html/vagrantsite" end end
You can verify the shared folder is from NFS by typing vagrant ssh and executing the mount command. The mount command should contain an entry that shows type nfs for one partition.
For Windows Server Message Block (SMB) protocol, modify the synced_folder to
web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type:" smb"
rsync
'rsync method is useful when
- NFS or VirtualBox shared folders aren't available in the guest machine.
- Processes that generate significant disk activity (I/O) on shared folders
- Vagrant can be used to control vm in remote locations (even in remote data centers over the public internet)
Pay attention to the synced_folder line.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "web", primary: true do |web| web.vm.box = "ubuntu/trusty64" web.vm.network "forwarded_port", guest:80, host:8888 web.vm.synced_folder "vagrantsite/", "/opt/vagrantsite", type: "rsync" web.vm.provision "shell", inline, "apt-get install -y nginx; ln -s /opt/vagrantsite /usr/share/nginx/html/vagrantsite" end end
When the files in the host machine are changed, we need to run vagrant rsync command on the host to synchronize the /opt/vagrantsite directory.
Customizing virtual machine settings in VirtualBox
The primary example is changing the settings of the virtual machine to use more (or less) system memory (2GB in the following example) and virtual processors (2 cpus).
Vagrant.configure(2) do |config| config.vm.define "web", primary: true do |web| web.vm.box = "ubuntu/trusty64" web.vm.network "forwarded_port", guest:80, host:8888 web.vm.provision "shell", inline: "apt-get install -y nginx" web.vm.provider "virtualbox" do |vbox| vbox.memory = 2048 vbox.cpus = 2 end end end
Start the vm with the vagrant up command. Verify the system has the amount of memory allocated by issuing the command head /proc/meminfo.
If we like to start a GUI on the vm, just add vbox.gui = true in the vbox block.
web.vm.provider "virtualbox" do |vbox| vbox.memory = 2048 vbox.cpus = 2 vbox.gui = true vbox.customize ["modifyvm", :id, "--bioslogofadein", "off"] end
The last line use the VBoxManage command to fade the VirtualBox startup logo immediately.
A full listing of available options to modify the runtime is available in the VirtualBox documentation http://www.virtualbox.org/manual/ch08.html.
Where is vagrant saving boxes files
http://stackoverflow.com/questions/10155708/where-does-vagrant-download-its-box-files-to
- Windows: C:/Users/USERNAME/.vagrant.d/boxes
- Linux and Mac: ~/.vagrant.d/boxes/
We can change the default directory by modifying the VAGRANT_HOME variable. See https://docs.vagrantup.com/v2/other/environmental-variables.html.
Linux
One time use.
export VAGRANT_HOME=my/new/path/goes/here/
Windows
set VAGRANT_HOME=X:\PATH\TO\VAGRANT # one time only setx VAGRANT_HOME "X:/your/path" # permanent
Double check
set VAGRANT_HOME
Vagrantfile
Vagrantfile is just Ruby.
Vagrantfile template
If we run vagrant init, we will get the following vagrantfile.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| # ... end
Precise32
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "hashcorp/precise32" end
config.vm
Config namespace: config.vm
The settings within config.vm modify the configuration of the machine that Vagrant manages.
- config.vm.boot_timeout
- config.vm.box
- config.vm.box_url
- config.vm.communicator
- config.vm.hostname
- config.vm.provider
- config.vm.synced_folder
config.ssh
Boxes
https://app.vagrantup.com/boxes/search
Download a box w/ initializing an env
vagrant box add ubuntu/trusty64
You can also quickly initialize a Vagrant environment with vagrant init ubuntu/trusty64.
We can also specify an URL to add/download a box.
vagrant box add http://servername/boxes/environment.box
Remove a box
Something like
vagrant box remove ubuntu/trusty32
List downloaded boxes
vagrant box list
List running VirtualBox machines
VBoxManage list runningvms
Create a new vagrant box/package from a VirtualBox machine
The final note is saved under Google Drive!!
The instruction at http://aruizca.com/steps-to-create-a-vagrant-base-box-with-ubuntu-14-04-desktop-gui-and-virtualbox/ works for me. Lots of information is given there like install vagrant public key, compact space, install OpenSSH server. Two things that are missing are
- rm ~/.ssh/known_hosts
- cat /dev/null > ~/.bash_history && history -c && exit.
(Naive approach. See the above link at aruizca.com for more accurate steps) Suppose we use vagrant to create a new box based on ubuntu/trusty64. We ssh to the box and install some software (eg r-base-core). We use vagrant halt to shut down the box. Now we can package this new box.
$ VBoxManage list vms # find out the vm name (vm is created by Vagrant) $ vagrant package --base=trusty64_default_1451665747284_29528 --output=mytest.box
To test the new box, we can copy the box to your local Vagrant cache.
$ vagrant box add mytest.box --name=mytestbox $ vagrant box list mytestbox (virtualbox, 0) ubuntu/trusty64 (virtualbox, 20151217.0.0)
Create a new directory so we can test the new box.
cd .. mkdir mytestbox cd mytestbox vagrant init mytestbox # create Vagrantfile
Note that we can not use vagrant up to start the box now. If we try to run it, we will get the following error about the SSH authentication.
default: SSH username: vagrant default: SSH auth method: private key default: Warning: Remote connection disconnect. Retrying... default: Warning: Authentication failure. Retrying...
I have to use Ctrl+C to stop the process and then use vagrant destroy to destroy the (unsuccessfully) running virtual machine.
Go ahead to modify the Vagrantfile created by vagrant init mytestbox command to use the correct SSH username/password. By default, Vagrant relies on a common public key that is used by most box publishers that allows access to an account called vagrant. After the first login, Vagrant will place a key in the appropriate account; so, if desired, the password can be removed from the Vagrantfile after the first boot. (verified!)
Vagrant.configure(2) do |config| config.vm.box = "mytestbox" config.ssh.username="ubuntu" config.ssh.password="ubuntu" end
Strangely, though I can run vagrant up, I cannot use the password I created when I try the vagrant ssh command. I found the solution is to make sure the username and password used in the Vagrantfile the same as the virtual machine; in this case since I am using the ubuntu/trusty64 box, I should use the vagrant/vagrant as the username/password in the Vagrantfile. That is the correct Vagrantfile should be
Vagrant.configure(2) do |config| config.vm.box = "mytestbox" config.ssh.username="vagrant" config.ssh.password="vagrant" end
Now we can run vagrant up and vagrant ssh commands. We could check R should be available in the virtual machine.
Create a template for the virtual machine images: Packer
- https://www.packer.io/
- https://youtu.be/6-7WjA-hHvg?t=10m44s
- Rolling Your Own Images With Packer
- Orchestrating a community-developed computational workshop and accompanying training materials by Sean Davis 2018
- Examples:
Provisioning
Running basic shell commands
Pay attention to the line containing config.vm.provision.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "shell", inline: "echo 'Knock Knock!' > /etc/motd" end
Once the machine is booted, access the machine with vagrant ssh command. The login shell will display the message created with the inline shell script.
Executing shell scripts in a Vagrantfile
VAGRANTFILE_API_VERSION = "2" $nginx_install = <<SCRIPT if [ ! -x /usr/sbin/nginx ]; then apt-get install -y nginx; fi # Default NGINX directory: /usr/share/nginx/html # Replace this with symbolic link to vagrant directory. if [ ! -L /usr/share/nginx/html ]; then rm -rf /usr/share/nginx/html ln -s /vagrant/html /usr/share/nginx/html fi SCRIPT Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "puppetlabs/ubuntu-14.04-32-nocm" config.vm.provision "shell", inline: $nginx_install config.vm.network "forwarded_port", guest:80, host:8080 end
External shell scripts
The working directory should contain both Vagrantfile and the shell script file.
Pay attention to the line containing config.vm.provision.
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "shell", path: "nginx-install.sh" config.vm.network "forwarded_port", guest:80, host:8080 end
The script file <nginx-install.sh> looks like
#!/bin/bash if [ ! -x /usr/sbin/nginx ]; then apt-get install -y nginx; fi # Default NGINX directory: /usr/share/nginx/html # Replace this with symbolic link to vagrant directory. if [ ! -L /usr/share/nginx/html ]; then rm -rf /usr/share/nginx/html ln -s /vagrant/html /usr/share/nginx/html fi
Execute the vagrant up command and open the browser at http://localhost:8080 in the local machine.
As a test of idempotency, execute the vagrant provision command. You shall see this command will exit quickly as all conditions within the script should be satisfied.
Using management tools/languages: Puppet
- https://docs.vagrantup.com/v2/provisioning/puppet_apply.html
- https://docs.vagrantup.com/v2/provisioning/puppet_agent.html
Vagrant machines use the Puppet agent infrastructure to perform provisioning operations on a machine. Puppet agents can function in one of two ways.
- By connecting to a Puppymaster to retrieve configuration information. A Puppetmaster is a server that is a centroalized location for systems to retrieve system configurations.
- By executing a puppy apply command to interpret and apply configurations locally. This is often referred to as the masterless Puppet approach.
Install Puppet
- Puppet Server
- Install Puppet Agent in Linux
- How to install Puppet master and client in Ubuntu 14.04 https://www.howtoforge.com/puppet-ubuntu-14.04
- http://www.bogotobogo.com/DevOps/Puppet/puppet_install_on_ubuntu_14_04_trusty.php
- https://help.ubuntu.com/lts/serverguide/puppet.html
- https://www.digitalocean.com/community/tutorials/how-to-install-puppet-to-manage-your-server-infrastructure
Masterless approach
We apply Puppet configurations locally with some reusable code (a Puppet module) that is obtained from the Puppet Forget repository.
- Vagrant box: it is necessary to use a Vagrant box that either has the Puppet agent installed or creates a bootstrapping script that configures packages repositories and installs the Puppet agent in the virtual machine. For simplicity, we can use a box provided by Puppet Labs.
- Host machine: the host machine also needs to install the Puppet agent. Having the agent installed on the host will allow for management of Puppet modules and resources by the host for Vagrant guests to be provisioned. A common example is to use the puppet module utility to manage and use modules.
The working directory has the following file structure
|-- Vagrantfile |-- puppet |-- manifests |-- modules
The Vagrantfile looks like (pay attention to the line containing config.vm.provision. The Puppet provisioner requires parameters to be set that define paths to our manifest and modules directories, as well as a manifest filename <site.pp> in this case).
VAGRANTFILE_API_VERSION = "2" Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| # A Vagrant box with puppet pre-installed. config.vm.define "web", :primary => true do |web| web.vm.box = "ubuntu/trusty64" web.vm.hostname = "web" web.vm.network "forwarded_port", guest: 80, host: 8080 web.vm.provision "puppet" do |puppet| puppet.manifests_path = "puppet/manifests" puppet.manifest_file = "site.pp" puppet.module_path = "puppet/modules" end end end
With the Puppet provisioner configured, we can write the Puppet code.
Configure Puppet We use code downloaded from the Puppet Forget to install the Apache web server
- Open Puppet Forget (http://forge.puppetlabs.com)
- Search for 'apache'
- Select the puppetlabs/apache module. There are 2 ways you can do this:
- If you have puppet installed on your host, install the module with the puppet module command: puppet module install --modulepath=puppet/modules puppetlabs-apache. This approach will resolve dependencies and download them as well as the specified module itself.
- Download the tar.gz file and extract it into the modules directory. As this method does not resolve dependencies, this is something that need to be done manually. For this apache example, as we can see from the metadata.json file, the module dependencies are concat and stdlib.
- With the module dependencies installed, we need to create a manifest file <site.pp> that will govern how resources and modules are used.
node web { class{"apache": default_vhost => false, } apache::vhost{"default-host": docroot => "/var/www/html", docroot_owner => 'www-data', docroot_group => 'www-data', default_vhost => true, logroot => '/var/log/apache2', port => 80, } }
This manifest file does 3 things: specifies an action for the web node, calls the apache class to install the Apache web server, and defines an apache::vhost type that will create a default virtual host for our web server.
The working directory now has a structure
|-- Vagrantfile |-- puppet |-- manifests -- site.pp |-- modules |-- apache -- lots of files and subdirectories |-- concat -- lots of files and subdirectories |-- stdlib -- lots of files and subdirectories
Run the vagrant up command in the working directory. Open http://localhost:8080 in a web browser.