Tips: Debug Saltstack States

Just hop under your current Vagrant VM and launch the SLS file locally:

vagrant ssh
sudo salt-call state.highstate --log-level=debug

Source files of the 2 tutorials on Vagrant and SaltStack available on Github

I've uploaded my source file on Github: VagrantSaltstackNginxSample for the Vagrant and Saltstack tutorials:

Virtualize your servers

Fill up your servers automatically with goodies

Fill up your servers automatically with goodies

Introduction

In my former article "Virtualize your servers", we created a virtualized server in few command lines. But... It's empty as the void in space (or so we think...). Still, you can log in easily with:

vagrant ssh

It is now time to fill it with some goodies: NGINX with all the bells and whistles.

Saltstack to the rescue

There are many provisionner available for Vagrant: Chef, Puppet, Docker, Saltstack, ... even simple bash scripts. A provisionner acts as framework for creating scripts that will fill your server wether they are physical or virtual. Note that as the servers may be virtual or physical, we will simply refers them as nodes. Nodes are the processing unit of your private cloud, wether it be a single to thousands machine.

Depending on the provisionner you choose, you may have more power for addressing use case scenarios such as:

• Managing a common configuration for each node with specialization for some of them.
• Configuring IP addresses and routes for each node.
• Managing states for each node: a state is an expected running configuration.
• Upgrading each node and letting the other knows about it while limiting the impacts on service unavailability.

I've chosen Saltstack as it covers these capabilities and the configuration files, the minions, that you write are simplistic and well organized.

Make Vagrant knows about Saltstack

Vagrant speaks Saltstack out-of-the-box. You only put in your little Vagrantfile where are stored your Saltstack files. Here, I put them in a directory named salt.

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # Name of the node
  config.vm.box = "UbuntuServer"

  # Import a preinstalled Ubuntu Server 12.04 LTS
  config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-i386-vagrant-disk1.box"

  # Add port forwarding to access service deployed in the node
  config.vm.network "forwarded_port", host: 8080, guest: 80

  # Synchronized folders
  config.vm.synced_folder "salt/srv/salt/", "/srv/salt/"
  
  # Use Saltstack as provisioner
  config.vm.provision :salt do |salt|
    # Set the directory where is stored your minion
    salt.minion_config = "salt/minion"
    # Maintain states
    salt.run_highstate = true
  end
end

Note the option salt.run_highstate = true. It tells  Vagrant to ensure the node state whenever it is executing it.

Here comes the minions

There are 2 ways of using Saltstack: locally or as a server which acts as a repository of minions. In our simple example, we use the most basic way, the local incarnation: masterless. Therefore, the salt/minion file is kept as its bare minimum:

# We run in masterless mode
master: localhost
# We use the local file directory
file_client: local

We use the default Saltstack directory structure to store our minion that will install NGINX:

.
├── Vagrantfile
└── salt
    ├── minion
    └── srv
        └── salt
            ├── top.sls
            └── webserver.sls

The top.sls file is your entry file to the installation. Here, there is only one formula. Thus, it is pretty straightforward:

base:
  '*':
    - webserver

The webserver.sls is called by the top.sls. It contains the formula to install and run NGINX as a service:

nginx:
  pkg:
    - installed
  service:
    - running
    - require:
      - pkg: nginx

Get ready!

Now, it's time to fire up everything. If you have just followed my previous tutorial "Virtualize your servers", you should already have downloaded the node's OS. We will destroy it. Yes. Destroy it. This will not re-download the node's OS. It will simply destroy its current customized image. Then, we will rerun Vagrant with our new and fresh configuration. This will do the following:

• Re-configure the node.
• Bootstrapping Saltstack into it.
• Launch Saltstack so that:
• It install NGINX using the node's OS package manager, apt-get in this case.
• Launch NGINX as a service.

vagrant destroy
vagrant up

Now, fire up your browser of choice and use the following URL: http://localhost:8080. This should display the NGINX default page just as if you had install it on a regular Ubuntu server.

Next steps

Vagrant and Saltstack are an incredibly powerful combo. In this example, I've made you destroy and reinstalled the node. It was only to ensure a clean state. While creating your server, you don't need to recreate everything and restart your node. Using the states provided by Saltstack, you just modify your SLS files and reprovision the states as you develop them. Iteration are done in a matter of seconds with the command:

vagrant provision

You can install other services as easily. There are some already prepared formulas that you can directly import or take inspiration from in the Saltstack formulas repository.

Virtualize your servers

Introduction

Virtualizing a server has many advantages over installing real servers. First, you can have many installments for testing or replicating existing ones. Second, it helps you develop installation scripts which will help you on your next projects.

OSX optional installations

OSX comes with no package manager except the AppStore. Though, it does the required common uses for casual users, it fall short when you need hundreds of tools. Thankfully, Brew is up to the task... if you are not too reluctant using the command line. Its installation is really simple:

ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

You have now access to almost every free and open source available packages.

Now, you can install an access to some already build packages as well as some proprietary software using Cask:

brew tap phinze/cask
brew install brew-cask

Here, you are. No more search attempts for finding a software. Almost every installations are available in a single command. If you don't know the name of the package, you simply search it on the command line. Here's an example for Firefox:

brew cask search firefox
==> Exact match
firefox
==> Partial matches
multifirefox

Installing VirtualBox

VirtualBox is a free and well maintained virtual machine, sometimes called hypervisor. You install it using the following command:

brew cask install virtualbox

Stay alert your password is required. Done! Nice.

Installing Vagrant

Vagrant is an helper installation program that will create for you a complete configured virtualized server. It uses a simple Ruby file as its configuration in order to produce reproducible images of your server. When you want to send a server to your coworkers, you only send this file. When you want to create a new server with some preinstalled tools, you just edit the configuration file. The same step as before is required:

brew cask install vagrant

After a required password, you got it now as an available new command.

Now your system is almost a Cloud controller (well, almost) :)

Installing and running your server OS of choice

With your new artillery, you are able to install every virtual machine image. Here, I depict the details for installing an Ubuntu server. Sure, you can choose every other OSes you may like (just Googles it :) ).

mkdir ~/VirtualBoxes
cd ~/VirtualBoxes
vagrant init UbuntuServer

This step create a simple configuration file named Vagrantfile that we will edit. Choose your favorite editor. Mine is Vim and you can turn on the syntax using the nice vim-vagrant plugin (which comes with a supercharge Vagrant command). As the Vagrantfile is a simple Ruby file, you should be able to activate the appropriate syntax highlighting.

Ready? Just edit the Vagrantfile and replace the line that starts with # config.vm.bow_url = "... with:

  config.vm.box_url = "https://cloud-images.ubuntu.com/vagrant/precise/current/precise-server-cloudimg-i386-vagrant-disk1.box"

That's all. Your server is running. Note that the installation is only done once :)

Check you virtual server

Launch VirtualBox in your Launcher and you can see it running.

My screenshot is in french but you should see where we get.

Turning off the server

Still in your command line? Vagrant helps you out also in this field:

vagrant halt