Multiple Vagrant VMs via PuPHPet with different configs

Question

I feel like I'm missing something fundamental. How do I use PuPHPet to define two machines in one vagrantfile, both Ubuntu 14.04, but one with mysql installed and one with elasticsearch? I see how to define multiple machines, but the config of each seems to be identical???

Answer 1

This isn't natively supported via the PuPHPet GUI, but... you can easily implement it.

Note: I will not offer support for the following via PuPHPet's github tracker. Please don't post tickets for this. However, I am using something similar to some freelance clients and it is working well.

PuPHPet has support for multiple config files, each extending and overriding the previous.

Take a look at puphpet/puppet/hiera.yaml:

---
:backends: yaml
:yaml:
    :datadir: '/'
:hierarchy:
    - vagrant/puphpet/config-custom
    - vagrant/puphpet/config-%{::provisioner_type}
    - vagrant/puphpet/config
:merge_behavior: deeper
:logger: console

Hiera loads config files backwards, so it goes config.yaml -> config-%{::provisioner_type}.yaml -> config-custom.yaml.

You can insert a new line below config-custom: - vagrant/puphpet/config-%{::server_type}

This will inject any custom settings within this file into the Puppet environment. However, you still need to make Vagrant aware of it because Vagrant reads from Vagrantfile to execute commands. You might also need to do some custom work for each different server type. Note that server_type in this context can mean something like proxy, app, db, jenkins, etc.

Within the PuPHPet root directory, where Vagrantfile and puphpet directory exist, create new directories named after your server types.

For example, create a new db directory, delete your Vagrantfile and create a new one within this new directory, and create a config-db.yaml inside the puphpet directory:

$ tree -L 2
.
├── db
│   └── Vagrantfile
└── puphpet
    ├── config-custom.yaml
    ├── config-custom.yaml.dist
    ├── config-db.yaml
    ├── config.yaml
    ├── files
    ├── puppet
    ├── ruby
    ├── shell
    └── vagrant

Your db/Vagrantfile contents can look like the following:

# -*- mode: ruby -*-

dir = File.dirname(File.expand_path(__FILE__))
dir = "#{dir}/.."

server_type = 'proxy'

VAGRANT_DOTFILE_PATH = "#{dir}/.vagrant";
currpath = ENV['VAGRANT_DOTFILE_PATH'];
if(currpath.nil?)
    currpath = '.vagrant';
end
if(currpath != VAGRANT_DOTFILE_PATH)
    ENV['VAGRANT_DOTFILE_PATH'] = VAGRANT_DOTFILE_PATH
    args = ARGV.join(' ');
    system "vagrant #{args}"

    if Dir.exists?('Directory Name')
        FileUtils.rm_r(currpath)
    end

    abort "Finished"
end

require 'yaml'
require "#{dir}/puphpet/ruby/deep_merge.rb"
require "#{dir}/puphpet/ruby/to_bool.rb"
require "#{dir}/puphpet/ruby/puppet.rb"

configValues = YAML.load_file("#{dir}/puphpet/config.yaml")

provider = ENV['VAGRANT_DEFAULT_PROVIDER'] ? ENV['VAGRANT_DEFAULT_PROVIDER'] : 'local'
if File.file?("#{dir}/puphpet/config-#{provider}.yaml")
  custom = YAML.load_file("#{dir}/puphpet/config-#{provider}.yaml")
  configValues.deep_merge!(custom)
end

if File.file?("#{dir}/puphpet/config-#{server_type}.yaml")
  custom = YAML.load_file("#{dir}/puphpet/config-#{server_type}.yaml")
  configValues.deep_merge!(custom)
end

if File.file?("#{dir}/puphpet/config-custom.yaml")
  custom = YAML.load_file("#{dir}/puphpet/config-custom.yaml")
  configValues.deep_merge!(custom)
end

data = configValues['vagrantfile']

Vagrant.require_version '>= 1.8.1'

Vagrant.configure('2') do |config|
  eval File.read("#{dir}/puphpet/vagrant/Vagrantfile-#{data['target']}")
end

We've now told Vagrant about the new config file at #{dir}/puphpet/config-#{server_type}.yaml -> #{dir}/puphpet/config-db.yaml -> ..

Create as many of these for as many server types as you require.

If you want Puppet to be aware of the server_type value you can pass it in as a facter. Update the vagrant/Vagrantfile-* files with:

puppet.facter = {
    'server_name'      => "#{machine['id']}",
    'server_type'      => "#{server_type}",
    'fqdn'             => "#{machine_id.vm.hostname}",
    'ssh_username'     => "#{ssh_username}",
    'provisioner_type' => 'local',
}

and it will be available within your Puppet manifests as $::server_type.

So what's actually going on here?

Well, in your config.yaml (ie the master config file) you can setup the very basics of your server farm. Each server might require packages like git or have specific ports opened by the firewall. You can add those values there.

Set server-specific applications to not install in your config.yaml:

mongodb:
    install: '0'
    settings:
        bind_ip: 127.0.0.1
        port: '27017'
    globals:
        version: 2.6.0
    databases: {  }

In your config-db.yaml you can set the DB-specific values as you wish:

mongodb:
    install: '1'
    settings:
        ensure: true
        package_name: mongodb-org-server
        bind_ip: "%{::ipaddress_tun0}"
        port: '27017'
        config: /etc/mongod.conf
        replset: rsmain
    globals:
        bind_ip: "%{::ipaddress_tun0}"
        version: 3.2.6
        service_name: mongodb
    databases:
         your_db:
             name: db_name
             user: db_user
             password: 'awesome_password'

Now your database server will pick up these changes and apply as needed. Your app server (or jenkins/proxy/etc) will still see mongodb as not needing installed from config.yaml. You can create fairly complex farms this way, all through a few simpel YAML files.

The last thing to note is that since this is a multi-server environment, Vagrant needs to know what server to work on when you run most commands. You do this by using the machine ID, which you will want to set in each config-#{server_type}.yaml file. Each file can have multiple machines defined for its type. Then simply append the machine ID to vagrant commands:

vagrant up db1, vagrant destroy -f db1, etc.

I hope this answers all your questions, or at least helps you get most of the way to what you are trying to accomplish!