Deploying a TIG Stack

This example shows how to use Bolt to configure and deploy metrics visualization using Telegraf, InfluxDB, and Grafana, all via existing Puppet Modules.

Download Project Files

project/
└── Boltdir
    ├── hiera.yaml
    ├── inventory.yaml
    ├── Puppetfile
    └── site
        └── tig
            ├── manifests
            |   ├── dashboard.pp
            |   ├── params.pp
            |   └── telegraf.pp
            ├── plans
            |   └── init.pp
            └── templates
                └── dashboards
                    └── telegraf.json

Provisioning Nodes

To follow this example and deploy the TIG stack you’ll need at least two nodes - one for the dashboard and one or more for the agents. You can use existing nodes in your system or create them using the provided Vagrantfile.

# -*- mode: ruby -*-
# vi: set ft=ruby :

NODES = 2

Vagrant.configure(NODES) do |config|
  config.vm.box = "centos/7"
  config.ssh.forward_agent = true

  NODES.times do |i|
    config.vm.define "node#{i}" do |node|
      node.vm.hostname = "node#{i}"
      node.vm.network :private_network, ip: "10.0.0.#{100 + i}"
    end
  end
end

You should then generate the SSH configuration for both nodes, which will automatically be picked up by Bolt. Once you’ve generated SSH configuration, make sure you can SSH into the nodes.

mkdir ~/.ssh
vagrant ssh-config | sed /StrictHostKeyChecking/d | sed /UserKnownHostsFile/d >> ~/.ssh/config

Installing Modules from a Puppetfile

Before you can use Bolt to install modules, you must first create a Puppetfile. A Puppetfile is a formatted text file that contains a list of modules and their versions. It can include modules from the Puppet Forge or a Git repository.

This example has a Puppetfile with the following list of modules:

mod 'puppet-grafana', '6.0.0'
mod 'quadriq-influxdb', '0.2.1'
mod 'puppetlabs-stdlib', '5.2.0'
mod 'puppet-telegraf', '2.1.0'

To install the modules in the Puppetfile, run the following command:

bolt puppetfile install

The puppet-telegraf module requires the toml-rb gem, so make sure to install it as well.

/opt/puppetlabs/bolt/bin/gem install toml-rb

Creating the Inventory File

Next, you’ll create an inventory file to specify which nodes to use as part of the plan. If you are using existing nodes in your system, replace node0 and node1 with your own nodes.

---
groups:
  - name: dashboard
    nodes: [ node0 ]
  - name: agents
    nodes: [ node0, node1 ]
config:
  ssh:
    host-key-check: false
    run-as: root

Examining the Plan

Now that all of the required modules have been installed and the inventory file is populated with nodes, we’ll take a look at the plan that will deploy the metrics visualization. In the site/ directory, you’ll find the following plan:

plan tig() {

  apply_prep('all')

  apply('dashboard') {
    include tig::dashboard
  }

  $dashboard_host = get_targets('dashboard')[0].name

  apply('agents') {
    class{ tig::telegraf:
      influx_host => $dashboard_host
    }
  }

  return('grafana_dashboard' => "http://${dashboard_host}:8080")
}

Plans let you compose different tasks together in meaningful ways and can have multiple steps, compute input, and process output. The first step in this plan installs the puppet-agent package and collects facts from each of the nodes in the inventory file.

Next, the first apply block will apply the dashboard manifest, which installs and configures both Grafana and InfluxDB. This manifest inherits a separate class called tig::params, which contains configuration parameters.

class tig::dashboard (
  String $grafana_password = $tig::params::grafana_password,
  String $grafana_user = $tig::params::grafana_user,
  String $grafana_url = $tig::params::grafana_url,
  String $influx_password = $tig::params::influxdb_password,
  String $influx_database = $tig::params::influxdb_database,
  String $influx_username = $tig::params::influxdb_user,

) inherits ::tig::params {
  class { 'grafana':
    cfg => {
      app_mode => 'production',
      server   => {
        http_port     => 8080,
      },
      security => {
        admin_user => $grafana_user,
        admin_password => $grafana_password,
      },
      database => {
        type          => 'sqlite3',
        host          => '127.0.0.1:3306',
        name          => 'grafananana',
      },
    },
  }

  class {'influxdb': }
  influx_database{$influx_database:
    superuser => $influx_username,
    superpass => $influx_password
  }

  grafana_datasource { 'influxdb':
    require           => Influx_database['bolt'],
    grafana_url       => $grafana_url,
    grafana_user      => $grafana_user,
    grafana_password  => $grafana_password,
    type              => 'influxdb',
    url               => 'http://localhost:8086',
    user              => $influx_username,
    password          => $influx_password,
    database          => $influx_database,
    access_mode       => 'proxy',
    is_default        => true,
  }

  grafana_dashboard { 'telegraf':
    grafana_url       => $grafana_url,
    grafana_user      => $grafana_user,
    grafana_password  => $grafana_password,
    content           => template('tig/dashboards/telegraf.json')
  }
}
class tig::params (
  String $influxdb_password,
  String $grafana_password,
  String $influxdb_database = 'bolt',
  String $influxdb_user = 'bolt',
  String $grafana_url = 'http://localhost:8080',
  String $grafana_user = 'bolt',
) {}

Credentials for signing into each service can be kept in a separate location so they aren’t part of the manifest code. This example stores this information in data/common.yaml:

---
tig::telegraf::database: "bolt"
tig::telegraf::username: "bolt"
# In the real world encrypt these with hiera eyaml or store externally
tig::params::influxdb_password: "hunter2"
tig::params::grafana_password: "boltIsAwesome"

The second apply block will apply the telegraf manifest, which installs and configures Telegraf, on each of the agents. Similar to the dashboard manifest, it will also inherit the tig::params class, which contains configuration parameters.

class tig::telegraf (
  String $influx_host,
  String $password = $tig::params::influxdb_password,
  String $database = $tig::params::influxdb_database,
  String $username = $tig::params::influxdb_user,
) inherits ::tig::params {

  $influx_url = "http://${influx_host}:8086"

  class { 'telegraf':
    hostname => $facts['hostname'],
    outputs  => {
        'influxdb' => [
            {
                'urls'     => [ $influx_url ],
                'database' => $database,
                'username' => $username,
                'password' => $password,
            }
        ]
    },
  }

  telegraf::input{ 'cpu':
    options => [{ 'percpu' => true, 'totalcpu' => true, }]
  }

  ['disk', 'io', 'net', 'swap', 'system', 'mem', 'processes', 'kernel' ].each |$plug| {
    telegraf::input{ $plug:
     options => [{}]}
  }
}

The last step in the plan returns results that you can use in other plans or save for use outside of Bolt. In this example it simply returns the address for the Grafana dashboard.

Running the Plan

You’ve downloaded and installed the required modules from the Puppet Forge, populated the inventory file with nodes, and set up configuration parameters in the dashboard and telegraf manifests. All that’s left is to run the plan.

bolt plan run tig

The result of running the plan will look like this:

Starting: plan tig
Starting: install puppet and gather facts on node0, node1
Finished: install puppet and gather facts with 0 failures in 47.67 sec
Starting: apply catalog on node0
Finished: apply catalog with 0 failures in 65.3 sec
Starting: apply catalog on node0, node1
Finished: apply catalog with 0 failures in 22.64 sec
Finished: plan tig in 135.65 sec
{
  "grafana_dashboard": "http://10.0.0.100:8080"
}

Success! If you navigate to the address in the plan’s result you’ll find the Grafana dashboard and be able to sign in using the credentials you used when configuring the dashboard. In this example the user is bolt and the password is boltIsAwesome.

Next Steps

Ready to learn more about how you can leverage Bolt in your workflow? Check out more examples or complete our Hands-on Lab which walks through each of Bolt's features in more detail.

Interested in showing others how you use Bolt? Take a look at contributing an example for this site.