Graphing Chef Application Deploys with Librato

This is a followup to my previous post on graphing Chef application deploys to Graphite. In my new role we use Librato for time series metrics instead of Graphite. One of the best features in Librato is the concept of metric annotation.  Annotations are very similar to the simple vertical lines I previously used in Graphite, but not only do they display in a more intuitive fashion, they include additional information.  In additional to the vertical line, annotations allow for a name, description, and even a URL to be displayed with the annotation.  This is really useful days / weeks later when you don’t necessarily know what version a line corresponds to.

We’re using annotation within Chef to create annotations for each of our applications during a deploy.  In order to accomplish this I created a LWRP in Chef for Librato metric annotations, and we notify to that LWRP when a deploy action is completed.  You can find the code from the provider below as it’s a bit too large to paste inline with this post.  With our core cookbook included, this provider is now available for any of our application cookbooks to graph deploys.  It can either be notified when a deploy resource updates or it can be used via the “after_deploy proc” functionality in the artifact cookbook, which we use for our application deploys.

artifact_deploy 'my_app' do
  version node[:my_app][:version]
  artifact "{node[:my_app][:version]}.tar.gz"
  deploy_to '/mnt/my_app'
  owner 'my_app_user'
  group 'my_app_group'

  after_deploy proc {
    if deploy? || manifest_differences?

      # graph the deploy to librato
      mycorecookbook_graph_deploy 'my_app' do
        version node[:my_app][:version]


One of the side benefits of all this rich deploy information within the annotations is the ability to de-dupe the annotations.  With multiple servers deploying our application we don’t want to fill our graphs with overlapping, duplicate annotations.  Using the Librato gem we can first query Librato first to see if the annotation exists, and skip adding another annotation if we find anything.  This adds to the complexity of this LWRP over my previous Graphite LWRP, but it makes it much more useful on graphs.

End Result

Librato Deploy


# Support whyrun
def whyrun_supported?

action :graph do
  if node.chef_environment  == 'production' || node.chef_environment  == 'staging'
    converge_by("Graph deploy of #{}") do
  else "Not graphing #{} as we're not in Staging/Prod."

def unique_annotation?
  annotations =
  annotation_name = (node.chef_environment + '_' + + '_deploys').to_sym

  # fetch annotations for this app. rescue because 404 == unique (first ever) annotation
    host_annotations = annotations.fetch annotation_name, :start_time => ( - 1800)
    return true

  # unassigned is our source. if its key is not there we got back an empty set and we're unique
  return true unless host_annotations['events'].key?('unassigned')

  # if any of the event titles match the desired title then we're not unique
  message = "Graphing deploy of #{} version #{@new_resource.version}"
  host_annotations['events']['unassigned'].each do |a|
    return false if a['title'] == message


# send the annotation to librato if unique
def graph_deploy
  require 'librato/metrics'
  annotation = (node.chef_environment + '_' + + '_deploys').to_sym
  creds = Chef::EncryptedDataBagItem.load('librato', 'keys')
    Librato::Metrics.authenticate creds['username'], creds['token']

    # bail out if this deploy has already been graphed
    return unless  unique_annotation?

    Librato::Metrics.annotate annotation, "Deployed #{} version #{@new_resource.version}", :source => "Graphing deploy of #{} version #{@new_resource.version}"
    Chef::Log.error 'An error occurred when trying to graph the deploy with Librato'


actions :graph

default_action :graph

attribute :name, :name_attribute => true, :kind_of => String, :required => true
attribute :version, :kind_of => String, :required => true

attr_accessor :exists


Important Note:

This provider requires the librato-metrics gem to be loaded into your omnibus Chef install.  In order to do this you either need to do a chef gem install during compile time or you need to set the following attribute with the chef-client cookbook:

default[‘chef_client’][‘load_gems’] = %w(librato-metrics)

Leave a Reply

Your email address will not be published. Required fields are marked *