Using Chef to Graph Deploys in Graphite

It’s pretty obvious at this point that I think Chef is a pretty amazing product.  I’m also quite smitten with Graphite for graphing the world, or at least the little part of the world that I’m responsible for.   Chef combined with Graphite can do some pretty amazing things, one of those things being the graphing of product deploys.

I rely on a little trick that Etsy first showed off (codeascraft – track every release ) where you can graph any value in Graphite as a vertical line when the value is 1 or more.  If you create a metric for deploys you can just send values of “1” every time you do a deploy.  Then you can overlay that data on top of your system or network metrics and search for patterns.  If you were to do this via the command line it would look something like this:

echo “servers.my_current_server.deploy 1 $(date +%s)” | nc graphite.mydomain.com 2003

If you want to run this same sort of thing via Chef you can just create an execute resource.  You can notify that resource anywhere in your recipe that you might consider a “deploy” action and you have ohai data that will allow you to send data to the right location.  Here’s an example:

execute “graph_deploy” do
command %Q[echo “servers.#{node.chef_environment}.#{node[‘fqdn’].gsub(‘.’,’_’)}.deploys.my_app_name 1 $(date +%s)” | nc graphite.mydomain.com 2003]
timeout 5
action :nothing
end

Now for the breakdown:  I setup my Graphite system with all servers are in a folder called “servers”, and under that things are broken out by Chef environment so I use the node.chef_environment variable.  From there I need to make sure the value goes under the current server.  I use FQDNs for my nodes in Graphite, but periods are used as the folder delimiter in Graphite so I need to replace the periods with underscores using gsub.  From there I create a folder for all my deploys since I run multiple applications on a system, and within that folder I create the actual metric with the same name as the service.  The resource times out after 5 seconds so if my Graphite server goes down Chef runs continue and the resource never executes on its own.  If I want to execute, I can notify an action of “run” from another resource.

And here’s the end result giving me a deploy to overlay on a few metrics:

Graphite Graph

One comment

Leave a Reply

Your email address will not be published. Required fields are marked *