A Complete DevOps Pipeline Demo Part 1

In work we’ve discussed tools and implementations we want, yet often we talk about each individually. This makes it hard to really evaluate what the fully implemented pipeline would be like to work with.

 

So to give an idea, I’m pulling together multiple docker containers and connecting them to provide a local infrastructure to test with. This is primarily built around the idea of ChatOps which has to primary benefits:

  1. Developers don’t need to login into multiple services, but can orchestrate everything from a single location
  2. The team will gain cohesion as they learn what everyone else is working on, as status updates come in from different services etc. Osmosis!

DevOps

I’m going to walk through several components:

Docker Machine

I’m running this on OSX and so using Virtualbox.

With the plan to run all these containers off a single docker machine, I’m boosting the memory to 8gigs. If this is happening behind a proxy, set the information and set the no_proxy info. Setting the no_proxy info may take a few tries – setup the machine, get the ip via

docker-machine create \
-d virtualbox \
devops;
docker-machine ip devops

For the rest of the article I’ll assume that the IP was 192.168.99.100. Now run the following code with everything together:

docker-machine create \
-d virtualbox \
--virtualbox-memory 8192 \
--engine-env HTTP_PROXY=<proxy host>:<proxy port> \
--engine-env HTTPS_PROXY=<proxy host>:<proxy port> \
--engine-env http_proxy=<proxy host>:<proxy port> \
--engine-env https_proxy=<proxy host>:<proxy port> \
--engine-env no_proxy=192.168.99.100
devops

 

Mattermost

The first step is setting up the central chat server. Otherwise you could always use Slack or HipChat, but since I work behind a proxy, it is easier to have it behind it and call outside.

docker run --name mattermost -d --publish 8065:80 mattermost/platform

Login to the server at http://192.168.99.100:8065 and setup the initial setup.

 

Now go to System Console:

Mattermost System Console Menu Select

Under the Service Settings, you will want to enable Incoming,  Outgoing Webhooks, and Enabling Overriding Usernames from Webhooks:

Mattermost Webhooks Enabled

Click Save at the bottom and switch back to your channel:
Mattermost Channels
Go to your Account settings, we need to setup the webhooks to work with our bot!
Mattermost Account Settings
Switch to the Integration tab, and click Incoming Webhooks, select the channel you want and click Add:
Mattermost Account Incoming Webhook
Right now, I’m setting this for the default channel – in an actual implementation I’d have a channel per project and potentially generic service channels. Note down this URL, we will use it with Hubot.
Now quickly back to the terminal to run:
docker exec mattermost ifconfig
And you should see the IP address of the container on the docker-machine. Now we are going to increment it by one, as the next container we kick off will register it – which will be our chat bot. Assuming mattermost returned 172.17.0.2, the bot will recieve 172.17.0.3.
Now similar to the Incoming Webhooks, go to Outgoing Webhooks and set the channel, a trigger if you want to, and set the callback URL to http://172.17.0.3:8080/hubot/incoming.
Mattermost Outgoing Webhook
Click Add and we are done with Mattermost!

Hubot

Mattermost is an alternative to Slack that you can manage yourself, and built as an alternative, it is made to integrate well with components built for Slack. Hubot is a chat bot that you can develop and thankfully someone created an adapter for Mattermost: https://github.com/renanvicente/hubot-mattermost
I cloned this to a local machine, called the bot Gort, and built a docker image up from it, based on an existing image.
You will want to update this, set MATTERMOST_INCOME_URL to be the Incoming Webhook URL, and get the token from the Outgoing Webhook and set the variable MATTERMOST_TOKEN to it.
############################################################
# Dockerfile file to build Mattermost Bot container
# AUTHOR: John Doyle
############################################################
# Pull base image.
FROM guttertec/nodejs
MAINTAINER John Doyle <john.doyle@fmr.com>
# Install Yo, Hubot
RUN \
npm install -g yo generator-hubot && \
apt-get update && \
apt-get install -y redis-server && \
rm -rf /var/lib/apt/lists/*
# Define default command.
EXPOSE 6379
RUN /etc/init.d/redis-server start
# Mattermost environment variables
ENV MATTERMOST_ENDPOINT /hubot/incoming
ENV MATTERMOST_CHANNEL town-square
ENV MATTERMOST_INCOME_URL http://192.168.99.100:8065/hooks/xzrqe49sbjg1mncerspt9a31qh
ENV MATTERMOST_TOKEN 63irrdrwejdrtxn1w7wesy6nzh
ENV MATTERMOST_HUBOT_USERNAME gort
# Create Hubot
ADD ./Gort/ /Gort
WORKDIR /Gort
CMD /Gort/bin/hubot -a mattermost

Save this file in the parent directory of the bot and call it Dockerfile.

Now we need to build this into an image with the following command (remove the proxy bit if not needed obviously..) :

docker build --build-arg http_proxy=http://<proxy host>:<proxy port> -t mattermostbot .

The first time it will take 15-20 seconds to compile, future changes will be far faster as it only deals with the deltas.

Now that the bot image is built, you can run the bot in a container:

docker run --name chat-bot -d mattermostbot

We can see the containers running:

docker ps

And we can check the logs of the bot:

docker logs chat-bot

Within mattermost, you can now check if the bot is alive:

Screen Shot 2016-01-31 at 7.26.36 PM

I’ll continue this next with the integration of Jira with Mattermost, and integrating Jira with Bitbucket and Jenkins.