Radify - Rapid Application Development

Using front-end build tools

Thu, 19 May 2016 00:00:00 -0400

This e-book was originally published in 2015 by Five Simple Steps. Five Simple Steps have closed their online store and graciously returned the rights to the authors, so we are offering this book for free.

Download as PDF

Introduction

This book is aimed at designers who also do front end development. These days, it seems that we all wear multiple hats in our jobs!

Since the turn of the century, web development has become far more sophisticated. This has led to more complexity, and many of us are finding we have to work with many new things. This can seem a little intimidating (I have certainly found myself wishing that the pace of progress would slow down!).

To take advantage of this sophistication whilst reducing its complexity, front-end developers use build tools. These tools allow us to work at a higher level and automate away repetitive tasks, thereby saving time and producing more sophisticated results. Automation streamlines our processes so that we can focus on creative work, rather than spending hours lost in the weeds of tedious tasks.

In this short book, we will go through some of the build tools that are available, what they can do for you, and how you can get started with them. I will refer to some specific tools, but this book will not teach you the specifics of a particular build tool; after all, each build tool is well documented online and in print already. Instead, this book functions as a primer on what build tools are and how to assemble powerful, efficient build workflows with them.

The goal of this book is that, once you have read it, you will be able to improve your workflow by incorporating build tools. I aim to give you the confidence and knowledge needed to dive into this world of powerful and flexible utilities, become more productive, and enjoy your work more as a result.

Let’s talk about workflow

Workflow is the set of tasks that one goes through when doing a job. Everyone’s workflow is different, and you might have multiple workflows for different aspects of your role. We will look at web development workflow and how to optimise it.

Basic web development workflow

In developing websites and applications, you probably have phases that look something like this:

Don’t worry if your workflow is different. I’ve just selected a number of common tasks; the principles found in book will apply no matter what your workflow is.

There are a number of problems with this common, simplistic workflow:

File size - there is no minification of the assets or compression of the images, so the application will perform slower than it ought.
Consistency - pushing to a server can be error prone: assets may be cached, transfers can fail, etc.
Speed of workflow - pushing and refreshing manually is time consuming and takes you out of the "zone" of your workflow.

Let’s add a few more tasks to this workflow to address these issues.

Creating an asset pipeline

To resolve the problems we have identified in the basic workflow, we can incorporate compression, minification, and concatenation. Minification compresses code without changing its functionality. Concatenation combines multiple files together into a single file, improving download speeds. Let’s add these tasks to the process:

This approach, commonly known as the asset pipeline, takes assets from an input area, transforms them, and delivers them to an output area. We will look at how to structure and automate an asset pipeline.

A common convention is to have two separate directories in the root (top level) of your project. The first is src, which contains source code - the files you edit. The second is build, which contains files that are generated and delivered to the end user. You never edit anything in build directly. The project’s specific location is not important, but this book assumes that the project is saved in /home/user/myproject on OSX/Linux or C:\Users\user\myproject on Windows.

In some projects, build may be called dist, which is short for "distribution".

Directory	OSX/Linux	Windows
Source	`/home/user/myproject/src`	`C:\Users\user\myproject\src`
Build	`/home/user/myproject/build`	`C:\Users\user\myproject\build`

Once build and src are separated, move any existing project files into the src directory. These are the files that you will edit. Everything in build will be generated using your build tool - we never edit files in build directly. Hands off!

It’s best to test something that’s as representative as possible of what the end-user will receive. Therefore, it’s a good practice to test in build, not in src, so as to catch any potential problems that the transformations might have introduced.

A full treatment of SCM (Software Configuration Management) is outside the scope of this book, but I will make one suggestion: when using a version control tool like Git, Mercurial, or Subversion, add the build directory to your SCM’s ignore file. You do not want to commit anything in build - it’s considered volatile. In SCM, only the source files are committed to your SCM, and these generate the build. This keeps commit logs clean and uncluttered by the build assets.

To learn about Git, check out Version Control with Git by Ryan Taylor

Here is how the asset pipeline might look:

Assuming that you have two javascript files, src/js/a.js and src/js/b.js, and two CSS files, src/css/first.css and src/css/second.css, the build would output build/js/all.min.js and build/css/all.min.css:

Because we have concatenated multiple assets into single files, we will have to replace the references to the assets. In src/index.html, we have four such references:

<head>
  <script src="/js/a.js"></script>
  <script src="/js/b.js"></script>
  <link rel="stylesheet" media="all" href="/css/first.css">
  <link rel="stylesheet" media="all" href="/css/second.css">
</head>

In build/index.html, this becomes just two references:

<head><script src="/js/all.min.js"></script><link rel="stylesheet" media="all" href="/css/all.min.css"></head>

At this point, the minified and concatenated assets, bundled up cleanly in the build directory, are ready to be pushed to the server. There are some problems, though:

Isn’t this process unnecessarily complex?
How do we know if there are problems with the code?
How do we manage this process?

Here is where we introduce the wonderful world of build tools!

Automated workflows with build tools

A build tool is software that automates the transformation of source code into a deliverable. Any time a change is made to the source code, the build tool should be run - usually from the command line. The build tool is how we implement asset pipelines:

Available build tools

There are many build tools available. This book focuses on the web and therefore looks at those tailored to HTML, CSS, and JavaScript. Grunt and Gulp, the two most popular, will be our focus because of their widespread adoption. There are many others, such as Broccoli, Middleman and Brunch, but this book’s principles apply to all build tools.

If your needs are simple, the package manager NPM can be used as a build tool - NPM along with Browserify can be quite a powerful combination. See http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/ for details.

The build file

The build file tells the build tool what to do. It defines a set of tasks that the build tool can perform. In Grunt, this file is called Gruntfile.js. In Gulp, it is called Gulpfile.js. As a rule, the build file goes at the root of the project, above the build and src directories.

Item	OSX/Linux	Windows
Source dir	`/home/user/myproject/src`	`C:\Users\user\myproject\src`
Build dir	`/home/user/myproject/build`	`C:\Users\user\myproject\build`
Build file (Gulp)	`/home/user/myproject/Gulpfile.js`	`C:\Users\user\myproject\Gulpfile.js`
Build file (Grunt)	`/home/user/myproject/Gruntfile.js`	`C:\Users\user\myproject\Gruntfile.js`

Tasks

Build tools run tasks. A task is simply an action or a set of actions that can be performed. For example, a task could be "copy files from src to build".

In most build tools, tasks are composable, meaning that a task can depend on or delegate to other tasks. For example, task copy-assets could depend on task clean, so that whenever copy-assets is run, clean is automatically run beforehand. The advantage of composability is that the code for the clean task won’t be repeated every time you need to call it.

Tasks are stored inside the build file; here is an example of a build file that contains some tasks:

Anatomy of a task

A task generally has these attributes:

Name: The name of the task. Most tasks can be run in isolation by passing the task’s name to the tool on the command line. For example, grunt clean or gulp clean would run the clean task.
Description: Some build tools supply an optional task description, an explanation of the task that doubles as documentation for people on your team.
Dependencies: A list of tasks that must be run before a certain task can run. For example, the build task may have clean as a dependency, meaning that if you call the build task, the clean task automatically runs first.
Functionality: This is the specific code or configuration that you write to implement your task. For example, the scripts task could minify and concatenate a set of files from src and then copy them into build/js.

Creating your first build file

We will create a build file that automates the workflow that we have defined. I’m not going to write the actual code here, as it will differ based for each build tool, but I will represent it in an abstract manner.

These tasks will:

Clean the build directory
Compress and minify assets
Copy the transformed results into the build directory

Here is the order in which the tasks will run:

Some build tools can run tasks in parallel; others will run them one at a time in sequence. In the diagram above, we have assumed that we can run scripts, styles, images and html in parallel.

Explanation

Task `clean` - cleans out `build` directory

This simple task cleans out the build directory by deleting everything in it.

Task `build` - calls the subtasks

Depends on task clean via the dependency mechanism discussed earlier, so when build is called, it first triggers clean. After clean completes, the build task delegates to the scripts, styles, images and html tasks.

Task `scripts` - transforms javascript

Takes src/js/a.js and src/js/b.js, minifies them, then concatenates them into build/js/all.min.js.

Task `styles` - transforms CSS

Takes src/css/first.css and src/css/second.css, minifies them, then concatenates them into build/css/all.min.css. For designers writing in Sass or LESS, styles will also compile those files into CSS before minification and concatenation.

Task `images` - compresses images

Takes the images in src/img, compresses them, and outputs them to build/img.

Task `html` - transforms HTML

Takes src/index.html and minifies it. It then modifies the HTML, directing the asset paths to build, not src. The output is then directed to build/index.html.

Once the build task completes, you have your complete website, ready for use in the build directory.

Plugins

For each task, there is an installable plugin for your build tool. This is great because it means you usually have to write very little code.

The Gulp team maintains a list of verified plugins and their community ensures that there is only one plugin for each task. In contrast, the Grunt community often offers a number of alternatives for each plugin. The grunt-contrib-* plugins are the best starting point because they are officially maintained.

The specifics of how to install and use these plugins are beyond the scope of this book.

Task	Gulp plugin	Grunt plugin
clean	`del`	`grunt-contrib-clean`
scripts	`gulp-uglify`	`grunt-contrib-uglify`
styles	`gulp-minify-css`	`grunt-contrib-cssmin`
images	`gulp-imagemin`	`grunt-contrib-imagemin`
html	`gulp-minify-html`	`grunt-contrib-htmlmin`

Look over the sample project for this book at https://github.com/gavD/5ss-build-tools. There, you will find example build files for Grunt and Gulp that follow this approach.

Automating your workflow

At this point, the areas in green are automated:

Design and editing source code in src isn’t automatable - if it were, robots would be doing our jobs. Therefore, we will omit these workflow stages from the future diagrams. The rest of these tasks, however, we can use build tools for.

Task `watch` - automating your automation

Right now, you are probably running the tool on the command line every time you make a change. This is pretty laborious, and also error prone - chances are you will make a change, forget to run the tool, reload the browser, and be confused as to why your changes aren’t showing up… I know I’ve certainly done that!

Thankfully, we can automate this. We will create a new task, watch, which will monitor the src directory, and whenever anything changes in there, it automatically runs the default task, which has the build task as a dependency:

Now, the build tool runs automatically when anything changes in src, so that any time you update your source code, the website is regenerated automatically. This automates the build:

Task `serve` - run a local server

Instead of constantly pushing code up to a web server, build tools allow you to run a server locally, which serves the build directory, so you’re seeing the compressed, minified, concatenated versions of files, just like your users will. This means that we can remove the "Push to server" stage of the workflow entirely:

Don’t worry if you were hoping to cover deployment in your build workflow; we will talk about deployment later.

Task `refresh-browser`: refresh your browser

One task that we spend a lot of time doing is making changes, then going over to a browser and reloading the page. A LiveReload will ensure that when a build occurs in response to a change to src, not only is the build task called, but also the browser is automatically reloaded:

We’ve now automated a huge amount of the workflow:

What do we use to write these tasks?

Task	Gulp plugin	Grunt plugin
`watch`	`gulp-watch`	`grunt-contrib-watch`
`serve`	`gulp-connect`	`grunt-contrib-connect`
`refresh-browser`	`gulp-livereload`	`grunt-contrib-watch` (built in)

Finished build file

We end up with a set of tasks that we can run. Here are the "top level" tasks, i.e., the ones that we will commonly call from the command line:

Task	Dependencies	Description
`default`	`build`	Default task so that when your build tool is called with no parameters, it simply delegates to the `build` task.
`build`	`clean`, `scripts`, `styles`, `images`, `html`	Minifies, concatenates, and compresses assets, then moves the compressed versions into the `build` directory.
`watch`	`live-reload`, `serve`	Launches the local server, which serves the `build` directory. Watches the `src` directory. Any change to any file inside `src` will trigger the `default` task. This is the task you will run when you are working on your project, and it will whirr away in the background keeping everything up to date for you.

What else can build tools do for us?

We’ve automated the workflow that we defined earlier, but build tools can do lots more. Here are a few examples.

Linting

Linting, also known as static analysis, is the automated inspection of your source code (files in src). Think of it as having a more experienced colleague checking your work and making sure that you haven’t made any common mistakes.

JavaScript can be automatically checked using a tool called jshint. This can check for common performance, security, and stylistic problems. Following the recommendations of jshint makes your code standards-compliant, and protects you from common clangers.

Similarly, CSS can be linted using tools like csslint. This can help you to automatically detect whether you’ve used invalid or deprecated rules. It can also help you to make more efficient CSS that the browser can use without friction.

HTML can also be linted!

I generally recommend obeying every recommendation that linters make - as legendary programmer John Carmack once put it, "If you have to explain it to the computer, you’ll probably have to explain it to your colleagues."

All linters can be "tuned" to your specific needs - for example, if you are breaking one rule for a good reason, then you can tell your linter to ignore it.

Here are some example tools you could use:

Functionality	Gulp plugin	Grunt plugin
Lint css	`gulp-csslint`	`grunt-contrib-csslint`
Lint javascript	`gulp-jshint`	`grunt-contrib-jshint`
Verify html	`gulp-htmlhint`	`grunt-htmlhint`

CSS preprocessing

CSS is great but it’s fiddly. A preprocessor makes CSS far easier to work with. I use LESS day-to-day, but there are many others, including Sass and Stylus. All preprocessors add features like variables, mixins, and so forth, which greatly reduce the amount of code you need to write and make it far easier to maintain.

Using a preprocessor, the styles task can transform styles into raw CSS. This means that you would never need to edit raw CSS; you could work entirely in the far more pleasant world of LESS (or whichever you prefer).

Functionality	Gulp plugin	Grunt plugin
LESS to CSS	`gulp-less`	`grunt-contrib-less`

Script preprocessing (transpiling)

Instead of plain JavaScript, you might be using something like CoffeeScript. You would need a transpiling stage, which converts from CoffeeScript into JavaScript.

You might also be using a more cutting-edge version of JavaScript than is available in current web browsers. At the time of writing, ES6 is a forthcoming version of JavaScript that is not available for general usage in many web browsers. Therefore, we can use tools to transpile it into ES5.

Once you have these tools in your workflow, you would have your CoffeeScript or ES6 code in src, and standard ES5 code in build. It makes debugging a little more difficult, but it can be worth it if you want to use cutting edge features.

Functionality	Gulp plugin	Grunt plugin
CoffeeScript transpiler	`gulp-coffee`	`grunt-contrib-coffee`
ES6 transpiler	`gulp-babel`	`grunt-babel`

Push to server

Build tools can even be used for deployment. There are all sorts of options here and it is out of our scope to cover them in detail, so here are a few examples:

Functionality	Gulp plugin	Grunt plugin
FTP	`gulp-ftp`	`grunt-ftp-deploy`
Github pages	`gulp-git-pages`	`grunt-gh-pages`
rsync	`gulp-rsync`	`grunt-rsync`

Best practice

Generally, you can use a build tool for any automation task. As with any software, build tools are open to misuse, however, so here are some guidelines to help you to make sure you are using the build tool as it was intended.

Never edit anything in the `build` directory

You should never manually edit anything in the build directory. Similarly, your build tool should never modify anything in the src directory.

If there is anything that you have to do in the build directory, you can almost certainly find a way to get the tool to do it for you.

Never let your build tool modify anything in the `src` directory

Conversely, if your build tool modifies your src directory, you are going to have a bad time. Such behaviour breaks the "pipeline" approach and can create circular actions that never fully resolve. Therefore, you need to be confident that your build tool will stay out of the src directory entirely.

Think of it like this: src is yours, build belongs to your build tool.

Short, sharp, composable tasks

Whilst you CAN create tasks that are huge and have a large amount of code, it is usually better to separate these into smaller tasks and compose them using task dependencies.

Your normal workflow should be encapsulated by a single task

You shouldn’t need to be constantly running a bunch of separate tasks. Instead, one task should rely upon or kick off the subtasks that it needs. Generally, you should use some kind of watch and run it in the background. As you do your normal work, it will detect changes, and then run everything it needs to.

If you’re finding yourself constantly having to stop your watch task and run things, then consider putting them into your main (default) task.

Have tasks that aren’t part of your main workflow as separate tasks

Conversely, for tasks that aren’t part of your regular workflow, don’t have the main workflow call them.

For example, you don’t want to deploy every tiny little tinker you make in your local workspace. Therefore, you might have a task like deploy that you only call once you’ve done all of your testing and you’re confident that it’s time to go live.

Don’t store credentials in plain text

Speaking of deployments, it might be tempting to store things like passwords or authentication tokens in your build file. This is generally a bad move from a security perspective - if someone accesses your build file, they could compromise your live systems.

A full treatment of security issues is outside of the scope of this book, but suffice to say, don’t store credentials in plain text and certainly don’t keep them directly in your build file.

Debugging with source maps

If you have "compiled" (minified, concatenated) JavaScript code in build, it will look different to your code in src. It will have had whitespace removed, variable names shortened - all sorts of changes that make it hard to debug. You can use a source map to help you here. A source map holds information about your original source files. Browsers like Chrome and Firefox support source maps, and enable you to debug as if you were using your original code. This is really handy as it means you are testing the same code the user would receive, but debugging it in its original form.

Similarly, with a CSS preprocessor like LESS or SaSS, the CSS in build is different to the CSS in src. LESS and SaSS support source maps as well, so you can use the same approach.

Being idiomatic

Try to ensure that your project is idiomatic. By that, I mean that you should follow the conventions that other people use for similar projects. This has the following benefits:

Easier to apply code snippets you find online
Easier to explain to people than some bespoke system
The wheel remains un-reinvented!

There are a couple of ways you could go about this. Firstly, you could clone an existing repository and modify it to suit your needs. Feel free to clone the sample source code for this book from https://github.com/gavD/5ss-build-tools.

The second option is to use a scaffolding tool. These are tools that create a "skeleton" project for your app that has the tools you will need installed and ready to use. So, you don’t have to create the build and src directories, and you don’t have to create your build file - your scaffolding tool does it all for you. Handy!

If you’re starting with a brand-new app, Yeoman is a great scaffolding tool that uses generators to get you up and running with a project structure. It supports a huge range of project types. For example, are you building an AngularJS app? Then use the yeoman angular generator! Working on a PhoneGap project? Then use the yeoman phonegap generator! No matter what you are building, there is likely to be a Yeoman generator that can help you to get started.

Go forth and build!

In this book, we started out by looking at common web development workflows. We then went on to look at how we can improve upon that workflow. We then took a detailed look at what a build tool is, what a build file is, and what tasks are. We then went through creating some tasks to automate the workflow.

Thanks for reading, I hope that this book has been a useful introduction to the wonderful world of build tools!

Thanks

Many thanks to the following proof readers for valuable insights and corrections:

Nate Abele
Andrew Canham
Kathryn Davies
Aryella Lacerda
John Mercer

Download as PDF

Use the button below to get the book in its original form.

Download as PDF

Picking up the ball

Wed, 27 Apr 2016 00:00:00 -0400

Recently, Mark posted an article about recovering from a failed relationship. This got me thinking that a certain amount of failure should perhaps be expected as part of doing business - not least because failure can be incredibly instructive.

I've made some great mistakes in my time. In my first job, I accidentally emailed some classic Gothic fiction to a large number of customers. Thankfully, I was not subjected to the fate as Poe's protagonists, but I've never forgotten the sweaty palmed terror as the realisation gradually dawned on me that I had failed to check the "test mode" box...

At conferences and meetups, in blog posts and tweets, successes are shouted. Perhaps there is something of a cult of competence. Most of us are obsessed with appearing hyper-competent, but really, we are not paid to be right all the time - we are paid to build value. Naturally, all organisations want to look as competent as possible, but the truth is that every individual makes mistakes, and no process can prevent them all. As organisations are made up of people, it follows that every organisation makes mistakes.

In a 2012 talk entitled "the loneliness of the long distance coder", Carey Hiles admitted to some priceless blunders. Four years on from that talk, I still hear people talk about how it was encouraging to hear someone admit to mistakes. Similarly, Craig and I occasionally talk about screw ups on the Never Out of Beta podcast, and the feedback from our listeners has been that it's good to hear experienced engineers talk as candidly as possible about their weaknesses.

If taking ownership of our fallibility can be encouraging to our peers, can there also be benefits to our organisations? Perhaps even benefits to our clients?

Prevention of and elegant recovery from failure

Pretending we are perfect as an organisation is fooling no-one, least of all ourselves. Accepting occasional failure as part of doing business enables us to be proactive on three fronts:

1. Reduce the likelihood of failure

We learn, grow and improve over time to make a better class of mistake. That is, instead of making "novice errors" with wide consequences, we make more sophisticated clangers that have fewer ramifications. A more experienced engineer, for example, will partition work in such a way that a failure in one module will not disable an entire system. The novice, having experienced fewer errors, may not think in such terms. The experienced engineer will still have problems, but they are of a better class - no longer the sweepingly disastrous bumblings of the neophyte.

2. Reduce the cost of failure

As part of accepting that failures occur, it is wise to make it easier to recover from problems. This can include technical aspects such as having a disaster recovery (DR) strategy, continuous deployment, and rolling backups. The non-technical side is even more important - you should also have good, honest, genuinely respectful relationships with your clients. Think about your relationships with your family - some of the same things that allow you to recover from arguments with your family - respect, trust - come into play when a deployment goes wrong and you have to tell the client.

3. Respond proactively to failure

This may sound cliched, but try treating a mistake as a lesson - it's an opportunity to improve your process, tooling or knowledge. The very fact that a failure has occurred demonstrates room for improvement.

Going public? Boundaries in response to failure

Sometimes, we can derive general instructions from a specific failure and publish those in the form of secondary deliverables. This is not something to be taken lightly, however - whilst we believe that we shouldn't shovel all our errors under the carpet, it's naive to think that we should broadcast every mistake.

Once we have resolved an issue, there are 3 sets of people that we may inform:

We should generally inform the first set, our colleagues, of most non-trivial problems - there are certain problems that can be resolved internally and need go no further. This is where an organisation does a lot of its learning and growing. Agile retrospectives are just one example of how this may be applied.

The second set of people, clients, should not be bombarded with every triviality, but if an incident seriously affected them, it is worth discussing with them how it can be prevented and whether things can be improved. Discussion of mistakes with clients should:

Be honest
Be timely (otherwise trust becomes eroded)
Be accompanied with a strategy to prevent the issue in future
Give them options, but leave them in charge

From the client's perspective, they should see our organisations respond proactively and professionally. This can actually improve trust and help us to work together with our clients: psychologically speaking, occasional failures, if recovered from professionally, actually improve the perception of an organisation.

Our first responsibility is to our clients and we must never leave them exposed; we should never disclose anything that can harm our client's reputation, business or security. If we can anonymise an incident, we may, with a client's consent, consider "go public" to our third set of people - the world at large - with some lessons learned from the problem. This diagram defines a potential flow for responding to a failure and maximising benefit without exposing a client:

Anything you "go public" with must be anonymised and must not give away any of the client's information

Is there an optimal failure rate?

We've suggested that occasional failures can help us to reassess our processes, tools and skills and demonstrate our ability to react, change and improve. If, however, we make mistakes frequently, are simply incompetent and do not benefit. If, however, we never make mistakes at all, perhaps we are stagnant and in a false sense of security. Perhaps there is such a thing as an optimal failure rate:

This hypothetical sweet spot could be expressed as "things rarely go wrong, but when they do, the organisation responds very quickly," a state of affairs which inspires maximum confidence in clients.

Failures can be classified into two camps:

"True" failures that affect your customers and their customers
Failures caught by process and automation

The latter category of failures prevented from ever manifesting "in the wild" by processes, practises and automation you have designed to avoid "true" failures.

The ratio of these failures can be compared to an Optimal Failure Ratio (OFR). This allows us to accept the potential for future failure and continually improve guards to protect our colleagues, clients and the world at large from ever having to experience them.

The reason why ratio is better than rate is because rate is just a indicator, ratio is a measure against something else. If you can't measure you're relegated to being only an observer. Failure rate is interesting, while ratio is actionable.

Wrapping up

All organisations and individuals make mistakes, but we can use them to learn, improve and build trust. There are even advantages to occasional failures, provided we respond to them professionally and do not exceed the Optimal Failure Ratio (OFR). We can also derive some useful secondary deliverables using lessons learned from failures.

Rise of the machine users

Wed, 06 Apr 2016 00:00:00 -0400

With the rise of microservice architectures, an increasing amount of what we do is weaving parts of Amazon Web Services (AWS) into distributed applications.

To manage these components, we use a large amount of automation. To use AWS programmatically, you need keypairs, which are associated with user accounts. In this article, we’ll take a look at our approach to managing these accounts.

We’ll talk about AWS here, but the principles in this article cover can be applied more broadly.

Common mistakes - overpowered users

Many organisations simply have one AWS user with administrative privileges and generate a keypair for this user, which they use for everything. Similar to the "God Object" anti-pattern, this is lazy and very dangerous for three reasons.

Firstly, if the API keys for this machine user are compromised, an attacker can do anything in the application.

Secondly, if an organisation is using a key/value pair for a "real" user, that user could end up locked out of their own account!

Thirdly, even if there is no compromise, the fact that automation has more privileges than it needs means that it can misbehave - for example, it might terminate all nodes when you only meant to terminate a certain group of nodes, or it might drop a database.

Better practise - machine users

The essence of security is the principle of least privilege - only give the minimal permissions necessary. It follows that our objectives in setting up our automation users are:

Limit what an attacker can do if a keypair is compromised
Use a keypair that is not tied to a real human being
Limit what an application can do if it misbehaves

To do this, we configure machine users. Machine users are user accounts that don’t have a login password — they can’t use the AWS console. They’re not people, they’re simply accounts that are created to fulfil specific purposes. When faced with an automation task, we create a specific AWS user to do that task and only that task — we never give it administrative privileges, or access to services it doesn’t need.

We generally set up multiple machine users for a project: one might have the responsibility of provisioning EC2 nodes, whereas another would be responsible for managing a static S3 bucket where the application uploads files. After all, provisioning servers in DevOps land and end users uploading files in userland are very different concerns, so it makes sense for separate machine users to be assigned these privileges.

This approach meets our objectives:

If a key is captured, an attacker can only do what that machine user can do.
If a key is captured, no human’s account is compromised. We have a clean separation of responsibilities between humans and automatons.
If the application misbehaves, it is doing so in a "sandbox" and the catastrophe will be contained.

There are further advantages:

The machine user can be invalidated in one fell swoop.
In logs, we can clearly see which machine users are doing what, rather than having some generic user doing everything. This helps us to identify misbehaviour.

Essentially, we have restricted each machine user on 2 axes:

What the machine user can do
Where the machine user can do it

Similar to the DevOps principle of "treat servers like cattle, not like pets", machine users should be treated as disposable. They are not precious things, so don’t give them a funky name that might cause you to get attached to them!

Wrapping up

Do you use cloud services? Are you using machine users? A good place to start is the AWS documentation on IAM (Identity and Access Management) roles.

Aspect-Oriented Programming In PHP

Tue, 29 Mar 2016 00:00:00 -0400

There are always units or features in an application which need to "cut across" many different parts of the architecture at once.

This problem is not really obvious when you are using a "full-featured" framework. Indeed, since your issue is probably a common one, there's a good chance the framework resolved this cross-cutting issue either by sacrificing separation of concerns or by providing an abstraction on top the framework to solve it. A number of frameworks use an event-driven architecture to address the cross-cutting problem. But there's always a point where the framework can't provide the control you need in a specific piece of logic. This is particularly the case if you're using a microframework or prefer building your own one with specialised libraries. The more your application respects the separation of concerns principle, the more your cross cutting needs will come to play a crucial role in your architecture.

Aspect-Oriented programming in PHP

Aspect-Oriented Programming, or AOP, is a programming paradigm that focuses on the organization and modularity of cross-cutting concerns. Use cases can be, for example, ACL, logging, error handling, or caching.

PHP's built-in (internal) assumptions (i.e. when you define a function/constant/class it stays defined forever) make the AOP paradigm difficult to implement.

Li3 was the first one to address the cross cutting issue by offering a filter mechanism to allows to filter method's logic through closures. The Li3's implementation requires manually adding some necessary boilerplate code to make a method filterable. With such a constraint, AOP techniques are limited to filtered methods.

The others well known AOP implementation in PHP are:

AOP (PECL extension)
Go!

The AOP PECL extension is an interesting approach and at the same time a risky gamble since it's a PECL extension which is not a widely supported. The other option is the Go! library which is an AOP implementation which patches PHP code on the fly to allow AOP techniques to be used.

There are other existing implementations, but most of them are workarounds based on top of proxies (to the best of my knowledge), an approach which has a lot of limitations.

A new kid on the block

Automatic code generation is not new in PHP, and is used in a number of libraries like ProxyManager to name one. And thanks to the adoption of Composer, Go! also demonstrates that Just In Time code patching was possible.

If code generation is easy, code patching is a bit more complicated. First, because PHP doesn't provide a built-in code parser, and also because there aren't a lot of libraries addressing the PHP code parsing problem. The best known is the PHP-Parser library. PHP-Parser is a great tool, but the fact it simply ignores whitespace formating in its generated abstract syntax trees makes it problematic to use for some code patching. Indeed, the code to be patched is the real executed code. So if you want to have your backtrace accurate on errors, the line numbers need to be respected in patched files.

So we used the Kahlan's JIT code patcher for this task. Kahlan is a new Unit & BDD test framework which allows stubbing or monkey patching your code directly like in Ruby or JavaScript, thanks to some JIT patching techniques. Under the hood, this library is based on a rudimentary PHP parser but it has proven to be fast and stable enough for our needs here.

The filter library is available at github.com/crysalead/filter and can be used like the following.

First, the JIT code patcher must be initialized as soon as possible (for example just after including the composer autoloade) like so:

include __DIR__ . '/../vendor/autoload.php';

use Lead\Filter\Filters;

Filters::patch(true);

Note that code patching will only work for classes loaded by the Composer autoloader. If a class is included using a require or include statement, or has already been loaded before the Filters::patch(true) call, it won't be patched.

By default all patched code will be stored in the /tmp/jit folder but you can set your own:

Filters::patch(true, ['cachePath' => 'my/cache/path/jit']);

Cached files will be regenerated automatically every time a PHP file is modified.

Warning! Because Filters::patch(true) is the no-brainer way to setup the patcher, you should keep in mind that all your code will be patched. Having all methods of your codebase (and vendor code) to be wrapped inside a filter closure can be time-consuming.

Fortunately, if cross cutting concerns play a crucial role in well designed code, it's only required in a couple of methods in a project. So, the prefered approach is to only patch the methods which are going to be filtered:

Filters::patch([
 'A\ClassName',
 'An\Example\ClassName::foo',
 'A\Second\Example\ClassName' => ['foo', 'bar'],
], [
    'cachePath' => 'my/cache/path/jit',
]);

That way, you can choose to patch all methods of a specific class, one method only, or just a couple of them.

The Filter API

Now that the JIT patcher is enabled, let's create a logging filter:

use Chaos\Filter\Filters;
use Chaos\Database\Database;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;

$logger = new Logger('database');
$logger->pushHandler(new StreamHandler('my/log/path/db.log', Logger::WARNING));

Filters::apply(Database::class, 'query', function($next, $sql, $data, $options) use ($logger) {
    $logger->info('Ran SQL query: ' . $sql);
    return $next($sql, $data, $options);
});

The above example creates a SQL query logger filter for the Chaos database library. More information on the filter API at github.com/crysalead/filter.

Conclusion

AOP is, in my opinion, the true answer for all cross cutting concerns. All other abstractions are at the same time unnecessary and most of the time limited to a specific scope. I have not given up hope that one day PHP will provide a built-in Aspect-Oriented Programming API, but for the moment, I think JIT code patching is the best option, since the benefits largely outperform the CPU overhead which is almost negligible when the JIT code patching is not applied globally.

Avoiding Bumbling Professionals

Tue, 15 Mar 2016 00:00:00 -0400

A few years ago April 15th started with a bang. We were embarking on a year-long journey of unanticipated costs, cleaning up our books, tax filings, and corporate records. The problem was, we had no idea it was coming. It started with an impromptu conference call: it was our certified public accountant (CPA) telling us that we’d need to file an extension and make huge estimated tax payments that were due by the end of the day.

But this was only the beginning: it took the next several months for the full extent of our CPA’s malfeasance to become clear.

Today we are in a completely different place thanks to our new accountant Regina O'Keefe (CPA, CVA) at the firm Gable Peritz Mishkin, LLP.

Looking back at the journey we took in 2014 from that shocking call to finding Gable Peritz Mishkin to setting things straight holds a number of important lessons for anyone looking to grow a business or clean up from a failed professional relationship.

No business is an island and any growing business can benefit tremendously from the hard work and pertinent expertise of the right professionals. But the stakes are high because choosing the right professional can be tricky and getting the wrong professional can serve the equivalent of a punch to the gut.

Here are some notes on how to find the best professional and avoid what I now refer to as a "Bumbler.”

Get To Know Yourself

This is the place to start. Make a short list of things that are important to your organization and will be absolutely necessary for a successful business relationship with a Professional. This might be as fundamental as mine were: “highly responsive and proactive” or as practical as “is comfortable with Google Docs.”

If you are very proactive maybe you can successfully work with a more reactive firm but it’s important to know for sure so that you don’t end up in a reactive/reactive relationship.

Defining the key requirements helped me narrow in on exactly who would be best for us. The two firms we shied away from had only part of the equation. The first was technically savvy but was not proactive and ignored several follow-up e-mails and phone calls. The second was proactive but was clearly uninterested in the fact that our firm doesn’t use paper fax machines to share documents.

Both looked great on paper -- well recommended, professional, and easy to talk with: But by knowing ourselves we avoided getting swept up in those things and could stay focused on finding the best possible match for us.

Do Your Own Research

We got burned by a Bumbling CPA that was/is very well-respected and enjoys glowing references from many businesses in our industry. They even market themselves as cutting-edge specialists for small, tech-savvy organizations. A seemingly perfect fit. A quick scan showed every indication that we should have been treated as well as everyone else. (And, to be fair, I’m sure there are clients who did not have the same disastrous outcome as us.)

But not only were we not treated fairly -- we were actively harmed. So how does one avoid being underserved (or even worse, sucker punched) by a Bumbler?

By the time we were again searching for our next CPA I knew to hold recommendations loosely and (once I had someone whom I felt met our personality well) was prepared with the following key questions:

How big is your client base & firm? (What’s the fish-size/pond-size ratio?)
What are your specialties? (Familiarity with our filing status, states, and industry?)
Who will be handling your account primarily? (Will we be quickly relegated to the lowest-paid team member?)

I asked these question and outlined the background of the damages we were trying to repair. Instead of a slick-talking sales approach or lofty corporate goals the answers I received from Regina were no-nonsense and pertinent to our unique situation -- and without ego. This is, I realized, the hallmark of a true professional: actually listening and responding.

Avoid Billing Conflicts

I doubt many CPAs do blanket use-it-or-lose-it style monthly retainers. Our Bumbler did. We signed up for a monthly retainer that included a defined list of online software and a number of tasks and quarterly meetings. This pricing plan seemed progressive at first but, in retrospect, it created a clear conflict of interest because we were committed to making the same high month-after-month payment without clearly defined deliverables associated with the cost. This pricing approach itself is fine but when combined with a lack of structure and definition it provides too much incentive to under-deliver. That is, the firm’s character must be strong or the relationship can become a money grab.

Pricing structure is often the clearest window into the heart of an organization.

Deposits and retainers are normal; don’t get me wrong; but unless you have a high level of trust, it’s important that billing is transparent, detailed, and is tied as tightly as possible to work actually performed. Billing that makes little room for accountability is a clear warning sign.

Be Sensitive to the other Warning Signs

If you are worried that you might already be working with a Bumbler, here are a few warning signs we ignored until it was too late:

The fees were growing and increasingly tied to gimmicks like secure file vaults and overkill processes that we didn’t ask for and never used. Worse still, they should have known this or inquired after a year or so. At some point we became a billing item and not a client.
Meeting schedules spelled out in the contract never materialized. Our Bumbling professional was not interested in leading the conversation or, it seemed, helping at all. This is critical because the passive approach was the opposite of how they characterized their service in the agreement. They weren’t being honest with themselves or us.
In the few meetings actually held the questions raised demonstrated a lack of basic knowledge of our business... even years (and a pile of money) into our working relationship. We had made a significant financial commitment which wasn’t being reciprocated.
Missed deadlines. We leaned on our CPA to be proactive and deal with legally-required deadlines without hand-holding. When several early tax deadlines were flagrantly ignored we should have taken decisive action.
There is a saying, “All style and no class.” One thing that irked me early on but that I couldn’t articulate until it was too late was the amount of effort Bumbler put into things other than doing the work we hired him to do. Beware when you see a lot of style but no class.

I hope that these tips will help you avoid a year like our in 2014. It’s always hard work to find the right partner and it’s often tempting to take the easy road (after all, you’re paying money to create less work not more, right?) but, in the end, a long term relationship that works will be worth the upfront cost of the search.

Have you ever had a Bumbler that hurt your organization and learned something important from the experience? Feel free to help future readers by adding a comment.

Routing in PHP - a Complete Benchmark

Tue, 01 Mar 2016 00:00:00 -0500

Over the last 2 years, FastRoute has become the de facto standard PHP routing solution. It's now used in popular frameworks like Slim3 and lumen, and is the foundation of other advanced routing libraries.

The basic idea behind FastRoute is to combine routes regular expressions together in chunks to minimize preg_match() calls. At a first glance nikis's article "Fast request routing using regular expressions" makes a lots of sense. After a couple of hours of benchmarking, however, the regular expressions combination doesn't seem to live up to its promises.

The story began when I was looking for a routing library. FastRoute was obviously one of the front runners. However, FastRoute doesn't support advanced features like reverse routing or subdomain routing out of the box, and despite the large numbers of routing libraries available, I didn't really find the one which suited all my needs. So, I decided to create a new routing library following nikic's precepts.

Once done, I decided to do some benchmarking to evaluate the final outcome - but the results weren't as good I'd hoped. And that's where things became interesting...

Establishing a Benchmarking Process

Having seen that my implementation did not perform as well as FastRoute, I decided to benchmark other routers. Benchmarks for php-router-benchmark also seemed pretty poor compared to FastRoute. So, I started to investigate to understand why FastRoute was beating its competition.

I quickly realized that the benchmarking script wasn't really realistic. Moreover, it only benchmarks the actual routing step and not the route creation phase. So I decided to create a more elaborate benchmarking script to carry out my investigations.

So to have a larger overview, I decided to benchmark the three different situations:

the best case (i.e when a request matches the first route for all HTTP methods)
the worst case (i.e when a request matches the last route for all HTTP methods)
the average case (i.e the mean which is probably the most realistic test).

There are, however, many different ways to configure routes. For example, in a controller/action strategy, each route generally matches a single HTTP method only. Conversely, in a REST approach, the same route's pattern can handle several HTTP methods. So, to take advantage of all different routing implementation's optimizations, I ran the benchmark using the following sets of routes:

in the first set all routes matches all HTTP methods.
in the second set all routes match only a single HTTP method.

Taken separately, none of these sets are very realistic, but it should allow us to triangulate a real world scenario.

I then picked up the following routing implementations to do the benchmarking:

The Results

Instead of copy pasting the whole benchmark results in this article, I'll just pick up the most interesting parts.

So the first bar chart is obtained with 100 routes using the route set number 1:

While the second bar chart is obtained with 100 routes using the route set number 2:

As you can see, performance really depends on the route set. The different performances obtained by FastRoute can be explained by how routes are stored internally. FastRoute indexes routes by HTTP method names. So when a PUT request is done, FastRoute only need to iterate over PUT routes. Whilst this optimization works well when routes matches only a single HTTP Verb, it doesn't help much when routes matches all HTTP Verbs. Furthermore, when FastRoute needs to process the whole routes collection the same way as Symfony does, Symfony reach better performances here.

Further Investigations

Many questions remains... How can a library using a classic routing strategy obtain better performances than FastRoute? Does FastRoute's performance has something to do with its "combined regular expression" strategy?

To answer these questions I decided to write a classic routing strategy for FastRoute. The created classic strategy simply works through a route's patterns one by one until one matches the request path. You can see the benchmark results below.

Note: below FastRoute* will stands for FastRoute using a classic routing strategy.

So, to focus on the combined regular expression optimization, the results below only benchmark the routing step (i.e we are simply ignoring routes creation here).

Results obtained with the route set number 1:

Results obtained with the route set number 2:

Fewer calls to preg_match() on combined regular expressions clearly shows better performance.

Routing in PHP is a process which is executed once per request and the routes creation can't be ignored any more, so below you can find benchmark results, which also include the routes creation step.

Results obtained with the route set number 1:

Results obtained with the route set number 2:

So in this last benchmark results, the conclusion is pretty unambiguous: the time saved by using combined regular expression doesn't really compensate for the extra complexity added by all intermediate introduced structures to make the approach faster.

Conclusion

The fact that FastRoute's strategy doesn't bring any additional performance to a classic routing strategy has at least one positive point: route instances can be responsible for their own matching logic, which simplifies greatly the codebase.

Trying to increase routing library performance is fine, but it's not the only way to solve a routing performance issue. There are, for example, libraries like li3_resources, which minimize route creation by a simple design abstraction. Indeed, in li3_resources, HTTP methods management has been delegated up to the Resource instance, so instead of creating one route per supported HTTP method, only one route accepting all HTTP methods is created and then it's up to the Resource instance to respond or not to it. This kind of separation of responsibility can also be used with a classic controller/action strategy. A single '/{controller}/{action}[/{args}]' pattern coupled with a class_exists() and a method_exists() can supports an infinite number of requests by defining only one route.

So, there are a lot of possibilities to explore in routing design - it seems there is room for improvement in clarity, expressiveness and performance.

On Boundaries

Thu, 19 Nov 2015 00:00:00 -0500

On Boundaries

Over the past few years, I've been on an exploration of how software, and the process of creating & maintaining it, has become so complicated, difficult, and expensive. One theme I'm continually led back to is the concept of boundaries.

The whole monolith vs. microservice thing is just a red herring for the fact that we suck at managing boundaries in any formal/explicit way.
— Nate Abele (@nateabele) June 10, 2015

Boundaries are omnipresent in all aspects of the software industry, yet rarely acknowledged or thought about as a first-class concept.

Whether abstract or concrete, having a unified mental model for reasoning about boundaries should yield some interesting insights. Here are a few examples from across the spectrum:

Queries & cursors are boundaries between databases and applications
HTTP messages are boundaries between apps and browsers, or apps and other apps
Queues are boundaries between apps and expensive jobs
Time is a boundary between different states in an execution context, (object-oriented) application, a database, a filesystem, the universe, etc.
Apps are boundaries between me and data or functionality that I care about
Installation is a boundary between me and an app
Programming languages that I don't know (or that haven't been invented yet) are the boundaries between me and the ideal expression of some problem domain that I face
Wireframes are boundaries for communicating about user interfaces between designers, project stakeholders, and developers
HTML & CSS are boundaries between wireframes and real user interfaces
Code is a boundary between ideas and computers

As the tech community, these are a few of the boundaries that we face today. Some of them are useful. For the ones that aren't, our future is on the other side of them.

The next wave of innovation in software development is going to come about by treating time and boundaries as first-class things.
— Nate Abele (@nateabele) April 22, 2015

Comic book continuity and Git rebase

Mon, 19 Oct 2015 00:00:00 -0400

When helping companies improve their development practises, one common sticking point for the pull request workflow that we often recommend is the need to occasionally git rebase feature branches - it can be hard to understand, and a lot of people are initially pretty scared of it.

In this article, I’m not going to give any code examples - there are thousands of articles that do that. Instead, I am going to illustrate the concept of git rebase using the world of comic books.

Retconning in comic books

Comic books often run for years. Top titles like Batman or Spiderman have been going for decades and exist in an enormous shared universe, making continuity difficult to manage. Periodically, a character’s origin story will be retold, and it might be different, which can change the canonical history of the storyline. This is known as retconning (retroactive continuity) in the industry.

Just like comic books, Git allows you to “rewrite history” to suit your application as it is now - you are not bound to the past. This also allows you to work on different features in parallel - just like two writers both producing separate stories at the same time (which happens all the time with popular characters like Wolverine).

We will work through an example here and explain Git in terms of comic books!

Establishing canon

An artist, Alex, creates a character in a story that we will refer to as A1:

This comic book (although short!) is a huge success, selling thousands of copies. This is like our first few commits to a Git repository - they establish something useful that other people can develop on top of.

Another artist gets involved

At this point, Alex starts writing the next story in the Rock Savage adventures (we will call this A2). The publishing company smells money, and hires another writer, Bryce, to write another story (B1) in parallel.

At this point, we can think of the situation like feature branches in git - both Alex and Bryce are working in parallel from Alex’s original work:

They come up with a story each.

Alex's second story (A2):

Bryce's first story (B1):

Alex and Bryce both send their comics off to the publishing companies for approval. This is like raising a pull request via Github or Bitbucket.

There are no conflicts, and the publishing company decide to publish Bryce’s story first. This is like merging a pull request via Github or Bitbucket. Therefore the continuity looks like:

The publishing company then accepts and publishes story A2 (again, this is like merging a pull request):

So far, so good! There are no conflicts with these stories, the continuity is in harmony. If this was git, we have three commits (stories) in a row in our canonical continuity (or master branch).

Mixing it up

Alex and Bryce get started on new stories (like creating feature branches in Git) based on A2. We will call these stories A3 and B2:

Alex's third story (A3):

Bryce's second story (B2):

Now, these stories are in conflict. In Bryce’s continuity, Rock Savage is willing to kill. In Alex’s, he is not. Whilst Bryce is still tidying up story B2, the publishing company publishes A3:

Alex’s work is considered canonical - story A3 has been published. Bryce’s story B2 is now in conflict with the canon. The publishing company looks at Bryce’s draft and realises that it does not fit the continuity. Bryce needs to take into account the new things that Alex has revealed about the Rock Savage character. Therefore, Bryce fetches an updated copy of the continuity by getting A3. This is like doing git pull on the master branch, so that you have the canonical story locally.

Then, having got the updated continuity, Bryce effectively rebases off A3:

Instead of being based on the story A2, Bryce’s story (or feature branch, in Git parlance) is now based on the story A3. This is what rebasing is - changing what a branch is based on.

Bryce now has the extra facts to hand and can modify story B2 so that it fits the continuity. This is analogous to conflict resolution in git.

Bryce then resolves the conflict by making the panel make sense within the updated continuity:

Bryce's second story (B2), retconned (rebased) from A3:

At this point, Bryce then sends off the updated draft of story B2 to the publishing company. This is like updating a pull request by doing a push to your feature branch.

This time, the publishing company is happy with Bryce’s story, and publishes it. Again, this is analogous to merging a pull request.

Lookup table

Here is our handy print-out-and-keep guide to Git concepts for a pull request workflow in terms of comic books continuities!

Repository: Comic book series
Commit: Issue of the comic
Raise pull request: Send first draft to publisher
Master branch: Canon continuity
Feature branch: Draft being worked on by writer/artist
Rebase: Updating continuity
Conflict resolution: Changing a draft based on an updated continuity
Update pull request: Send a new draft to the publisher
Merge pull request: Publish an issue of the comic

How true is this?

Of course, both comic books and Git can have far more complex histories than we have hinted at here. Furthermore, the metaphor of comic book continuity is not perfect (see Spolsky’s law of leaky abstractions). Hopefully, though, by reading this post you feel confident that you understand what Git rebase is for, how it is used, and why it is such a crucial tool when you are working on different things in parallel.

End-to-End Functional Testing With Kahlan

Tue, 13 Oct 2015 00:00:00 -0400

End-to-End Functional Testing is the process of testing interactions between multiple modules of an application from a "start to finish" perspective.

Since End-to-End tests represent real-world scenarios, the DSL (Domain-Specific Language) used to describe such scenarios needs to be as expressive as possible. It is therefore important to use tooling that is flexible enough to allow the creation of semantically meaningful tests.

In this article, we are going to build a basic DSL (similar to Capybara) on top of Kahlan and Mink to perform End-to-End tests. Here is the final result:

namespace testing\spec\suite;

describe("Google", function() {

    it("displays the Google logo", function() {

        browser()->visit('http://www.google.fr');
        expect(element('#hplogo')->getAttribute('title'))->toBe('Google');

    });

    it("can search for and find Kahlan", function() {

        browser()->visit('http://www.google.fr');
        page()->fillField('q', 'Unit/BDD PHP Test Framework for Freedom, Truth, and Justice');
        page()->pressButton('btnG');
        wait(page())->toContain('Kahlan');

    });

}, 10);

If you are familiar with Jasmine-like testing frameworks, you should have been able to understand the code above. Let's dig deeper to see how things as been wired up under the hood.

Note: you can find the code used in this article at github.com/jails/testing.

Configuring `kahlan-config.php`

Setting up the WebDriver server

To interact with browsers, we are going to need a WebDriver server like Selenium (or similar). For this part, we shall use the WebDriver Manager library. This library allows you to keep Selenium server binaries up to date and also provides an API for easily starting a Selenium server. To use it inside Kahlan, we are going to configure kahlan-config.php like so:

use Peridot\WebDriverManager\Manager;
use box\Box;
use code\Code;
use code\TimeoutException;
use filter\Filter;

$box = box('spec', new Box());

$box->service('manager', function() {
    $manager = new Manager();
    $manager->update();
    return $manager->startInBackground();
});

Filter::register('run.webdriver', function($chain) {
    $process = box('spec')->get('manager');
    try {
        $fp = Code::spin(function() {
            return @fsockopen('localhost', 4444);
        }, 5, true);
        fclose($fp);
    } catch (TimeoutException $e) {
        echo "Unable to run the WebDriver binary, abording.\n";
        $process->close();
        exit(-1);
    }
    return $chain->next();
});


Filter::apply($this, 'run', 'run.webdriver');

Note: In the code above, box() is the rudimentary Dependency Injection Container included by default in Kahlan, but you can use your own implementation if you prefer.

The first step in the script above was to create the 'manager' service, which runs the WebDriver server and returns the created process when invoked.

The second part of the script consists of configuring the 'run.webdriver' closure to run just before the 'run' step (i.e. before the test specs run).

Code::spin() is a simple utility function which loops over a closure until a timeout is reached or until the executed closure returns a non-empty value (another approach to this could be using the sleep(x) pattern).

Note: In this example we are going to use the Selenium driver, but since Mink supports a lot of different drivers, pick the driver that suits you best.

Setting up the Mink instance

In order to use Firefox to runs our suite of specs, we need to also setup Mink accordingly using the MinkSelenium2Driver driver.

use Behat\Mink\Driver\Selenium2Driver;
use Behat\Mink\Mink;
use Behat\Mink\Session;

$box->service('mink', function() {
    $selenium = new Selenium2Driver('firefox', null, 'http://localhost:4444/wd/hub');
    $mink = new Mink(['firefox' => new Session($selenium)]);
    $mink->setDefaultSessionName('firefox');
    return $mink;
});

Filter::register('register.globals', function ($chain) {
    $root = $this->suite();
    $root->mink = $mink = box('spec')->get('mink');

    $root->afterEach(function() use ($mink) {
        $mink->resetSessions();
    });
    return $chain->next();
});

Filter::apply($this, 'run', 'register.globals');

In the script above, the 'register.globals' filter has been created to inject the mink variable to all scopes and make it available in all specs. Similarly, the closure passed to afterEach() will be executed after each specs in the suite completes its run. This will help to keep specs isolated, without having to explicitly tear down state in each spec.

Registering a custom matcher

In Kahlan it's possible to register a matcher to be used only for a specific class name. This feature can be used in our example to match with Mink's Behat\Mink\Element\Element instances. Note that all of these instances have a getText() method, which returns the textual representation of an element. So, to avoid having to deal with getText() everywhere, and in the interests of a consistent API, we can delegate these checks to a specific matcher:

use kahlan\Matcher;

Filter::register('register.matchers', function ($chain) {
    Matcher::register('toContain', 'testing\spec\matcher\ToContain', 'Behat\Mink\Element\Element');
    return $chain->next();
});

Filter::apply($this, 'run', 'register.matchers');

You can get the full matcher code here.

Post-suite cleanup

When the entire test suite has completed, the following script will clean things up properly:

Filter::register('cleanup', function ($chain) {
    $box = box('spec');
    $box->get('mink')->stopSessions();
    $box->get('manager')->close();
    return $chain->next();
});

Filter::apply($this, 'stop', 'cleanup');

Setting up the DSL

In order to write meaningful specs, we would also like to create a DSL similar to the Capybara one. To this end, we are going to write some helper functions to interact with the browser, the page and elements à la Capybara:

function browser($session = null)
{
    return Suite::current()->mink->getSession($session);
}

function page($session = null)
{
    return browser($session)->getPage();
}

function element($selector = 'body', $parent = null)
{
    $parent = $parent ?: page();
    $element = $parent->find('css', $selector);
    return $element ?: new ElementNotFound($selector);
}

These helper functions are not exhaustive, and you can of course add whatever functions you need to the DSL.

Dealing with asynchronous expectations

When interacting with a browser, elements appear or disappear from a page asynchronously. To perform asynchronous tests, you can use the kahlan waitsFor() statement, which runs a closure until some condition passes or until a timeout is reached.

We are going to use this waitsFor() statement to create the following additional helper function:

function wait($actual, $timeout = 0)
{
    return waitsFor(function() use ($actual) {
        return $actual;
    }, $timeout);
}

The last step is then to include the helper in the kahlan-config.php file:

require __DIR__ . "/spec/api/helpers.php";

And you are done! You should now be able to configure your own environment to write End-to-End tests which fit your requirements.

Wrapping up

End to End tests are a vital part of any testing strategy. With a little configuration, Kahlan allows you to write your End to End, Unit and Characterisation tests within a consistent testing environment.

Characterisation Tests for Legacy Projects

Tue, 22 Sep 2015 00:00:00 -0400

A legacy project is any project that you or your team did not create. It might have been left to you by another organisation, or it could have been brought in as part of a competitor buy-out.

Many legacy projects are still in use because they are robust, they do the job, and they are mature. There are, however, many common challenges that come with maintaining or developing on a legacy system. We all have to deal with these challenges at least some of the time.

Risks of Legacy Projects

The risks in legacy projects include:

Poor comprehension. When we take on a project, we take on its conceptual model and programming paradigm, which may be alien to us. Even if it follows patterns we are familiar with, there will be design decisions and technical debt of which we are unaware.

Stale dependencies. Many legacy projects use deprecated frameworks or languages.

Lack of tests. The most common problem with legacy systems is lack of tests.

Most legacy systems I have worked with have exhibited these risks. In the worst case, I had to deal with code that was over a decade old with no documentation, had no tests, was built in an archaic language, and did not even have a build script to compile it!

Thankfully, there are strategies for dealing with these kinds of projects. This article will focus on one of those strategies: using characterisation tests.

First, though, let us take a moment to consider the nature of change in software so as to have context for our objectives.

The Nature of Software Change

There are two sources of change to software: intended and unintended change.

Intended change is deliberate. It is intentional, such as a bug fix or new feature that did not break anything. Unintended change is mistakes, misunderstandings and side effects.

To avoid unintended, haphazard change, we need to manage change - we can’t make a set of alterations to an unknown product and expect it to work. As software professionals, we have a responsibility to make sure that all change to the system is intended change.

Characteristics of Change

Here we identify characteristics that change should and should not have:

Should be

Intentional - deliberate change that has the effect that it is supposed to.
Atomic - discrete alterations which affect one thing only. Has no side effects. See our blog post "perfect pull requests" for related information.

Should not be

Haphazard - poorly thought out, ad-hoc, and unstructured.
Side effect-y - change that does not affect only one part of the system, but has unintended side-effects elsewhere in the product.

By asserting that change matches our desired criteria, we begin to stabilise our understanding of our legacy project. One mechanism to help us to achieve this is characterisation tests.

Characterisation Tests

Michael Feathers defines characterisation tests as "tests that characterize the actual behavior of a piece of code". A characterisation test is likely to use similar or even the same tools as unit tests. They are similar to unit tests, except that they are only the what, not the why - they snapshot what a piece of code does, but do not necessarily represent any kind of understanding of the system under test.

They are used for detecting unintended change:

Producing Characterisation Tests

There are two ways to produce a characterisation test. Firstly, they can be produced manually using a unit test tool. I recommend choosing something that allows you to mock and stub easily, such as Kahlan.

Secondly, characterisation tests can be generated. BlackBoxRecorder in .Net and JackBoxRecorder for Java allow a sort of "record and playback" approach.

Example Characterisation Test

Here is a very simple snippet of code:

function foo() { return "bar"; }

To characterise this function, we can write a test such as the following (which uses the excellent PHP testing tool, Kahlan):

describe("foo()", function() {
  it("returns bar", function() {
    expect(foo())->toEqual("bar");
  });
});

Now, if foo() changes in any way, we will know about it (as all characterisation tests should run on commit, as per our article 4 Principles of DevOps).

Characterisation Test Workflow

When a commit is made, all tests are run. We then make a decision on whether that change was what was intended based on our "managing change" criteria as defined earlier.

We do this by checking the change against our criteria. Is the change intentional and atomic? Is it free of side effects? Our characterisation tests help us to establish this by failing if something is no longer doing what it used to be.

Let’s assume we have the following code:

function foo() { return "bar"; }

class ApplicationDescriber {
  public static function getName() { return foo() . " app"; }
}

With the following characterisation tests:

describe("foo()", function() {
  it("returns bar", function() {
    expect(foo())->toEqual("bar");
  });
});

describe("ApplicationDescriber", function() {
  describe("getName()", function() {
    it("returns the name of the application", function() {
      expect(ApplicationDescriber::getName())->toEqual("foo app");
    });
  });
});

Then, someone makes the following change:

function foo() { return "boop"; }

When the tests run, they will tell us:

Before this change to foo()
- foo() returned "bar", now it returns "boop"
- Class ApplicationDescriber method getName() used to return "bar app", now returns "boop app"

Not only did the change to function foo() trip its own test, it also tripped the characterisation test for class ApplicationDescriber! This has exposed a coupling in the system - changes to foo() affect function getName() in class ApplicationDescriber. At this point, we would step back and examine this change against the criteria we defined earlier:

Was it an intention of this change to affect the application name from ApplicationDescriber::getName()?
Should the system be refactored to decouple these components?

This is, of course, a very simple and contrived example, but it serves to illustrate the practical usage of characterisation tests.

Lifecycle

Here is the lifecycle of a characterisation test in 4 phases:

Phase 1: Panic

Let’s say we are assigned a feature to work on. We start off with no knowledge of the feature and minimal confidence.

Phase 2: Characterisation

We characterise this feature. Characterisation increases our confidence because, if a change happens, we will know about it, but it does not necessarily increase our understanding.

Phase 3: Understanding

Just because we have characterised something doesn’t mean we’ve understood it. We therefore refactor and write proper tests. Our understanding can now develop from a position of stability, helping to ensure that change is intentional.

Phase 4: Confidence

Once we are comfortable with our tests and documentation, we can throw away our characterisation tests. We don’t need them any more - there’s no need for this safety net because characterisation tests have no context. Semantically they don’t mean much; they are just a snapshot that we can now move past.

Why throw characterisation tests away?

They are only useful when you have low confidence and low test coverage. As confidence and test coverage grow, characterisation tests can actually become a drag on your velocity. Let’s illustrate this with an example.

You have two controllers that both use a DateParser. You then either write or generate characterisation tests for your project. You now have a characterisation test for each.

Let’s say you make a change to the date parser.

A change to the date parser might well invalidate all three tests, tripping three alarms! Of course, this depends on a lot of variables, such as how well factored the project is and how well written the tests are. Regardless, you can start to get false positives. When we start getting false positives in tests, people lose confidence. Therefore, these characterisation tests are now stale.

When is a characterisation test stale?

A characterisation test is stale when you have unit tests around the same region of code. Remember that characterisation tests are not based on an understanding of a system - they simply describe what it does right now. Once you have unit tests which reflect an actual understanding of the system, throw away the characterisation tests as they are now exerting drag rather than adding value.

Strategy for Using Characterisation Tests

1. Don’t change until you have tests around the code

Without tests, you risk making unintentional changes.

2. Delete when they become stale

False positives cause people to lose confidence in tests, so throw away characterisation tests after they are made redundant by unit/integration tests.

3. Separate out characterisation tests

Do not mix in characterisation tests with your unit tests; instead, put them in a separate directory. Characterisation tests are not based on an understanding of the system; they are more like a snapshot of the application’s functionality, so they should not be mingled in with more mature tests.

4. Make increasing test coverage a deliverable

There is always pressure to produce features, which can result in perceived pressure to skip testing. This is a totally false economy; if change is not managed, side effects can cause a legacy application to crumble in a matter of weeks. Instead, make increasing test coverage an achievable, trackable sprint deliverable - for example, "increase test coverage by 1% this sprint".

5. Be aware of the broader picture

Characterisation tests must not be all you use to stabilise your legacy projects. Rather, they are a part of a broader picture of stabilisation. Here are some complementary strategies, all with the goal of stabilisation:

Wrapping Up

Taking on a legacy application can be fraught with peril, and it is vital to ensure stability as quickly as possible. We, as testing professionals, have a responsibility to make sure that all change to the system is intended change. Characterisation tests are a great stepping stone towards assuring that change to a legacy application is intentional and atomic, rather than haphazard and unpredictable.

Announcing Radiian

Tue, 08 Sep 2015 00:00:00 -0400

At Radify, we greatly favor immutable infrastructure, which we described in two previous posts, Reducing Infrustration and Immutable Demo Nodes. We also outlined a simple method for setting up immutable infrastructure on AWS with Ansible. We have now gone a step further and created Radiian (RADify Immutable Infrastructure for ANsible), a tool which automates the process of creating a ready-to-go Ansible playbook to deploy immutable infrastructure on AWS.

Radiian on NPM

What is Radiian?

Radiian scaffolds an Ansible playbook for deploying immutable infrastructure on AWS. This playbook provides a template for deploying whatever Ansible roles you like in an immutable fashion. It will take care of all the immutable infrastructure side of things (standing up nodes, tearing them down, etc), leaving you free to focus on what you want installed on your nodes.

The generated playbook takes care of the following:

Tagging old EC2 nodes for termination
Standing up new EC2 nodes
Adding the new EC2 nodes to the Elastic Load Balancer (ELB)
Terminating tagged old EC2 nodes

Your job is then just to configure the Ansible roles that you need to provision your EC2 nodes.

Workflow

Radiian has a simple workflow:

What Radiian is not

As you can see from the diagram above, Radiian creates an Ansible playbook, but is not responsible for running and debugging that playbook. For example, from the diagram it is clear that Radiian is not responsible for creating and saving your private .pem key. Radiian is not an Ansible installer, nor does it set up AWS for you. If you're having trouble with Ansible, there is extensive documentation. If you're new to AWS, Amazon explains how to get started.

The only service that Radiian provides is to scaffold a template Ansible playbook for deploying immutable infrastructure on AWS.

What Sets Radiian Apart

Immutable Infrastructure makes Radiian different from many other Ansible tools. Yeoman's Ansible generator creates a playbook, but without immutable architecture. Radiian does not tie you to Ubuntu, or to DigitalOcean, or to Docker. All Radiian seeks to do is to layout a simple project template that relies upon immutable infrastructure.

Sample Use Cases

A new project: If you want to create a new project using immutable cloud infrastructure, Radiian is exactly what you need.
An existing project: You like the idea of immutable infrastructure, but aren't sure how to migrate your existing work to such an architecture. Radiian gives you a minimal playbook from which you can begin to migrate your project to immutable infrastructure, AWS, or both.
"Tinkering fatigue": "Who has changed what in our servers? Who has been tinkering around in EC2?" If you get tired of asking such questions (i.e., you are suffering from configuration drift), Radiian's approach to AWS architecture will keep you from having to ask those questions ever again. All changes to your AWS configuration are preserved in your Ansible playbook, which in turn is kept under version control history, so you can always know who changed something and when it was done. More importantly, every update to the playbook destroys the old nodes and creates new ones. At all times, your architecture is exactly what is described in the playbook.

Video Walkthrough

Moving forward

Radiian offers a minimal framework to get started with immutable infrastructure. Thanks to Radiian you can quickly and efficiently create a new project on, or switch an old project to, immutable infrastructure.

Radiian on NPM

In Search of the One True Gulpfile

Wed, 26 Aug 2015 00:00:00 -0400

Ever since switching from Grunt to Gulp, one of my first steps when starting a new JavaScript project has been to copy the Gulpfile from my previous JavaScript project, and modify it accordingly. Modifications are necessary firstly to suit the subtly different needs of each new project, and secondly of course, for taste.

While comparing Gulpfiles over time is an interesting little window into how my style and preferences have evolved, it pains me to have so much subtly duplicated functionality spread across so many projects, none of which are kept consistently up-to-date.

Aside from my own meandering laziness, I believe this is partly owing to Gulp's design, and partly what Gulp was created in reaction to. Whereas Grunt expected you to cram every varying combination of options into declarative configuration, which is easy to store and move around, but not very expressive, Gulp encourages composition through the definition of many small sub-tasks, which is highly expressive but tends to result in tightly-coupled processes & repeated steps, and managing configuration is left up to the developer. Both approaches leave something to be desired.

The middle path

Furthermore, while Gulp's chaining style makes for terse syntax, and its factory-function style makes it easy to configure individual process steps, it presents no clear way to configure the process itself. The ideal is a unified way to configure both the steps and the process. Also, a way to cleanly separate the process from the configuration, compose them in one place, and extract out any project-specific details, so that one Gulpfile can rule them all.

Fortunately, most of our projects have similar needs, and follow a relatively consistent directory structure:

build/: intermediate build files
dist/: production-ready distributable files
spec/: BDD spec files, suffixed as *Spec.js
src/: library/application source files

Most new JS projects are written in ES6 (ES2015), so our needs for each project generally look something like:

Compile source & tests to a particular module format (usually CommonJS or UMD)
Run tests using built files
Generate minified and non-minified production builds
Generate sourcemaps
Watch source & tests for changes during development, re-run tests
Run a dev server to serve files for in-browser testing (sometimes)

After reviewing a few of our most recent projects, I was able to come up with a Gulpfile that meets the above requirements, but also satisfies my desired constraints on process & configuration. Without further ado, weighing in at 55 lines (at the time of this writing), here is the listing in its entirety:

Important things to note:

As of Gulp 3.9, files named Gulpfile.babel.js are automatically transpiled from ES6, which for us means that Gulpfiles are no longer the one bit of a project where our brains must revert to old-school JavaScript
Almost all of the configuration is encapsulated in 2 blocks: one for file and directory paths, one for build settings
Project-specific configuration, such as the name of the build file and the port on which to run the dev server (where applicable) has been extracted out to package.json in a custom key called (naturally) build
All possible build steps are listed out in one place (pipeline(), lines 29-36); since the order of each step relative to others is always consistent, configuration can be used to toggle them on and off — which is made easy with gulp-if (aliased as gif() above)
A few small utility functions (build(), chain()) are used to map named build configurations (with optional overrides) to a flattened-out .pipe() chain, and tighten up function-handling syntax (bind())
Though it's not really necessary for low-complexity scripts like this, I've gotten into the habit of declaring things as const whenever possible (even though JS consts aren't really real constants)

Each build task is then assigned a function with a named configuration, which can be merged with passed parameters (in the form of command line arguments in the case of 'dist').

The end result is that almost everything is expressed exactly once. Not only that, it allows for any configurable option to be overidden from the command line, for free. For example, if I wanted a minified production file with sourcemaps and AMD-formatted modules, I could run gulp dist --min --smaps --modules=amd. This approach also has the benefit of programatically enforcing project structure conventions.

While the above is only appropriate for JavaScript library projects (it doesn't address CSS or other front-end tasks), the structure and patterns serve as a good basis for a generalized Gulpfile, and project-specific tasks can always be added transparently by introducing a requireDir-style pattern.

So that's my take. If you too are obsessive enough about build configuration to have a philosophy for managing it, I'd love to hear about it.

AWS Lambda Workflow

Wed, 29 Jul 2015 00:00:00 -0400

AWS Lambda is definitely the New Hotness - it allows you to run functions in the cloud with no persistent infrastructure!

Lambda can be described as "able to respond to predefined events by launching AWS resources and executing functions, without the user needing to manage the underlying infrastructure". Lambda is essentially user-created functions that run on demand in the cloud. This has the following benefits:

Only pay for what you use - no ongoing costs
Hugely scalable

In some projects, we were able to do away with EC2 nodes or containers entirely. For example, we recently built an application with no infrastructure at all - only a web site in S3, and functions executed in Lambda which talked to a Firebase database. This meant that we had absolutely no web nodes at all to run our code:

What about workflow?

This is a novel way of doing software; it is certainly quite different to much of what we have done before. We have a few principles that we like to apply to our development practises, and so, one of our first questions was, "how can we incorporate this into our workflow?". This includes considerations like:

Automation - How can we automatically deploy our application?
Speed of development - How can we run it locally so we don't have to keep uploading our code through Amazon's console every time we make a change?
Testing - How can we test our code?

Working with node-lambda

Whilst we can use the AWS console to manage our Lambda functions, we can get a smoother development workflow using node-lambda. This tool allows both pushing of Lambda function and, crucially, means we can run our Lambda functions locally.

To get started, run:

node-lambda setup

You now have two new files - .env and event.json. .env is a file that node-lambda uses to configure your function - what its name is, which role is used to access it, how much memory, and so on. event.json is the data that node-lambda will pass to your function when you invoke it with node-lambda run.

Local testing with node-lambda

Let's assume that your function is in a file index.js and looks like the following:

exports.handler = function(event, context) {
    console.log( "event", event );
    context.done( );
};

Also, let's assume that in event.json you have:

{
    "foo":"bar"
}

You can run it with:

node-lambda run

Because it posts whatever data it finds in event.json to your function, you should see:

event { foo: 'bar' }

To verify it's running truly offline, turn off wifi, unplug your ethernet, and then run node-lambda run again. This proves that it's running locally and not connecting to AWS.

Deployment with node-lambda

To set this up, configure .env so it has all the right values:

AWS_ENVIRONMENT=development
AWS_ACCESS_KEY_ID=YOUR_KEY
AWS_SECRET_ACCESS_KEY=YOUR_SECRET
AWS_ROLE_ARN=your_amazon_role
AWS_FUNCTION_NAME=your_function_name
AWS_MODE=event
AWS_MEMORY_SIZE=128
AWS_TIMEOUT=3
AWS_DESCRIPTION=
AWS_RUNTIME=nodejs

Most of these defaults will work without modification, but the ones you really need to pay attention to are:

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY: access key assigned by AWS. See our post Painless Immutable Infrastructure with Ansible and AWS.
AWS_ROLE_ARN: looks something like arn:aws:iam::1234567890123:role/lambda_basic_execution
AWS_FUNCTION_NAME: the name of your function. Must be unique. node-lambda will automatically suffix the version number from package.json.

Running the following will publish your function:

node-lambda deploy

Check out .env for the configuration of the function. Essentially, it creates or updates the function {AWS_FUNCTION_NAME}-{version}, where version is taken from your package.json. If your function name is foo and your package.json has version 1.2.3, function foo-1.2.3 will be set up in Lambda. Here's an example:

Troubleshooting `node-lambda deploy`

When telling .env which role to use, make sure you use a role ARN, not the function ARN. I got:

'validation error detected: Value \'arn:aws:lambda:REDACTED:REDACTED:function:REDACTED\' at \'role\' failed to satisfy constraint: Member must satisfy regular expression pattern: arn:aws:iam::\\d{12}:role/?[a-zA-Z_0-9+=,.@\\-_/]+'

I had to enable these Managed Policies for the role lambda_basic_execution:

AWSLambdaBasicExecutionRole
AWSLambdaRole

Recommended practise

Modularise your code and unit test it outside of Lambda - your lambda function should hand off to a library that you have built and tested separately using more conventional tools. This way, it's easy to unit test your code in a normal workflow.
Test locally with node-lambda run. This verifies that it all integrates well.
Use node-lambda deploy to update your function in the cloud.
Use the version number is package.json for versioning.
Be careful about invoking Lambda directly in browser. Yes, you can do it, but be careful not to expose your client key and secret key.

Wrapping up

These are practises that have worked well for us so far; hopefully this has given you a few ideas to help you to get started with working with AWS Lambda.

Other resources

Adam Neary: A Gulp Workflow for Amazon Lambda - you can use Gulp to create zips and upload them to Lambda.
gulp-awslambda - allows you to push Lambda to AWS as part of your Gulp workflow.
grunt-aws-lambda - allows both pushing of Lambda function and local development
Adam Neary: Developing and Testing Amazon Lambda Functions.
Amazon's AWS Lambda FAQs.
Why AWS Lambda is a masterstroke from Amazon - Janakiram MSV.

Understanding NGModelController By Example - part 2

Fri, 05 Jun 2015 00:00:00 -0400

In our previous post on NGModelController, we had a look at the way that it all hangs together and worked through a quick example of building our own custom directives with powerful data transformations. In this second post in the series, we will look at how we can apply validation to our directives.

View the code!

Parsers: bigger than they look

Parsers have two purposes:

to convert the $viewValue that is represented in the browser into the underlying value. We saw this in the our previous post on NGModelController.
to sanitise or validate the $viewValue - so we can apply validation in the view layer.

It is this second purpose we are going to look at here: adding validation that asserts that blue is not a valid colour selection for our directive. You can view the example code on JSFiddle.

Adding validation

Again, we are developing our example from part 1 a little bit further. Let's modify the HTML:

<div ng-app="RadifyExample" class="container" ng-controller="ColourPickerController">
  <form name="myform">
    <h1>Colours</h1>
    <label>
        <h2>Foreground</h2>
        <colour-picker ng-model="foreground"></colour-picker>
        <p ng-show='myform.$error.foreground_badColour' class="bg-danger">No blue please!</p>
    </label>
    <label>
        <h2>Background</h2>
        <colour-picker ng-model="background"></colour-picker>
        <p ng-show='myform.$error.background_badColour' class="bg-danger">No blue please!</p>
    </label>

    <div ng-style="{'background': background, 'color': foreground }" class="results">
        Results
    </div>
    <div>
        <button ng-disabled="myform.$invalid" ng-click="test()" class="btn btn-primary">Submit</button>
    </div>
  </form>
</div>

Notice that we've:

Created a <form>
Added <p> paragraphs that say "No blue please!"
Added a button that is disabled if the form is invalid

Now, let's add something to the controller:

.controller('ColourPickerController', function($scope) {
    $scope.background = 'F00';
    $scope.foreground = '000';

    angular.extend($scope, {
        test: function() {
            alert("Submitted!");
        }
    });
})

All this does is alerts "Submitted!", so we can easily see whether the button has been disabled or not.

Because parsers are a chain, we can have as many as we like, so we are going to insert a new one. We are not transforming the value in this parser (the first purpose of parsers); instead, all we are doing is the second purpose - validating the view value. Therefore, we are going to insert it BEFORE the current parser in our code:

// add validation
ngModelCtrl.$parsers.push(function(viewValue) {
  var blueSelected = (viewValue.red === "0"
      && viewValue.green === "0"
      && viewValue.blue === "F");

  ngModelCtrl.$setValidity(
      iAttrs.ngModel + '_badColour',
      !blueSelected
  );

  return viewValue;
});

ngModelCtrl.$parsers.push(function(viewValue) {
  return "#" + [viewValue.red, viewValue.green, viewValue.blue].join('');
});

Now, if you select blue for either of the colours, the button should be disabled and you should see a warning.

How does this work?

As we noted, we can have as many parsers as we like. We could do all the transformation and validation in a single parser function, but splitting it up into individual validation rules and transformations makes things a bit more modular and composable.

So, this gives us the validation - which sets myform.$invalid. Because the directives are bound to model values inside myform, if validation fails in either of them, myform is considered invalid. We've also added the paragraphs with the bg-danger class, which only show if there is a specific problem. It's always important to tell the user WHY you're not prepared to accept their data, otherwise you just look passive-aggressive and crazy! ;-)

Notice that we have two potential errors that can be set on the form: foreground_badColour and background_badColour:

ngModelCtrl.$setValidity(
  iAttrs.ngModel + '_badColour',
  !blueSelected
);

We use iAttrs.ngModel from the enclosing scope - which allows us to separate out the error messages so we can show the validation messages in the correct places.

What else can I do with this?

We've seen that parsers have two purposes: validation and transformation. We can apply multiple validations - as many as we like - to verify various things about our data. This is an incredibly powerful for providing rich, sophisticated controls for our data!

Painless Immutable Infrastructure with Ansible and AWS

Wed, 27 May 2015 00:00:00 -0400

In our blog posts Reducing Infrustration and Immutable Demo Nodes, we talk about our approach to immutable infrastructure and the benefits we have seen from employing this approach. In this article, we explore a practical project example so you can get up and running with immutable infrastructure in less than an hour!

Overview

Ansible is incredibly powerful and features a whole lot of tools for manipulating AWS (Amazon Web Services) infrastructure.

We're going to start with the simplest project that we can; let's assume you have an application that you want to deploy onto immutable EC2 nodes. Your app could be anything - PHP, Python, Ruby, NodeJS - it doesn't matter!

We are going to work through an example, step-by-step, to show you how it could be done in a simple way in Ansible.

Order of events

Here's what we're aiming for. It's a simple method for immutable infrastructure which is based on adding nodes to the load balancer, then destroying all the old nodes.

Tag any running instances of your project (if there are any) with oldMyApp=True
Provision new instance(s)
Add new instance(s) to the load balancer
Destroy the nodes you tagged with "oldMyApp=True"

This has the following advantages:

If it fails at any point, the load balancer keeps running with its existing nodes - no downtime!
...and any nodes that's didn't provision correctly will automatically be removed on the next run
It's super simple!

Step 1: Setting up AWS

Before we get started, we are going to do a few setup tasks to get our infrastructure ready.

Create a security group, e.g. sg_myapp - see Amazon's docs
Create a load balancer, e.g. lb-myapp - see Amazon's docs
A domain in route53 - see Amazon's docs

Step 2: Create a directory for your Ansible playbook

Pretty much any structure will work. Let's go with:

/ansible - our playbook
/yourapp - your application

Step 3: Dynamic inventory

We use dynamic inventory so that Ansible can work out what nodes we have in AWS. So, in directory /ansible/inventory, put the ec2.ini and ec2.py files from the dynamic inventory.

Create /ansible/ansible.cfg for configuring Ansible to use the dynamic inventory. We suggest:

[defaults]
ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}
hostfile = inventory/ec2.py
remote_user = ubuntu
[ssh_connection]
pipelining = True
ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o StrictHostKeyChecking=no
control_path = /tmp/ansible-ssh-%%h-%%p-%%r

We also recommend having /ansible/inventory/vagrant and a /Vagrantfile so that you can run this playbook to start a virtual machine locally that is representative of production. A full treatment of this topic is out of scope of this article, but take a look at our article, 4 Principles of DevOps, for an investigation

Step 4: Amazon access key and secret key

Also create /inventory/aws_keys. This is a file that will simply include your security information so that Ansible can communicate with Amazon Web Services.

This file should contain information like the following. Your specific values will vary. Amazon's documentation tells you how to get these values.

export AWS_ACCESS_KEY_ID='HJFLKJ32T32TGEKJG3'
export AWS_SECRET_ACCESS_KEY='HJFLKJ32T32TGEKJG3HJFLKJ32T32TGEKJG3'
export EC2_REGION='us-east-1'

Also, export the .pem from AWS and put it in /ansible/myapp.pem - follow Amazon's documentation on EC2 keypairs. This key will be used by Ansible to set your SSH credentials so that it can talk to AWS.

Step 5: any specific roles your playbook may need

We will have a directory /ansible/roles. These are the roles that install your application. This is out of scope of this article as it will depend on your application.

Step 6: create playbook to tag the old nodes

Create /ansible/tag-old-nodes.yaml

---
- hosts: all
  gather_facts: false
  sudo: no
  tasks:
  - name: Get instance ec2 facts
    action: ec2_facts
    register: ec2_facts
  - name: Add a tag "oldMyApp" to all existing nodes, so we can filter them out later
    local_action: ec2_tag resource={{ec2_facts.ansible_facts.ansible_ec2_instance_id}} region='us-east-1' state=present
    args:
      tags:
        oldMyApp: true

All this does is, using the dynamic inventory, goes through the existing nodes and adds the tag "oldMyApp" (set this tag to something more relevant to your app).

Step 7: add playbook to stand up nodes

Create /ansible/immutable.yaml. Read our comments carefully, please don't simply copy/paste this!

---
- hosts: localhost
  connection: local
  gather_facts: false
  vars:
    instance_type: 't2.micro'
    region: 'us-east-1'
    aws_zone: 'b'
  tasks:
  - name: Launch instance (Ubuntu 14.04 hvm)
    ec2: image='ami-9eaa1cf6'
         instance_type="{{ instance_type }}"
         keypair='myapp'
         instance_tags='{"Environment":"myapp","Class":"myapp-immutable","Name":"myapp (immutable)"}'
         region='{{region}}'
         aws_zone='{{ region }}{{ aws_zone }}'
         group='sg_myapp'
         wait=true
    register: ec2_info

  - add_host: hostname={{ item.public_ip }} groupname="myapp,ec2hosts"
    with_items: ec2_info.instances

  - name: Wait for instances to listen on port 22
    wait_for:
      state=started
      host={{ item.public_dns_name }}
      port=22
    with_items: ec2_info.instances

# Run your specific roles that install and configure your application
- hosts: ec2hosts
  gather_facts: true
  user: ubuntu
  sudo: yes
  roles:
    - myapp

- hosts: ec2hosts
  tasks:
  - name: Gather ec2 facts
    ec2_facts:
  - debug: var=ansible_ec2_instance_id # You can remove this if you like
  - name: Add newly created instance to elb
    local_action:
        module: ec2_elb
        region: 'us-east-1'
        instance_id: "{{ ansible_ec2_instance_id }}"
        ec2_elbs: "lb-myapp"
        state: present

Step 8: add playbook to destroy old nodes

Create /ansible/destroy-old-nodes.yaml

---
- name: terminate old instances
  hosts: all
  tasks:
    - action: ec2_facts
    - name: terminating old instances
      local_action:
        module: ec2
        state: 'absent'
        region: 'us-east-1'
        keypair: 'myapp'
        instance_ids: "{{ ansible_ec2_instance_id }}"

All this does is remove any instance that has the tag "oldMyApp". This tag is actually passed in from the kick off script.

Kick off script

Create /ansible/provision.sh:

#!/bin/bash

set -u # Variables must be explicit
set -e # If any command fails, fail the whole thing
set -o pipefail

# Make sure SSH knows to use the correct pem
ssh-add myapp.pem
ssh-add -l
# Load the AWS keys
. ./inventory/aws_keys

# Tag any existing myapp instances as being old
ansible-playbook tag-old-nodes.yaml --limit tag_Environment_myapp

# Start a new instance
ansible-playbook immutable.yaml -vv

# Now terminate any instances with tag "old"
ansible-playbook destroy-old-nodes.yaml --limit tag_oldmyapp_True

Make sure provision.sh is executable (e.g. chmod u+x provision.sh). Now, all you have to do, to totally refresh your infrastructure with brand new provisioned nodes, is run:

./provision.sh

Is this a good approach?

This is a solid starting point that we use for lots of small projects. It works just great for us for these projects. With more complex projects, we use more sophisticated approaches.

Larger projects may wish for something more full-featured, for example, creating a new load balancer, running tests, before switching over Route 53.

Wrapping up

This is just one quick-and-easy way of getting started with immutable infrastructure. We hope that it's given you some confidence to start deploying immutable infrastructure of your own!

Immutable Demo Nodes - how Radify adopted immutable infrastructure and what the benefits have been.
Four Principles of DevOps - the principles we apply to our infrastructure and operations.
Reducing Infrustration - we adopted an immutable infrastructure approach for our web and API nodes. We talk about what it is, why we did it, and what the costs and benefits have been.

Enforcing Your Pull Request Workflow

Wed, 20 May 2015 08:00:00 -0400

Whether commercial or Open Source, most software projects follow some sort of guidelines for ensuring smooth processes and quality outcomes. On a small team, it's easy to enforce these rules informally, but to scale up, tools like automated builds and coding-standards checkers are indispensable.

PR.js on Github

Nowhere is this more acute than Open Source projects. The dueling objectives of keeping barriers to entry low while ensuring the quality of contributions means tools that provide automatic feedback and guidance always win over lengthy contributor guides and human review.

At Radify, we run & contribute to a number of different Open Source projects and, while we have many tools for validating the quality the code itself, we've found ourselves having to repeatedly coach would-be contributors on how that code should be submitted. Things like:

which branch a given change should be submitted to
that commits for a single, logical change should be squashed
details a pull request should include

And everybody's favorite:

how to format commit messages

As feedback cycles get longer and further away from a contributor, the more likely they are to become discouraged and give up entirely, a loss of unknown potential: could that person have become your next top bug crusher, doc writer, or issue triager?

We looked around for ways to automate this, but found that the list of existing solutions was... anemic... at best. So, we went ahead and made one.

Enter PR.js. It validates your project's PRs against a(n optionally configurable) set of rules. It requires no signup, no setup, and no changes to your project. It has built-in rules for most of the above, but you can easily roll your own rules for anything you like. Internally, we'll be able to use it for things like validating embedded metadata in commit messages, ensuring user-facing features have a screenshot and test script, and that branches are correctly identified for demo deployment.

We now have immediate, linkable, browseable feedback for all of our Open Source projects, and we're working on further integration with GitHub to manage private projects, as well as provide feedback directly in PRs via the Build Status API.

In the wild

What kinds of PR workflow rules do you use on your OS project or dev team? Is this something you'd use? Let us know, and if so, feel free to fork and contribute your own rules.

Understanding NgModelController By Example - part 1

Wed, 06 May 2015 00:00:00 -0400

AngularJS’s ngModel.NgModelController is extremely powerful, but it can seem a bit daunting, what with its $formatters and $render function and $parsers... How do we get our heads around it?

View the code!

This article assumes you’ve got a little bit of experience with AngularJS. You don’t have to be a brain scientist or a rocket surgeon or anything, so long as you’ve used it a little!

I developed this article from the notes I wrote as I was learning NgModelController - I found the learning curve a little steep so hopefully this can help somebody else hit the ground running!

What is ng-model?

ng-model is what Angular uses to two-way data bind values to controls, e.g.

<label>Username
<input ng-model="use.name">
</label>

What if you want to display a custom control for a complex value, though? What if you wanted multiple fields to represent a single value? That’s where NgModelController comes in!

What is NgModelController?

NgModelController is the same thing that Angular uses to bind ng-model to input boxes and menus and so forth and it has a lot of functionality that you can hook into and take advantage of.

What is NgModelController for?

NgModelController has the following advantages:

Allows you to set up validation on the ng-model
Ability to transform the underlying model to and from a view value that can be used to have a sophisticated interface
Supports transformation of the underlying value into a format that can be displayed differently, and transformation back to the underlying model
Allows you to monitor when the value has changed and whether it’s clean or dirty
Neat encapsulation - make the directive self contained

You would use NgModelController when:

You’ve got something you want to bind to ng-model and you need a sophisticated interface to it.
You want to render a value differently in the UI to how it is represented in the underlying model. For example, a calendar control.
You have specific validation you want to apply. For example, validating that a dateis within a certain range.

The values and life cycle

To allow the conversion of a piece of data into a format it can be manipulated in, and back again, NgModelController works by adding a $modelValue and a $viewValue as well as scope values local to the directive. This means that you have:

The actual, real value.
NgModelController.$modelValue - the internal value that gets synchronised with the ACTUAL value that’s linked by ng-model.
NgModelController.$viewValue - the value that $modelValue gets translated to.
scope values - used to bind things in your directive to. Usually a copy of NgModelController.$viewValue.

Here are where the values exist:

NGModelController handles synchronising them through the following process:

Yikes! This looks a bit complicated but we are going to go through a worked example bit-by-bit to show how it all works. At some point it will “click” and you will harness the power for yourself!

Example - a colour picker

I’ve made an example colour picker - it’s not amazing but it should illustrate the principles! It allows you to select None, Some or Lots of Red, Green and Blue channels respectively. Under the hood, that’s a single value (e.g. “#FA0”), but in the interface it is three select boxes:

You can play with the code yourself here: http://jsfiddle.net/cox0sba1/

We have two instances of our colour-picker custom directive and we use ngModelController to convert between the three channel values that the user sees (R, G, B) and the single underlying string that represents the value that ng-model is bound to.

The app namespace

We define an Angular module called “RadifyExample”:

angular.module('RadifyExample', [])

See the code

The controller

All the controller does is set up background and foreground colour values:

.controller('ColourPickerController', function($scope) {
    $scope.background = 'F00';
    $scope.foreground = '000';
})

See the code

The HTML bit

<div ng-app="RadifyExample" ng-controller="ColourPickerController">
    <h1>Colours</h1>
    <label>
        Foreground
        <colour-picker ng-model="foreground"></colour-picker>
    </label>
    <label>
        Background
        <colour-picker ng-model="background"></colour-picker>
    </label>

    <div style="background: #{{ background }}; color: #{{foreground}};" class="results">
        Results
    </div>
</div>

See the code

The two values - foreground and background - from the controller are bound by ng-model to a custom directive colour-picker. This gives us two colour pickers.

We then have a little area “Results” that simply renders the background and foreground in a div with the word “Results” so we can see what colours the user has selected.

The directive

First up is a simple template that shows three <select> controls.

.directive('colourPicker', function() {
  var tpl = "<div> \
     R <select ng-model='red'> \
         <option value='F'>Lots</option> \
         <option value='A'>Some</option> \
         <option value='0'>None</option> \
     </select> \
     G <select ng-model='green'> \
         <option value='F'>Lots</option> \
         <option value='A'>Some</option> \
         <option value='0'>None</option> \
     </select> \
     B <select ng-model='blue'> \
         <option value='F'>Lots</option> \
         <option value='A'>Some</option> \
         <option value='0'>None</option> \
     </select> \
 </div>";

See the code

We then have some initialisation - we restrict to ‘E’ for element, assign the template and create an isolate scope. The next bit is where we require 'ngModel', which gives us all the NgModelController functionality.

  return {
      restrict: 'E',
      template: tpl,
      scope: {},
      require: 'ngModel',

See the code

Now we have the real meat of the code - the link function with its injected NgModelController instance ngModelCtrl:

      link: function(scope, iElement, iAttrs, ngModelCtrl) {

See the code

Inside the link function, we do a bunch of things. Firstly, we set up formatters (which convert the model value to view values):

Set up formatters

          ngModelCtrl.$formatters.push(function(modelValue) {
              var colours = modelValue.split('');
              return {
                  red: colours[0],
                  green: colours[1],
                  blue: colours[2]
              };
          });

See the code

Formatters can be a whole chain of functions. We are using just one. All we do is take the $modelValue - which is the underlying “actual” value within NGModelController and is a 3 character string - and split it so we have an array of 3 elements [red, green, blue]. We return these as an object, and they get set on the view value.

Set up render function

          ngModelCtrl.$render = function() {
              scope.red   = ngModelCtrl.$viewValue.red;
              scope.green = ngModelCtrl.$viewValue.green;
              scope.blue  = ngModelCtrl.$viewValue.blue;
          };

See the code

The $render function takes the values that are on the $viewValue of ngModelCtrl and puts them onto the local scope. In other words, it takes the $viewValue and renders it to the screen. This means that in the directive’s UI we have access to red, green and blue.

All we are doing here is taking the view value and putting it on the scope variables for access in the HTML template.

Set up a watch

          scope.$watch('red + green + blue', function() {
              ngModelCtrl.$setViewValue({
                  red: scope.red,
                  green: scope.green,
                  blue: scope.blue
              });
          });

See the code

We are telling the scope to pay attention to the three variables that we have put onto it - red, green and blue. When one of them changes, Angular, in its next digest cycle, will set the view value on the ng model controller, keeping the interface up to date and everything in sync.

Set up parsers (convert view value to model value)


          ngModelCtrl.$parsers.push(function(viewValue) {
              return '#' + [viewValue.red, viewValue.green, viewValue.blue].join('');
          });

See the code

Just like formatters, you can have multiple parsers, and again we’re just using one. All it does is join together the $viewValue’s [red, green, blue] array back into a 3 character string.

Step by step - what is happening in the colour picker?

Let’s take a look at what is actually going on when we use our colour-picker directive.

When you change one of the `<select>` field inputs…

So, when you change any of the <select> field values, that changes the scope value. Standard angular! This then kicks off a few things:

The scope value has been changed
The watch function picks up that something has changed on the scope value
Watch calls $setViewValue, which sets the $viewValue
Change to $viewValue sets kicks off the parser chain $parsers
Parser chain updates the underlying $modelValue
Some kind of Angular magic happens!
The real model is updated. Everything in synchronised!

If something changes the underlying value...

So what happens if something outside of the directive changes the real value? A chain of events is kicked off which synchronises everything:

The real model is changed outside of NGModelController
Angular says "hey NGModelController! The model has updated!"
NGModelController updates $modelValue
The $formatters chain is be triggered, and converts the $modelValue to the $viewValue
The $viewValue has been updated by $formatters
Because the $viewValue has been updated, the $render function gets kicked off
The $render function updates the scope values
The watch function picks up that something has changed on the scope value
Watch calls $setViewValue, which sets the $viewValue
Change to $viewValue sets kicks off the parser chain $parsers
Parser chain updates the underlying $modelValue
Some kind of Angular magic happens!
The real model is updated. Everything in synchronised!

You can observe this by adding <input ng-model="background”> and changing the value in there and you can see that formatters is kicked off.

Wrapping up

We hope that this article has helped you to get your head around ngModelController! Please let us know if there are any inaccuracies in this article, or how we could make it better!

Read part 2!

Testing ES6 with Karma, RequireJS, and Angular

Wed, 22 Apr 2015 00:00:00 -0400

Well, that’s a lot of tools to list in a blog post title! We have developed a simple plugin for Karma that adds shims for both ES6 and ES5. This allows you to use cutting edge language features, AND be able to unit test it, right now, even if you’re using AMD!

karma-es6-shim on NPM

You can view it at npmjs.com/package/karma-es6-shim. We have also published a sample project that shows how it can be used github.com/radify/karma-es6-shim-example.

So why is this necessary?

I was recently assigned a small project that was written in ES6 and AngularJS that used AMD modules for module source code. It was a tiny proof of concept, which used Babel (AKA 6to5) to transpile the ES6 source code in the src directory into outputted ES5 compatible JavaScript in the build directory.

Even though Babel was used, the code didn’t work in native Chrome without ES6 features switched on. I’d get problems like Object.assign not being a function. Object.assign is not supplied by default with Babel/6to5 (see this link); It’s part of ES6 that is not fully ratified yet - see Mozilla's Object.assign() page, which explains why it wasn’t transpiling in, Therefore, in the HTML, an ES6 shim was supplied:

<script src="https://cdnjs.cloudflare.com/ajax/libs/es6-shim/0.22.1/es6-shim.min.js"></script>

Without this shim, errors would occur; with the shim included, the prototype worked smoothly.

The first thing I wanted to do was to characterise it with a few unit tests before I started developing it further, so I could be confident I wasn’t breaking anything. I installed Karma with its PhantomJS plugin, but I started getting errors like:

PhantomJS 1.9.8 (Mac OS X) ERROR TypeError: 'undefined' is not a function (evaluating 'Function.call.bind(String.prototype.indexOf)')

After quite a lot of googling, I realised that PhantomJS does not support the entire feature set of ES5 (let alone ES6!). Therefore, I included the Karma ES5 shim:

npm install --save karma-es5-shim

Then in my karma.conf.js I added it:

    frameworks: ['jasmine', 'requirejs', 'es5-shim'],

Now I was getting the error message:

    TypeError: 'undefined' is not a function (evaluating 'Object.assign(this,this._default)')

So, I knew that this meant that I needed the same es6-shim - it was the same error as I got in browser if I removed the es6-shim.

So, what I did is created my own version of karma-es5-shim that also included es6-shim, naming it karma-es6-shim. I installed it:

npm install karma-es6-shim

Then in my karma.conf.js I updated to the new shim:

    frameworks: ['jasmine', 'requirejs', 'es6-shim'],

At this point, my tests started passing in Karma/PhantomJS, AND everything in browser is still working. Hooray!

Alternatives to karma-es6-shim

So, is this the right way to add support for incomplete ES6 features? It works for us with our particular combination of tools, but there are other options…

Babel has a polyfill, which provides most of the non-finalised features of ES6, including Object.assign. Feeding Babel the optional: [“runtime”] engages these features. For example:

var pipe = gulp.src(src).pipe(babel({ optional: ["runtime"] }));

It seems, though, that it doesn’t work with RequireJS. From the docs:

The package babel-runtime is required for this transformer. Run npm install babel-runtime --save to add it to your current node/webpack/browserify project. babel-runtime does not support AMD module loaders like RequireJS.

Possibly with some massaging, it could be made to work but I didn’t have any success. Conclusions ES6 is very much a moving target; as we have seen, even features like Object.assign are not yet set in stone. Our karma-es6-shim works for us, for now, to allow us to use same shim in the browser as we use in our unit tests. It’s not as neat as it would be to load the polyfill within Babel, but it’s “good enough” for now!

Roll on widespread ES6 support!

Navigating the Minefield of Metrics

Fri, 10 Apr 2015 00:00:00 -0400

Values first, numbers second! We need to steer our projects, and to do that we need data.

Here is an excerpt from Gavin's book which talks about measurement:

Measure things, don't just use guesswork and "intuition". If you say to a project manager "I spent 4 hours improving performance" and you don't give any figures, then you might as well have been playing CounterStrike! … Get into the habit of providing figures whenever possible.

This helps us to make rational, informed decisions about our projects. It can, however, be extremely difficult to pick what to measure and how much stock to place in those measurements. In this article, we look at things to bear in mind when defining metrics for software projects.

We are looking at project-level metrics here. We will consider organisation-level metrics in a future post.

Bad Metrics - The Danger Zone!

Ever been in this situation?

If you've ever heard something like this, where people are trying to game metrics, then you might be in the danger zone!

(source: quickmeme.com)

Uh oh! Looks like a case of bad metrics!

In this example, the budget for the project is tied to the time spent, but it is not being honestly reported. There are many reasons why this may be, but management cannot make informed decisions if the metrics are poorly chosen or inaccurately reported... How has this happened and what can we do to prevent this kind of misuse and gaming of metrics?

Avoiding potential metrics pitfalls

Let's take a look at three potential pitfalls of poorly applied metrics and what we can do to avoid these dangers.

Danger 1: misunderstanding what a metric is

Metrics may been seen to be "the thing itself" rather than a measure of something. A thermometer in water does not show the temperature of all the water, rather it shows the temperature at a single point.

An example misunderstanding is common in agile development. "Oh no, only 20 points got delivered this week, we normally do 40!"... This does not necessarily mean catastrophe; the estimates could have been off, or someone could have been sick. Points are very useful, but are not "real things", they're just used for sizing and planning.

Preventing misunderstandings - understand that the measurement is not the thing!

Metrics are helpful only so long as it is entirely grasped that a metric is, at best, a view of an aspect of a thing and is NEVER the thing in and of itself. For example, yes, it's worth paying attention to story points (we pay VERY close attention to them!), but they are not bricks, they are not cement, they are very abstract and we see them for what they are.

Measurements, metrics and KPIs

Let’s have a look at how we can sensibly approach metrics and define our terms.

I want to draw a distinction between 'metric' and 'measurement'. A measurement is a value over time, and a metric is a measurement with an attached set of decision-making constraints. That is, we can measure stuff, but not all of what we are measuring should be used to guide the project. So, in this pyramid, not all measurements become metrics.

To help us work out what measurements to use as metrics, one approach is to can measure and track absolutely everything we can, going back as far as possible, then explore the data. See what looks interesting, find unexpected correlations, then try to derive metrics from that.

The most critical metrics, we can use as KPIs (Key Performance Indicators). These are the things that tell us whether our project is succeeding or failing. We have to pick these incredibly carefully, as we will see later, but when wisely chosen, they can give us a lot of insight into what is going on.

Approaching it this way means that we don’t get overwhelmed, and we keep in sight what our project-guiding metrics are without getting bogged down in the ones we just record in case they become useful later.

Danger 2: poorly chosen metrics

As we saw in the pyramid, not all measurements may actually be useful. It is important to continuously evaluate - is this metric telling us anything useful? The caution here is not to assume that just because something is recorded, it has value. With that said, something might prove to have value later, so it’s worth recording, just don’t sell the farm based on what a single measurement is telling you.

Choosing good metrics

The metrics should not be the focus of the project; rather they are a way of measuring what is going on and having telemetry into the project. Far more important are the values that drive us - what kind of project are we going to be? What matters to us? What matters to this project?

For example, on one recent project, Radify worked with the client to define three values that were most important to them, We then implemented metrics against these values. Here are those values:

Predictability
Continuous feedback
Quality

As stated before, we had been measuring everything we could against the application from the outset, and it was therefore quite easy to pick out representative measurements and turn them into metrics. We're not going to go into huge detail here but here are some of the things we picked:

Metrics for Predictability

Track progress against estimates and report back on every sprint
Expected vs actual velocity
Scope creep (changes to sprint after start)

Metrics for Continuous feedback

Number and timing of deploys
ApDex, error rate, uptime, CPU, memory and disk utilisation
Application-specific metrics (e.g. number of orders placed today, number of abandoned orders)
Continuous integration build results

Metrics for Quality

Number of bugs in production
Code coverage

We then create project dashboards using DataDog so we can see at-a-glance what our metrics are telling us. We iterate on these metrics in every sprint retrospective and are always fine-tuning what goes on the dashboard. Hopefully, you can see that we've picked metrics based on what is actually important to the project based on what is important to the client.

Danger 3: badly applied metrics resulting in toxic culture

An organisation or project that focuses on metrics with too narrow a gaze runs the risk of poisoning its own culture.

Where metrics are used individually and narrowly or as a stick to beat people with, this results in people “gaming” the metrics to appear more productive. Humans being humans, we tend to reverse engineer management’s perceived desired outcomes.

Worse still, toxic metrics can cause serious disenchantment. Toxic KPIs can be particularly harmful as they and can encourage people to focus narrowly on their KPIs to the exclusion of overall cohesion and cooperation (see this article, for example) . This is why linking metrics to individual reparation and performance reviews is dangerous - many people will pull up the drawbridge and focus on only what is tied to their pay reviews, not what is best for the project as a whole. This might mean that they spend less time helping colleagues or thinking outside the box.

One of Radify’s main driving principles is "no silos", because we want everyone to understand the project as a whole so we really want to avoid toxic metrics.

Preventing toxic culture - think team!

Metrics must NEVER be used as a stick to beat people with.

Singling people out creates silos as each person looks to their own interests rather than thinking about the project or organisation holistically. Metrics should never be individualized; the project team as a whole improves together or falls behind together. This means that people will feel free to report non-technical metrics honestly and openly because they don’t feel like they will be persecuted for the results!

Characteristics of good metrics

We have seen that bad metrics are poorly understood, badly chosen and toxically applied. Conversely, good metrics have three characteristics:

Well understood: the team understands what the metric means and what it is and isn't telling them. The team understands the relative importance of KPIs and other metrics.
Well chosen: the metric reflects the values of the project.
Non-toxic: the metric is applied positively to guide the project, not as a stick to beat people with.

Benefits of good metrics

The whole company and everything that happens within it can be decomposed into a set of systems. Complex systems are easy to create and hard to understand. The more things we measure and the longer the timeline, the better we understand the system. Eventually, our understanding of the system(s) will be almost intuitive, as the impact of changes will be immediately observable (for varying values of 'immediate', depending on the system). This will allow us to steer our project with greater confidence.

Well-designed KPIs provide quick insight into trends and summary information, while supporting drill-down into more detailed metrics. This allows an organization to see where it's doing well and where it requires improvements and/or course adjustments.
- Jonathan D. Becher

When to use your metrics

There are two main points at which we use our metrics in a project:

Reactively: if something goes bad, especially in production, we get notified right away! See Four Principles Of DevOps and The Bot Who Cried Wolf for details.
Proactively: we look at our dashboard in every sprint retrospective, and take time to interpret what it is telling us. This allows us to steer what we do. For example, if performance is bad, we may allocate some time in the next sprint to this as a non-functional requirement.

Wrapping up

We hope you’ve found our ideas on metrics thought-provoking. When properly chosen, metrics can be extremely valuable in guiding your project. When poorly chosen or applied, they can be toxic to morale and can give false confidence or misleading information.

We’ve got a lot more to say on the subject, so if you want to talk, please get in touch! If your organisation needs help getting measurable results, we can help you!

What do you measure? Is it helpful? Have you ever been a victim of toxic metric culture? Were you able to fix it? Let us know in the comments box below!

Refactoring Repositories

Fri, 27 Mar 2015 00:00:00 -0400

Source code repositories using tools like Git, Subversion or Mercurial are essential to any modern software project and it is important that they are efficiently organised. We recently merged multiple repositories into a single codebase and found it gave us significant efficiency gains. In this article, we take a step back and look at how we should architect our repositories - when we should merge, and when we should draw apart.

How we got to this

On a recent project, we were working with four source code repositories:

UI
API
Ansible playbook
Release tool

Having them being separate was causing us three points of pain:

Pain point 1: branch management

We had to keep branches in check across repositories. So, if I created a branch for FEATURE-A, and it had UI and API changes, I had to have a branch in each repo named exactly the same.

Pain point 2: difficulty of review

Multiple pull requests had to be raised for any user stories which touched more than one part of the system. This led to us having to list dependencies for any given pull request. Therefore, if one pull request for a feature was merged but its dependency was not, the system would be in an inconsistent state.

Pain point 3: difficulty of automation

We had to do some gymnastics with our automated demo branch deployment. For example:

UI change detected in branch FEATURE-A
Is there a corresponding API branch FEATURE-A?
    YES
        Deploy UI FEATURE-A and API FEATURE-A
    NO
        Deploy UI FEATURE-A and API MASTER

It meant, for example, that any changes to our Ansible playbook, which handles all our deployment/provisioning and infrastructure, were not tied to the corresponding API or UI code, so if a given feature branch required a different package to be installed via Apt, for example, we had to switch the Ansible branch on Jenkins. Not an elegant solution!

This also led to potential race conditions, whereby multiple builds could kick off together due to pushes to two repos just four seconds apart:

This was a problem because e2e was a single environment and we could not test it in parallel, so this meant we had to create a fairly complex build pipeline with dependencies, or to have some kind of quiet period:

Furthermore, our release tool was pretty baroque - it had to check out all the other 3 repositories, run some checks, prompt for a tag, merge to production and push. It was only 60 or so lines of node.js code, but it had to be maintained and tested regularly, just like anything else.

Tackling it head on

It was abundantly clear that these repositories were linearly related and that our workflow was being made overly complex by our repository layout. We figured that we could make some gains here.

Our approach to process and tooling change loosely follows the scientific method - we only change a single process or tool per sprint, then we look into it in our retrospectives. We had identified these pain points and scheduled in some time to investigate merging the repository.

Once we had merged the repositories into one, we observed three great benefits:

Easier branch management. One feature, one branch!
Easier reviews. No dependencies! No need to track multiple pull requests against a single Pivotal Tracker story!
Simpler, faster, more consistent demo environment deployments. We were able to ditch our release tool entirely and strip out dozens of lines from our Ansible playbooks and Jenkins jobs.

So, in this particular case, we believe we made a good decision to merge our repositories. When is the right time to merge, though? Is it always the answer? Let’s take a step back and look at the general case:

To merge or not to merge? That is the cliché...

It’s very tempting to separate an application into separate repositories, particularly if they are being worked upon by separate teams. By combining, however, we may run the risk of reducing our separation of concerns. What goes together and what goes apart is a really important consideration for software developers, and one that merits a great deal of careful consideration!

Repository smells

A code smell is defined on c2.com as "a hint that something is wrong in your code", and seems to have been coined by Kent Beck. There now exists a whole taxonomy of code smells, which I’d recommend all developers familiarise themselves with. In the light of this, I define a "repository smell" as a similar concept, whereby a repository or multiple repositories of code exhibit similar hints that something is awry.

I define five repository smells. Two of them should lead us to consider merging code repositories, the other three should lead us to consider drawing code apart into separate repositories.

Smell 1: High correlation in changes

In our case, there was a linear relation between the number of pull requests in our UI and API libraries. In fact, most commits required PRs to both. It therefore made enormous sense to merge them. I’d define this as there being a strong correlation in changes.

If components change together and a change to one implies a change to another, consider combining their repositories.
If you find that you frequently have to make changes in groups across multiple repositories, then perhaps you have a good case for combining them.

Smell 2: Intellectual property dispersal

You may have a client that has an application with multiple facets, which may be in separate repositories. It is often easier to manage permissions for third party access in one repository. Also, you are grouping the intellectual property into one place. Having dispersed intellectual property may increase the likelihood of it ending up somewhere it shouldn't!

If multiple repositories are the intellectual property of a given client and are closely related, consider combining them together.

Smell 3: Feature envy

Much like the feature envy code smell, in OOP, repositories may seem to be overly interested in code from other repositories. This can result in importing a much larger amount of dependency code than is strictly necessary.

Let’s say, for example, that you have a little tool that your repo uses but you also use it in other projects, and you’re checking out the whole project every time just for this one tool.

If you find that an entire repository is checked out just for one small self-contained module, then you likely have a case for it being in its own repository.

If a project is a standalone tool, for instance, I generally advocate having a separate repo with its own documentation, tests and so forth. That way, it’s much better in a “wood for the trees” sense. If a project is a compact, redistributable, standalone tool, then put it in its own repository (and consider open sourcing it as a secondary deliverable).

Smell 4: Too many eggs in one basket

Closely related to code envy, but this is more about looking at the repository as a whole rather than a particular part of it. When areas of a repository are basically unrelated and change at different rates entirely, this may be considered as too many eggs in one basket. For example, you might have multiple tools in one library that aren’t actually related and perform unrelated tasks.

If you find that one subtree of a repository changes in isolation to the rest of the repository, you may wish to break it out.

Smell 5: Permissions minefield

I once worked for a large organisation that kept all their code in a single repository. This was really taking it too far in my opinion! What that can result in is security issues. You’re essentially either trusting everyone with your entire code tree, or nothing at all, which is not a great degree of granularity! Of course, many tools, such as Subversion, do permit subtree permissions, but this can get scrappy quickly!

If you find that you are having to manage granular permissions, it might be a sign that you need to break out a repository into multiple different repositories.

Merging git repos

If you'd like to merge your repositories, a good place to start is this article about Git subtree merges.

A Word of Caution

Make sure that you don’t use merging to a single repo as a shortcut to making all your components as self-describing and self contained as possible and managing dependencies correctly. It may be a convenience in some cases, but it can also result in your repository becoming chaotic.

Thanks to package managers like NPM, Composer or Bower, dependency management in 2015 is much less of a nightmare than it used to be. With powerful tools like Git, a bad repository architecture need not be something you need to live with forever.

Wrapping up

As our application was reasonably small, and functioned as a whole, it made a lot of sense to merge it. Sometimes, though, it’s far better to keep things separate, and the guidelines we’ve defined made here should help you to know when.

Have you had success or pain from your repository organisation? Can you think of any repository smells we haven’t thought of? Let us know what you think in the comments section below! If you have an unwieldy software project, perhaps we can help you!

Faster PHP code coverage

Tue, 10 Feb 2015 00:00:00 -0500

Or how we switched to a powerful new test framework without rewriting any code!

Inspired by both Behaviour Driven Development (BDD) and fluent test libraries like JSpec or RSpec, Kahlan is an exciting new test framework for PHP which allows tight, expressive tests. Kahlan allows you to stub or monkey patch your code directly like in Ruby or JavaScript, without needing any PECL extensions!

Kahlan on Github

To make testing faster, easier, and - let’s be honest - more enjoyable, we wanted to switch our Li3 (AKA Lithium) projects to use Kahlan rather than the native test utilities. Rewriting all our tests in a new framework would be crazy; no client is going to pay us to sit there doing that! We therefore decided to keep the existing tests as they were and simply write any new tests in Kahlan. Simple, right? Just one gotcha - how would we merge the code coverage statistics from both the Kahlan and Li3 tests? We needed to see our overall test coverage so that we could see where we needed to improve our testing. To that end, we needed to work out how to run our tests in one go and have the union of the coverage reported. Kahlan author Simon Jaillet got to work on implementing a wrapper!

Implementation

The first idea was to create a single spec in the Kahlan environment to encapsulate all existing Li3 legacy tests. The main drawback of this approach is that it would mix both the Li3 & Kahlan console reporting, which was not ideal. Thankfully, the Kahlan workflow is highly configurable and so we decided to run all of our Li3 legacy tests after the Kahlan specs had finished to keep console output clean and the two suites separate. To get coverage of our legacy tests, we needed only to enable the code coverage reporter again during Li3 test execution.

Here’s how we did it, step by step!

Step 1: Creating a custom config file

Kahlan uses a simple PHP file as config file, so you don’t need to learn any new funky syntax to configure the workflow the way you want. By default, Kahlan looks for a file called kahlan-config.php, so let’s create one:

touch kahlan-config.php

Step 2: Registering some namespaces

Kahlan works in tandem with Composer and uses its autoloading mechanism. Unfortunately, we were using a legacy installation of Li3 which uses a dedicated loader and some libraries that were not Composer compatible. So, the second step here was to dynamically add some namespaces and a class map to Composer so that all autoloading was done by Composer:

/**
 * Adding custom namespaces to Composer
 */
Filter::register(‘app.namespaces', function($chain) {
    // PSR-0
    $this->_autoloader->add('spec\\', __DIR__);
    $this->_autoloader->add('lithium\\', __DIR__ . '/libraries');

    // PSR-4
    $this->_autoloader->addPsr4(app\\', __DIR__);

    // PSR-0 Li3 Libraries
    $this->_autoloader->add('li3_access\\', __DIR__ . '/libraries');
    $this->_autoloader->add('li3_mailer\\', __DIR__ . '/libraries');
    // etc .

    // AWS class paths
    $awsPath = __DIR__ . '/libraries/li3_aws/libraries/aws-sdk-for-php';
    $this->_autoloader->addClassMap([
        'AmazonAS' => $awsPath . '/services/as.class.php',
        'AmazonCloudFormation' => $awsPath . '/services/cloudformation.class.php',
        'AmazonCloudFront' => $awsPath . '/services/cloudfront.class.php',
        'AmazonCloudSearch' => $awsPath . '/services/cloudsearch.class.php',
        // etc.
    ]);
});


Filter::apply($this, 'namespaces', app.namespaces');

Filter::register() & Filter::apply() are the two necessary methods to customize the spec execution workflow. The first one aims to attach an name (in this case, ’app.namespaces’) to a piece of logic (i.e the closure). Once you have registered your logic, you only need to attach it to a workflow entry point (in this case, ’namespaces’).

Note: Kahlan doesn’t use an event based approach to customize the execution workflow; rather it uses an AOP (Aspect Oriented Programming) approach. For more information on AOP, check out https://github.com/crysalead/filter.

Step 3: Configuring the coverage reporter to only scan a couple of directories

To generate accurate code coverage, we needed to exclude some parts of the tree from being parsed; for example, we don’t need coverage for the tests folder. So in the same way as we did for namespaces, we provided a filter to customize how the code coverage reporter should be initialized:

use kahlan\reporter\Coverage;
use kahlan\reporter\coverage\driver\Xdebug;

/**
 * Initializing a custom coverage reporter
 */
Filter::register('app.coverage', function($chain) {
    $reporters = $this->reporters();

    // Limit the Coverage analysis to only a couple of directories only
    $coverage = new Coverage([
            'verbosity' => $this->args()->get('coverage'),
            'driver' => new Xdebug(),
            'path' => [
                'controllers',
                'models',
               // etc..
            ]
    ]);
    $reporters->add('coverage', $coverage);

    return $reporters;
});

Filter::apply($this, 'coverage', 'app.coverage');

Step 4: Creating the logic to run Li3 legacy tests

Once the coverage reporter configured, we needed to reproduce how Li3 tests are executed inside the Li3 test layer. This specific task has been integrated in the $runLi3Suite closure and $runLegacyTests just adds some coverage logic around to encapsulate the whole execution of legacy tests to be able to retrieve some code coverage data for these tests:

use lithium\console\Request as ConsoleRequest;

$runLegacyTests = function($kahlan) {


    // If the coverage option is enabled, manually enable it for Li3 tests
    if ($this->args()->exists('coverage')) {
        $coverage = $this->reporters()->get('coverage');
        $coverage->before(); // Starts to log coverage
    }

    $runLi3Suite = function($path) {
        $test = new Test([
            'request' => new ConsoleRequest(['input' => fopen('php://temp', 'w+')])
        ]);
        return $test->run($path);
    };


    // Run the Li3 `’tests’` suite
    $success = $runLi3Suite('tests');

    if (!$success) {
        echo "Lithium tests failed\n";
        $kahlan->suite()->status(-1); // Force Kahlan to fail (i.e. exit(-1))
    }


    // Stop Legacy Code coverage
    if ($this->args()->exists('coverage')) {
        $coverage->after(); // Stops to log coverage
    }
    return $success;
};

Step 5: Creating the test execution workflow

This is the final step and actually the most obvious. Here we simply added a filter around the 'run' endpoint which runs all the logic together. In this filter we also added a couple of initializations related to the Li3 framework and declaring a constant which will be checked in the li3 bootstrap file to disable the Li3 built-in autoloader:

/**
 * Boostrapping & running the specs
 */
Filter::register('app.run', function($chain) use ($runLegacyTests) {
    // Used to bail out the Li3 autoloader initialization in test environment.
    // Indeed Composer will do all the job here.
    define('KAHLAN_ENVIRONMENT', 'true');

    // Lithium bootstrap file
    require __DIR__ . '/config/bootstrap.php';

    // Run Kahlan specs
    $result = $chain->next();

    // Run li3 legacy tests
    $runLegacyTests($this);

    return $result;
});

Filter::apply($this, run, app.run);

Step 6: Adding a --legacy option for making the transition easier for developers

During development, sometimes we want to run the legacy tests, and sometimes not. Thankfully, Kahlan makes adding custom CLI (Command Line Interface) arguments easy!

We added a new option, --legacy, to the command line. If this parameter is supplied, the li3 legacy tests are executed and if not they are simply ignored. We just needed to add at the top of the Kahlan config file:

// Adding a, custom option for legacy tests
$args = $this->args();
$args->argument('legacy', ['type'  => 'boolean']);

and then we simply added at the top of the $runLegacyTests closure:

$runLegacyTests = function($kahlan) {

    if (!$kahlan->args()->get('legacy')) {
        return true;
    }
    // ...

Results

This article is not so much about the wrapper to Li3 tests itself, but more to demonstrate that no matter how complex your test setup, Kahlan can be integrated into your workflow.

Switching to another test framework is not an harmless decision, but we started to use Kahlan a couple of months ago on one of our team’s projects and we already see huge benefits using it. We write specs a lot faster and easier and the created specs are a lot more reader friendly for the rest of the team. Furthermore, being able to focus the code coverage report right on a single class or method provide a new spec writing experience for better code coverage.

Even though Kahlan requires some extra cycles for “Just In Time” code patching, it actually ends up making our test suite faster overall for 2 main reasons. Firstly, because we were able to rewrite time consuming integration tests to simple unit tests. Secondly, due to the Kahlan code coverage heuristic, we can make builds with code coverage stats an order of magnitude faster.

Benefits

GREAT SPEEEEEEEED! Both in writing and running tests.
Coverage stats on every build - we saw 40 minutes come down to 2 minutes!
No wastage - we didn’t have to throw away or rewrite our old tests!
Easier test writing with Kahlan

Get involved

Kahlan is open source and MIT licensed! Feel free to get involved by using and contributing to this exciting PHP test framework!

There’s also no reason that Kahlan couldn’t be integrated with tools like Behat or PHPUnit - the sky’s the limit!

Have you come up against slow code coverage or test writing? What did you do about it? Let us know below!

Three Myths of Software, part 3 - The Software Buyer Myth

Mon, 02 Feb 2015 00:00:00 -0500

We aren't a software business, we just rely completely on our custom application. (This is the third installment of a series on the Three Myths of Software Projects. If you haven’t already, check out Myth #1: The Features are Everything and Myth #2: The Genius Geek.)

This myth is the most difficult to absorb because it's easy to look at your software as just one more investment. A business leader might also have a strong perspective on the value of the software in the marketplace. (For example: A taxi company might think of taxi coordination software as being a matter of logistics and not customer service.)

Let's explore the following scenario: Your custom application (aka software) is complete, and it's helping your business grow. You find yourself in the middle of another busy day when, suddenly, the application stops working.

Are you still able to service your customers, even if it’s painfully inconvenient?
Does the outage impact other businesses besides your own?
Can you call an outside vendor who is completely responsible for restoring service?

If you answered no to all these questions, then you are in the software business. This is because your application is:

proprietary
mission-critical
supported in-house

Recognizing that you are in the software business is the first step to becoming competent at managing the ongoing development of your company and software. Once you’ve begun thinking about your software as part of your business and not an expense, you can manage your investment more soundly. How you perceive ongoing expenses should also change because being a software company will require an ongoing commitment to development. As technology changes, you'll need to constantly maintain your software to ensure compatibility and stability.

Have you been affected by any of these myths? Got any of your own? Let us know in the comments below!

Three Myths of Software, part 2 - The Genius Geek

Mon, 26 Jan 2015 00:00:00 -0500

We just need a skilled code-slinger to wrestle this ornery application into shape. Enough talent and this will be easy, right?

(This is the second installment of a three post series on the Three Myths of Software Projects. If you haven’t already, check out Myth #1: The Features are Everything).

Hollywood often projects the stereotype of a genius programmer working alone to create a highly-sophisticated and successful application. That’s a myth, but it's still attractive: you have a great idea, and you know how it needs to work or work better. All you really need now is a skilled geek who knows all the technical stuff. Maybe.

Here are four questions to consider:

Who's going to manage the project? Programming and project management are completely different skills. The ability to assess priorities and unexpected developments while ensuring accountability is critical to moving a project forward. A project manager also provides a direct line of communication that won't interrupt the development work.
But can't I manage the project myself? Unless you or someone in your group is a project manager with technology experience, there is a chance that the most important technical decisions will be made unilaterally by the developer without any input from you. Even the most transparent developer will naturally lean towards familiar technologies and won’t consider the impact to the overall project.
How broad is the technical scope? There are an increasingly diverse number of options and methodologies available nowadays, each with distinct advantages and disadvantages. A lone developer isn't as likely to have the breadth of perspective necessary to weigh all of the options. Similarly, a lone developer will have a limited area of specialization. Work made outside his specialty may be of lower quality or more costly due to his learning curve. The larger the project, the more dramatically these issues will impact the project’s success.

This applies similarly to architecting the project. It's very difficult to maintain focus on the architecture while building features. (See Myth #1: The Features are Everything).
How will you find the right developer? Finding a developer is difficult. It's even more difficult to find a lone developer who can be trusted with the success of your project. The larger your budget, the more experienced (and independent) your candidates are likely to be. The opposite is also true: small budgets rarely engage top talent. Even if you've answered the previous three questions, striking the right balance of budget, skill and independence is key to ensuring the right developer on the right project.

Sharing and challenging each other with multiple perspectives, backgrounds, and skills is invaluable to ensuring a strong outcome.

An interesting resource that documents how this is the case is Walter Isaacson’s The Innovators.

Next up is Myth #3: The Software Buyer Myth.

Three Myths of Software Projects

Mon, 19 Jan 2015 00:00:00 -0500

The backstory for most software projects is predictable: A business opportunity is identified, and features are listed that will exploit that opportunity. At some point the list of items starts to take shape as a tool or application. Suddenly you're staring at a full-blown software project full of ideas and promise but without a clear path to accomplish any of it.

But first a little background. This is about managing successful development projects, not how to write a program. The difference is that a successful development project will be managed toward a defined and non-technical goal. To be successful at writing a program requires more than the delivery of something technical, which ignores the business goals that prompted the effort in the first place.

To look at it differently: If every application is born out of a goal to achieve something, the success or failure of the application should be measured against that initial goal, not simply the delivery of a computer program. Describing our effort as “managing a successful development project” helps us maintain sight of real success.

We bring this perspective to every conversation, and it helps us recognize when the conversation becomes application-centric instead of success-centric. We've identified three of the most common reasons this happens; here is the first of Three Myths of Software Projects:

Myth #1: The Features Are Everything

If we build all these cool features, we'll have the best application ever.

Most people can most easily articulate their application as a list of features. At its simplest, this list includes some functions that produce some results.

A common (and cliché, at this point) example would be a blogging application. We could write that the following features would make a basic blogging application:

Admin authentication
Text editing & styling
Ability to publish text

Seems pretty good, right? If you have these things, you'll have a blogging application. If you rewrite this list in sentences, then you'll have what we call stories. (This is part of the Agile approach to managing software.) These same features rewritten in natural language might look like this:

An admin should be able to log in and change credentials.
An admin should be able to create, edit, and delete essays.
An admin should be able to publish essays.

You can see how the second list fills in some of the details assumed, but left undescribed, in the first list. In fact, the first list now seems a bit oversimplified compared to the second list. But this second list isn’t accurate or comprehensive either. This is the trap of The Features Are Everything Myth: A successful application is simply a combination of features. No matter how detailed your stories or feature lists are written, your custom application is inherently more complicated.

The trap of the features list is that they always seem accurate and comprehensive but always turn out later to be vague and oversimplified.

Before you launch any serious investment, be sure to document the initial goals you were seeking before you began. Those are your success factors and should be compared with the delivered features at regular intervals. If you do this, you’ll have a steady target and will be more likely to achieve measurable success.

Next up is Myth #2: The Genius Geek.

A Note From The Editor

Thu, 01 Jan 2015 07:47:04 -0500

I am not much of a relationship expert. My usual advice? That imagined afternoon where you see true love become reality? That’s probably going to stay imagined. I guess that’s not advice, that’s just me playing the odds. If I were to imagine the same scene with just a bit more self-applied cynicism it might turn out closer to a tawdry episode from "Girls" or "Eastbound and Down". Which would be a lot messier and a lot more sad.

But like every drama with exposed flesh hanging on end-of-episode hooks, there are promises and triumphs. It is entertainment, after all. Still, even with all the excitement, these tropes eventually fall a little flat. Excitement morphs into morbid curiosity or even loyalty to fictional characters in a stubborn wait for meaningful resolution. Even that gift of meaningful resolution is eventually reformed into a search for something new. Or, if a promise isn’t completely crushed, maybe it’s a search for an even better triumph.

Searching is probably the wrong word here, because searching involves finding and finding is actually pretty boring. It’s presumptuous anyway: the idea that one actually knows exactly what one is looking for.

To be clear, I’m not talking about that recent Amazon search for spicy pumpkin seeds. I’m talking about important stuff like: What is it exactly that you do? Why should it matter? That sort of stuff.

These are tough questions that everyone knows how to ask but are unprepared to answer for themselves. They are, however, exactly the sort of questions that have dominated the past year here at Radify. Among the answers? A new logo (Thanks, Josh!), a few shakeups, and so... much... work.

This web site is really just a small, but pretty accurate taste. I think it represents well our triumphs and our struggles: it’s a beautiful veil of colored pixels concealing so many gray conversations about direction and priorities and strategies. Tough questions and answers, now artfully represented by flashes of light and color. Thanks to our newest team member, Andrew, it’s all looking pretty good.

More than anything, I hope that the website gives a sense of how Radify is different. All of us share a deep excitement for technology and the web and experiences and all the stuff that allows us to do great work. But these things are the boring and more methodical parts of our work. These are the procedural searches. The fact is, our passion is discovering, and understanding this has been a big part of 2014 for us. Today, at the beginning of a new year, we join together stronger than ever with a common vision of a world where technology grows into a silent servant to human imagination.

Happy 2015 from Radify.

Forward-Only Deployments

Fri, 28 Nov 2014 09:16:01 -0500

Any update to a production system is a potential disaster. No matter how hard we try to make things absolutely perfect, errors in production can occur. So, when a problem does occur, what should we do?

In this document, I define a regression as “a bug that was introduced by a commit” and we will use this term for any problem in production that was caused by a code change.

It’s all gone wrong!

So you’re maintaining a system for a client. You’re doing continuous deployment. You’re using immutable infrastructure. You’re using feature branches to demo features and it’s all going well. Until one day, your monitoring system (or, worse still, your client! See our post "4 Principles of DevOps" for info on avoiding your clients being the ones to report bugs) sends you a message “production is down!” Your blood runs cold — what do you do?

A cursory examination suggests that the problem corresponds to a recent deployment (New Relic, for example, allows you to mark deployments, which is very useful for diagnosing bugs in production). We need to pull the latest code off, and fast!

We can basically do one of two things when such a regression occurs:

Option 1: Rolling back

When the dreaded production bug occurs, you can simply tell your Continuous Deployment system to push out an old version. Therefore, you’re pushing out the last known good tag.

What’s wrong with option 1?

It might seem good at first, but pushing out an old version creates several problems:

The system state is not the same as HEAD.
People can come along and deploy by committing to production, thereby re-breaking it without realising.
There is a reliance upon person-to-person communication, and everyone is busy and it’s easy to miss things even if we are diligent.
Requires additional automation - you would require some kind of “deploy old tag” functionality outside your existing deployment mechanism.

Option 2: Forward-only

Create a reverse merge, commit it, tag it and push it:

So, instead of putting an old tag onto production, we have created an additional commit (dd3a) which reverts the breaking commit (998b). This means that we are only ever going forwards. We are only ever using the release tool to push releases. Nothing “magic” is happening.

Why am I recommending option 2?

Forward-only means that you’re only deploying whatever is the latest version. It can’t get out of sync. Advantages:

HEAD and deployed code are not out of sync.
Lower risk of rolling out bad code accidentally. In option 1, you could end up continuing to develop against a broken HEAD.
Code is in a working state in the repo as well as on the production environment.
Does not break your existing automation - you don’t need a “special case” to push old tags.
Works particularly well with immutable infrastructure

Wrapping up

We advocate forward-only deployments for the reasons mentioned above. Obviously, this is for critical production problems only. It’s the simplest, fastest way of getting production back into a working state without compromising repository, automation and development integrity. Of course, you still have to find and fix your problem and re-integrate the changes from the breaking commit, but that’s a separate issue! The purpose of this article is simply to define a strategy to buy you some time while you get things sorted properly.

Do you agree or disagree with our strategy? How do you manage production bugs?

Immutable Demo Nodes

Fri, 14 Nov 2014 07:42:45 -0500

In our recent post “Reducing Infrustration”, we described how Radify adopted immutable infrastructure and what the benefits have been. In this post, we talk about how we integrated immutable infrastructure with our feature branch workflow.

Automatically deploying feature branches

A commit to a feature branch are automatically picked up by our Continuous Integration server, Jenkins. If the build passes, then it is automatically deployed via Ansible to a brand new EC2 node and given a CNAME so that it is reachable at {branch}.demo.{client}.{tld}:

A developer pushes code to a branch. If it passes the build on Jenkins, an Ansible playbook is kicked off, which:

stands up a new micro instance based on our Radify base box
installs the application
creates a CNAME
notifies the team that fresh code is available for testing
finally, updates our minimalist open source branch manager, StationMaster

If a commit is made to an existing feature branch, this means that the existing micro node is thrown away and a new one is stood up in its stead with the latest code on it and that all the team know when a feature branch has been successfully updated and deployed.

Automatically removing expired feature branches

It’s important to keep the branch count in check - after all, every node costs money! Therefore, on every commit to any project, we run a script that goes through and checks for any expired branches. If it finds any, it deletes the CNAME, burns the node, and removes it from StationMaster. Here it is in pseudocode:

FOR EACH branch on demo
    DOES IT still have an active branch?
        YES
            leave it alone
        NO
            delete CNAME {branch}.demo.{client}.{tld}
            terminate node
            remove from StationMaster

Why do this?

For us, this replaced the way we used to work, which was having a medium instance with multiple workspaces on there with an Nginx configuration for each one. This meant that the branches performed well, but there were several problems:

We couldn’t throw away the node. Because the node was shared, we couldn’t just bin it in the same way as we do with the rest of our infrastructure. This meant we had to keep on top of manual patching.
Deployment method was not identical to production. See our post, 4 Principles of DevOps, on why this doesn’t match our principles.
The Nginx config was SLIGHTLY different to accommodate multiple workspaces. Only ever so slightly, but in my experience, we want to be absolutely as close as humanly possible to production.
Multiple workspaces on the same box makes environment variable configuration impractical, which was restricting us and overcomplicated our Ansible playbook.

So now, we have demo nodes that are handled exactly the same way as production. Great! There are, of course, at least four drawbacks.

Firstly, performance: micro instances do not perform nearly as well as the others. That said, it’s actually quite good to test your application on smaller hardware than it is targeting, it’s sometimes a useful way of getting an idea of where bottlenecks may be.

Secondly, they are not behind load balancers in the same way as production nodes are.

Thirdly, the cost. Micro instances are as cheap as it gets in AWS, but if you have a lot of branches that are long-running, it can mount up. To mitigate this, being disciplined about closing any unused branches is essential.

Finally, having demo nodes does obviously slightly increase the surface area of your application.

Benefits to the client

All the tools we’ve built to do this are fairly generic, extended slightly per-client. It means that:

Features can be tested before integration, meaning the dev team can check one another’s work
Integrations can be tested in an integration branch
Greater confidence that testing on demo is representative of production
Full automation - no need for dev team to spend time poking about on boxes
Ability to quickly push out alternative branches for client A/B testing

What do you think?

We've found that for demonstrating features, these small, cheap, disposable environments are ideal for us. Do you automatically push out feature branches for testing? Would you like to? Not sure where to start? Think it’s a daft idea? Let us know in the comments below or feel free to get in touch with us directly if you'd like some help setting this up!

Announcing Angular-G11n

Fri, 24 Oct 2014 13:13:07 -0400

A recent project required powerful yet simple G11n support in AngularJS. It had to be lightweight, easy to use, and to support switching locale on the fly. We also needed to be able to export to and import from CSV so that our client’s translators could quickly and easily add new languages and update existing ones.

angular-g11n on github

angular-g11n has the following features:

Set up in minutes
Extremely simple JSON format
AngularJS service accessible from JavaScript code
Filter accessible from templates
Interpolation (e.g. “Hello {{ name }}” can output “Hello Dave”
Can respond to empty, singular and plural cases (e.g. “no messages”, “one message”, “40 messages”)
Export to and import from CSV
On-the-fly language switching
Complete sample implementation
You can access G11n from code as well as from the template. So, if you needed to translate something in a service or controller, that's no problem - with angular-g11n!

Sounds good? Head on over to github.com/uor/angular-g11n and check it out!

The Bot Who Cried Wolf

Fri, 24 Oct 2014 13:13:07 -0400

At Radify, we practise Continuous Delivery and have all sorts of tools and processes that support us in this. Many of these tools have notification abilities, and it’s tempting to turn on ALL THE NOTIFICATIONS! Unfortunately, this led to a case of “the Bot who cried wolf”, in that they grew to be ignored, and viewed merely as annoying noise. It’s not so much about crying “wolf” when there is no wolf, rather the bot could be seen as as essentially going to a zoo, standing by the wolf cage, and screaming WOLF! WOLF! THERE’S A WOLF! LOOK! RIGHT THERE IN THE WOLF CAGE! IT’S A WOLF!

Now, if you were in your local store shopping for carrots, you’d want to know if there was a wolf. At the wolf enclosure in a zoo, not so much. The bot is dumb, though, and by default it just screams at every wolf it sees. How can we make this bot smarter without discouraging it and making it cry? After all, it exists to alert us of wolves...

It was getting crazy up in here

Before we started addressing our notification strategy, a typical merge to master could trigger the following barrage notifications in our IRC channel:

Github: HEY I GOT A COMMIT, U JELLY?
Pivotal Tracker: THIS IS RELEVANT TO MY INTERESTS, AH YES, THIS IS SOMETHING ON THIS HERE TICKET, IS IT NOT?
Jenkins: I’m starting a build!
Jenkins: AWWW YIS this commit is now all UP IN MY PIPELINE… I’m starting a build!
Jenkins: OK it’s fixed now, good work, and now I’m pushing it out to dev via Ansible! Bet you can’t wait!
Jenkins: I’ve deployed code to dev!
Github: This has been merged!
Pivotal: I am updated as well!
Jenkins: RIGHT! Time to deploy to staging!
Jenkins: … OK it’s deployed to staging!

Noise annoys and this was way too much! This meant that important messages were getting lost amongst the chatter of the bots. It’s pretty tempting to simply turn on ALL THE MESSAGES, but is that really a good idea? This made us think; what’s actually a USEFUL notification?

What is the best policy?

One thing we realized is that, quite often, notifications became a proxy for things that we wanted to know the status of; information that our other tools did not readily present us with. One example: the build status of in-development branches. Notifications are a transient medium, which makes them a poor fit for persistent information, like the status of something. Resolving this was one of the driving forces behind the creation of StationMaster, our branch status tool (as promised, we’ll talk more about StationMaster in an upcoming post).

After some consideration, our notification policy came to be; “if you receive a notification, it should be for (a) something that you need to do something about, OR (b) it should be letting you know that something important has happened that, if questioned upon, you should be aware of”. Otherwise, we have too much wolf-crying!

Once we’d discussed our strategy, we centralised our notifications into Leeroy (which is what I, and probably a thousand other devs, named our Jenkins IRC bot). Only Leeroy (and occasionally Pivotal) speaks in the channel now. Other notifications come via email or pull request build statuses in Github fed back from Jenkins - that way, only the people involved have to see them, others are not distracted, and the channels become less noisy. Here are the notifications that we commonly use:

Team notifications

These are notifications that go to the entire team. These include:

New code on demo! We auto-deploy each feature branch so that it can be tested by manual QA (see our post “Four Principles of DevOps” for more on our workflow).
Something has been merged to staging and deployed!
A release to production is about to happen! Everyone pay attention! When we do a release to a customer, everyone in that project’s channel should be aware.
Build failure of releases. No release should be partial or incomplete due to our policy of immutable infrastructure for our API and UI nodes.
A new release went out. Phew!

All of these are of relevance to two or more members of each project team.

Direct to user notifications

These are notifications that only go to the person who they are of interest to.

Email of build failure/fixed. We generally use feature branches for development - Alice doesn’t need to know if Bob broke the build. Therefore, only Bob gets broken build notifications for builds that Bob broke. Alice (and the rest of the team) don’t need to know about what’s going on with that feature branch until Bob is ready to merge it.
Pull request builder. So, if a pull request build fails, only the people involved need to know about it. If, however, a release is in progress and fails, everyone on the team needs to know, because that’s headline news (due to our immutable infrastructure policy, if it does fail, nothing will happen from the client’s perspective, but the dev team definitely need to jump to attention!).

Notify us!

What is your organisation’s notification strategy? How do you balance signal and noise? What are the weaknesses in our approach? Let us know in the box below!

Reducing Infrustration

Fri, 03 Oct 2014 09:22:54 -0400

Radify recently adopted an immutable infrastructure approach for our web and API nodes. In this article, we talk about what it is, why we did it, and what the costs and benefits have been.

What is immutable infrastructure?

Immutable infrastructure, or an immutable deployment, is where infrastructure never changes - it is simply completely replaced wholesale when a deployment happens. It is an attempt to control the amount and location of state in a system. Instead of the historical pattern of having a group of servers and maintaining them over time, with immutable infrastructure you stand up new servers on every deploy. You then install your application on them, add them to the load balancer, then remove the old ones and destroy them. You can achieve rapid results by having a custom base box, which you provision in advance, so that only your code needs to be deployed.

Why use immutable infrastructure?

I’ll be honest, part of the initial impetus was to be part of the "cool gang". All the other kids are doing it, why not us? Initially, several objections raised themselves. Foremost amongst these was, and I could be wrong, but it seems to me that a lot of the people who talk about immutable infrastructure are fairly large, high traffic, product-based organisations. Was immutable infrastructure out of our reach as a relatively lean agile software consultancy? And is it necessary? Most of our projects are B2B client projects — we’re certainly some way from “Netflix traffic”!

The more we discussed it, however, the more sense it made. We had a strong build pipeline already (described briefly in 4 Principles of DevOps), but we were finding that the packages on our servers were getting out of date. Our projects have at least 5 stages (local development, test/demo, end to end testing, staging and production) and each stage has 1..n nodes – so although we’re not exactly talking Facebook numbers, updating these servers could be a pain. Small example: updating them via apt over Ansible worked okay most of the time, but occasionally servers would require reboots, and we would have had to stagger those… it just became a lot of hassle. Heartbleed was a couple of hours spent patching and testing that we didn’t want to go through again.

There were other issues with having to maintain running nodes. They could get out of sync – if, for example, we stood up a new node and added it to a cluster, perhaps the new node may have a newer version of a given library? This never caused us any problems, but potentially, it could have done. It’s best to have every node in a cluster be utterly identical, otherwise debugging can become non-deterministic.

Immutable infrastructure means that we can simply stand up a new set of nodes every time, and these will be provisioned with the latest packages from the get-go and to have each environment be in a known state, so it seemed to us that it was worth investigating!

Telling the story

I ran some tests and initially a full provisioning of one node for a particular client from a bare bones Ubuntu LTS AMI (Amazon Machine Image) took nearly 15 minutes. This diagram shows a greatly simplified version of this (I haven’t listed everything we install, just some examples):

We initially evaluated Packer for this project, but it didn’t really seem to give us anything that our existing tooling wasn’t already giving us. I didn’t like that it used a local Ansible, which installed rather more dependencies than I would have liked. So, we decided to refactor our existing playbook to take out the common stuff that is the same or similar between clients and create a radify-base-box playbook which builds an AMI. We update this AMI nightly to make sure that it’s always ready to go with the latest security fixes and so forth. Then, applying the client’s playbook took only two or three minutes per node. The diagram below shows this:

This solved our issue of slow deployments, and also allowed us to have a common base image across similar projects which is consistently configured. If we make some key change, we can even do that cross-project. We are also able to use this base playbook with our Jenkins cluster, which is also provisioned by Ansible, so that we have the very same infrastructure across the board. Pretty nifty!

Benefits and challenges

Benefits

Encourages you to log "off box" - we use PaperTrail for this
Far less time spent babysitting servers. This is the "treat servers like cattle, not like pets" approach. If a server goes nuts, we have its logs, and we can simply burn and replace it immediately with zero downtime.
Insulated from the risk of an external dependency being down. For example, if we install some node modules on our base box, this protects us from NPM having issues. Updating our base box periodically means that even if it’s down for some time, we are not far behind the latest patches.
Prevents the temptation to "hand hack" servers
Boxes are in a known state and have the latest security patches
Disks can’t fill up with logs etc
Consistent infrastructure cross-project
No real change to the cost - unused nodes are destroyed shortly after they are removed from the load balancers.
Massive reduction in the amount of playbook code. Lots of red lines in the pull request! A couple of examples:
- The playbook that updated the code but not the environment? Gone!
- The playbook that updated the dependencies? Gone!

Challenges

There are various challenges with immutable infrastructure:

Time spent working on automation. I think that this work has great value to Radify as a business, and to our clients, but nothing comes for free. I spent perhaps 3 days setting this up, plus our team’s time reviewing it and several meetings discussing it.
Some people may be uncomfortable with the concept of disposability in services.

Wrapping up

The 12 Factor approach isn’t for everyone - it certainly has its critics - but for us we have found it makes things clean and the moving parts don’t grind together and throw springs all over the place. Adopting immutable infrastructure has been a big step along this road for us.

Not having to worry maintaining nodes has been something of an administrative weight off our collective shoulders. What’s really quite encouraging is that we have been able to do this despite being a fairly small team.

4 Principles of DevOps

Wed, 02 Jul 2014 12:02:47 -0400

There are lots of misconceptions about “DevOps”. Some people think it’s a set of tools (e.g. “Puppet is our DevOps tool”). Others think it’s a role in the organisation (e.g. “Charlie does our DevOps”). It’s neither of those things - I suppose I’d describe DevOps as a way of thinking. At Radify, we work from 4 principles:

Holistic system thinking
No silos
Rapid, useful feedback
Automate drudgery away

In this article, we look at these principles one at a time, demonstrating how they drive our process, tooling and workflow.

Principle 1 of 4: Holistic system thinking

Holistic system thinking simply means thinking about the whole system – in fact, the whole project and the ecosystem around it. It is the opposite of thinking just about “my little bit”.

As a coder, it is often tempting to try to solve all problems with code. Got a problem? Throw code at it until it goes away! For a long time as a developer, my only tool was the sledgehammer of code, even when the nails were actually screws. Or rawl plugs, whatever they are – you get the picture. As I matured and learned from the people around me, I realised that many problems were not, in fact, code problems at all.

Defining the problem space

This Venn diagram shows the total space of problems that we encounter day-to-day (orange). The green is the ones that code is the right answer to. The red is problems that you can solve with code but you really shouldn’t. As Lemmy put it - ”just ‘cos you got the power, that don’t mean you got the right”! The rest of the space is problems that must be solved by non-code means.

There are all sorts of ways of solving our problems. Here are a few examples:

In code
At the database layer
In operations - e.g. adding more hardware, switching our service providers
Communication - some problems can be solved simply by talking
Writing a quick and dirty one-off script
Outsourcing to a dedicated service or company

Holistic system thinking is about, as a member of an organisation, being able to think about how that problem can be solved in as many ways as possible, and taking into consideration every aspect of the product, the customer, the infrastructure and so forth.

How we apply holistic system thinking

Thinking outside of code

A lot of problems can be solved without “doing any work”. For example, if a customer comes to us with a requirement, we don’t immediately go into “heads down, writing code” mode. There are often other solutions - as a tech company, you are hired for your expertise. Sometimes, it’s simply a training issue, or perhaps there is a workaround that can save months of work. The temptation to write code for everything is one that plagues all of us, but it’s rarely the only solution.

Holistic system thinking allows us to say things like “we don’t necessarily need to fix this performance issue in code. That could take us days. We can simply add more nodes in EC2 for the time being for a few dollars a day”. That’s not to say that is always the right approach, but it’s important to look at each problem and try to assess the approximate cost of addressing it head-on and whether there are other ways of resolving it.

Some problems can be solved with a quick-and-dirty one off script. Others need a carefully architected system. Knowing the difference is very difficult, there are no hard and fast rules, but at least having the option to think about the whole system, the hardware it runs on, and the client you are serving gives you a lot more options! This can make you faster, leaner and more profitable.

Asking for ... help?!

Never be too proud to ask for help. That is in no way a weakness! In this age of service oriented architectures, outsourcing to specialist services can be a massive time saver, and can result in more secure and stable solutions. Furthermore, it is often prudent to bring in a consultant for short periods. A long period with a consultant can create unhealthy dependencies, but in the short term, it can give a project a great boost.

Principle 2: No Silos

A silo is the way of thinking that says “this is my little bit, I’m responsible for just this little bit”. Silos are destructive; they lead to territorial behaviour, break down team cohesion, and in extreme cases, can slow development to an absolute crawl. One of the main concepts of DevOps is to break down the “wall” between devs and operations - so it’s no longer about developers “throwing code over the wall” to be deployed by systems administrators. We take it further than that - for us, it’s also about breaking down walls between dev, marketing and graphic design - as many roles as possible.

KPIs (Key Performance Indicators) are one of the very worst causes of silos, be very careful if you’re using them. They can cause people to focus on a narrow objective to the huge detriment of the organisation at large. In fact, my personal opinion is that they are utterly poisonous, although perhaps I go too far in this.

How we apply no silos

There are a few facets of our workflow that I will highlight to show how we use the principle of no silos.

Pull request workflow

Our pull request workflow helps us to share information. It gives the following benefits:

A second set of eyes. I’ve lost count of the number of times I’ve solved a problem, and somebody has reviewed it and pointed out a subtle flaw, or perhaps a way I could solve it more elegantly. This results in a much cleaner codebase.
Knowledge sharing. Say Alice writes a feature, and Bob reviews it. This means that Bob has at least SOME idea of what Alice’s feature does and how it works. Should Alice be vapourised by marauding space pirates, the cost to the business is slightly less.
Less ego in programming. I have to be egoless, and accept the feedback of my peers. One’s natural inclination to take feedback as criticism can be set aside, and this is healthy.

Anyone can deploy

One of the main silos can be a “release person” who has to do the release every time. This can cause changes to stack up. We prefer to have the approach that no-one should be afraid to trigger a deployment and it should be easy. We should have confidence in our code as it has gone through so many feedback loops before it can get near production (tying in to automate away the drudgery and rapid, useful feedback).

Regular communication

We want everyone on our team to share knowledge. We hold daily standup meetings and exchange ideas and questions.

Writing documentation

Documentation is absolutely a deliverable. This comes in the form of working code and tests first and foremost, but a “runbook” style README.md that tells you how to, for example, install, start, and stop an application, and where the logs are, is incredibly useful. Common gotchas should be listed in there also. Any time anyone asks me a question, that’s generally a prompt for me to document something.

Principle 3: Rapid, useful feedback

We want to know about problems before our customer does. We want as few problems as possible to make it into production, and if a problem DOES reach production, we want to have the right telemetry in place to be able to diagnose it rapidly.

We adopt a ‘defense in depth’ approach - each phase of our workflow has feedback built into it.

How we apply rapid, useful feedback

Automated feedback

On our local workspaces, we write code and tests and implement features. This is the tight feedback loop that Kent Beck describes in his book “Test-Driven Development”. Once we’re happy with our work, we push it up to a feature branch.

The feature branch is automatically tested with all the unit and integration tests, plus static analysis and whatever other tools we are using. If all that passes, then end to end (e2e) tests are run. If these are OK, then a deployment is triggered to a shared dev workspace that can be tested. We have an in-house tool called StationMaster which provides a list of these workspaces and when they were updated, as well as allowing us to remove stale branches.

At this point, it’s been tested by all our automation (tying into automate away drudgery) and it’s ready for manual testing!

Manual feedback

Here is our manual feedback loop:

This loop only kicks in if all the automation has passed; we don’t waste our time reviewing anything that hasn’t passed our tests already. The shared branch that is deployed via StationMaster can be clicked through to straight away by the person reviewing the pull request. This ties closely into no silos as it shares knowledge and improves our confidence in our work. See my blog post about how we do pull requests for more information on this.

The merge to master operation is simply clicking the pull request merge button once the reviewer is happy. This automatically deploys to staging. At this point, the features are shown to the customer, giving us even more useful feedback! If it’s OK in staging, the feature can be merged to production and out it goes into the wild!

Feedback from beyond production

So that takes us up to production - however, once our code is “in the wild”, we still need to know what’s going on! We need to know things like:

How is the application performing?
Are errors being thrown?
Are any nodes down?

The idea is that we get rapid and useful feedback beyond our development cycle. We use all sorts of tools to gain telemetry on our applications, including:

Log aggregation. If you’ve ever had to go log diving, you’ll know what a nightmare it can be if your app scales across multiple nodes. We use Papertrail to aggregate logs from multiple web servers in a nice, searchable web interface.
Server monitoring. You never want your customer to be the one telling you there are performance problems! We have monitoring running on all our nodes. We use Pingdom, NewRelic and the AWS alerting service.
Stack trace reporting. If there is a code level bug, you want to know about it before your customer does. We use NewRelic for this.

The tools you use may be different; we re-evaluate our tools all the time. It’s the principle of getting rapid, useful feedback that is important!

Non-technical feedback

How are we working as a team? Is our strategy effective? Are we making good use of our time? These are all useful questions to ask. We hold regular sprint planning and retrospectives as well as discussion meetings on company strategy.

Principle 4: Automate drudgery away

I often describe this as “constructive laziness”, which is a cheeky way of saying that our time is precious - it’s vital that we do not squander it! Humans are very good at innovation, problem solving and creativity. Conversely, we are utterly dreadful at drudging, repetitive, tedious tasks. Therefore, automation is key!

We’ve talked about getting rapid useful feedback and you may have noticed that there is a lot of automation. Tools are great, but the important thing is that we are not doing anything manually that we can automate - our brains are amazing things but they are not machines. It’s best to use our time to solve interesting problems and be creative, rather than grinding through things like deployments.

How we automate drudgery away

Representative environments throughout our pipeline

It’s really important that, when developing a system, our development environments are as representative of the production environment as possible. Therefore, we provision all our development environments using the very same Ansible playbook that provisions our test, feature demo, staging and production environments. Therefore, at every stage of the pipeline, there’s one less thing that can go wrong - one less chance for the the “it works on my machine!” bomb to go off!

This diagram shows some of the environments we would typically use on a project, and shows that they are all provisioned identically. This allows us to have confidence that the application we produce will behave the same on production as on our local dev environment. This ties closely into holistic system thinking.

Workflow

Here is a diagram summarizing some of our automation:

Notice that CodeShip uses Ansible to automatically provision environments and run tests, static analysis and run deployments. It’s all set up so that we use the most standard tools that we can, to try to keep it as open and transparent as possible (tying into no silos). Much of the feedback we talked about earlier comes to us through automation. Our process for standing up new nodes is automated. This frees us up to work on more interesting problems!

Our advice

You may have noticed that all our principles relate to one another; there’s a huge amount of overlap:

It’s all part of one picture; whilst we love technology, we are not “technology driven”, in that it’s not about the tools - the tools are an implementation detail. It’s about what you want to achieve, and what your values are. Figure out what’s important to your organisation - we suggest you work from principles rather than “we need to use Ansible” or whatever. This way, you can always improve, always move forward, and you don’t become narrow and entrenched.

Once you’ve got your principles - in effect, your vision for the organisation - implement goals step-by-step; you don’t have to do it all overnight. If you’re operating from principles, rather than strictly defined regiments, you can be flexible and try things, see what works. It’s about reducing the cost of failure, and more than that, it’s about failing as fast as possible, identifying what isn’t working, and improving.

Want help with your organisation’s operations? Got a vision that you don’t know how to carry out? Get in touch with us!

Perfect Pull Requests

Mon, 23 Jun 2014 08:40:22 -0400

What would you want to read?

At Radify, we use a whole battery of automation to test our work. This is great and it helps us to keep our quality high. It’s not the whole story, though - manual feedback is just as vital as automated feedback! As such, pull requests are a really useful part of our workflow.

This diagram shows how we operate. A developer writes a feature, and pushes it to a feature branch, which is automatically deployed to a dev URL for testing (more on this in a subsequent post). Another developer reviews and merges this code.

This is nothing new, of course, and lots of companies adopt this paradigm, because it gives the following benefits:

A second set of eyes. I’ve lost count of the number of times I’ve solved a problem, and somebody has reviewed it and pointed out a subtle flaw, or perhaps a way I could solve it more elegantly. This results in a much cleaner codebase.
Knowledge sharing. Say Alice writes a feature, and Bob reviews it. This means that Bob has at least SOME idea of what Alice’s feature does and how it works. Should Alice be vapourised by marauding space pirates, the cost to the business is slightly less.
Less ego in programming. I have to be egoless, and accept the feedback of my peers. One’s natural inclination to take feedback as criticism can be set aside, and this is healthy.

Writing a good pull request

So, what makes a GOOD pull request? Well, the way I think about it is; “what would I want to know if I were reviewing someone else’s pull request?”. Here are what I think the characteristics of a good pull request are:

1. Includes unit/integration tests

Tests show the code executing. Submitting a pull request with no unit tests does not demonstrate how the code has been used, or how the developer is thinking. Unit tests are a whole

2. References a ticket

A pull request must, in some way, link to your issue tracker. There should pretty much always be a ticket that you attach your work to. This gives you increased visibility in your workflow. You should use a commit message format like:

[#187412] Implement the flux capacitor

* Add the Y shaped shiny thing
* Tests demonstrating hooking the shiny thing into the drivey thing

In this instance, [#187412] is our issue tracker number. Most issue trackers like Pivotal or Jira can hook into Github to update tickets automatically, so your work is visible to your team and even to your customers; they can see “hey, Amy has finished the flux capacitor story and it’s ready for review”.

You might also wish to use microformats in your commit message!

3. Is neatly squashed into a single commit

If it’s only you working on branch by yourself, then consider squashing before committing. A tidy history is much easier to work with than sporadic features, it has the following benefits:

All the changes for a single feature are grouped together
A commit can be reverted wholesale if there’s a problem, rather than piecemeal
Much easier to search history

It’s seldom a good idea to rebase a shared branch, but on your own branches, be hygienic and squash things!

4. Is not created until all automated tests pass

A pull request should only be submitted once a feature has passed your static analysis, unit tests, end to end tests and whatever else you have in your pipeline.

5. Has a comprehensive description

As well as having automated tests, a good pull request should list the steps to test a feature. I generally use the following format:

Title: [#4321521] Implement the territories interface

# Description

* Add an interface for maintaining territories
* Tests for territories interface and model updates

# Dependencies

* Relies upon myotherproject/pulls/42

# Screenshots

(screenshots go here)

# Test script

* Log into http://4321521-territories-interface.dev.ourcompany.com using admin credentials
* Click on ‘admin’
* Click on ‘territories’
* Make sure you can remove a territory, and that this causes an XHR that deletes the territory from the API
* Make sure you can add a new territory, verify in the database
* Make sure you can edit a territory, verify in the database
* Check that you can’t create a blank territory

Firstly, we have a description. This is likely to be taken from your commit that you’ve squashed down to.

Then, dependencies is a list of any other pull requests this one relies upon - for example, you may have separate projects for your API, UI and provisioning scripts, so cross- reference them here to prevent things from getting out of sync.

Screenshots are optional but they give people an at-a-glance visual - often, if things look dreadful, or something is clearly wrong, it saves a lot of time if the reviewer can see that from a screenshot.

Finally, I include a test script. You might have user stories that already have acceptance criteria; if that’s the case, then simply link to them. Sometimes, though, you don’t have that, so it’s really helpful if you show your test script here. Think of it this way - if you’re trying to review someone else’s work in a part of the system that you know nothing about, would you even know how to test it? It can save time and confusion to list steps here.

For example...

Get your coat...

So, that’s how I think when I construct a pull request - “in the reviewer’s shoes, what would I want to know?”. Sometimes this can be a bit over the top but generally it takes me less than two minutes to craft my pull request. I think that it can save my colleagues a lot of time if I structure it in this way.

So, what do you think? Do you use a pull request workflow? What kind of information do you include in yours?

Git, Microformats and Metrics

Tue, 10 Jun 2014 10:05:05 -0400

Wouldn’t it be useful to know, project-by-project and over time, how much time your team is spending on finding and fixing bugs? It could sure make for some pretty graphs and visualisations!

This article shows you how to use microformats in your Git commit messages to gather useful and interesting metrics on your software projects.

Why gather metrics?

Determine how much time spent finding and fixing bugs. You may think your team are working on features most of the time, only to be shown that actually they are spending most of their hours firefighting! If this is the case, it’s better to know rather than carry on regardless.
Track differential over time. Do bugs get quicker to find as the team grow more familiar with the client’s business aims and codebase? Are things taking longer and longer to diagnose and fix? This could be a sign that you need to stop and address technical debt. In our particular case, we have developed some hypotheses about tooling and methodologies to support isolating and identifying defects, and we want to test their effectiveness over time.
Automatable — it’s not a great deal of effort. If this information could be useful in the future, then now is a good time to start at least recording it!

Defining a microformat

Many tools already support a commit message format you’re likely familiar with: using [#Numeric ID] to reference a ticket or issue, as employed by Trac, GitHub, PivotalTracker and others.

When tracking your own data, you should use a format that’s easily parseable by humans and machines, and is easily distinguishable from normal commit message content.

For ease of use and consistency, we decided on the following format:

Fix pagination bug

Pagination was causing a buffer overflow by repeatedly hammering the server for every page in parallel rather than requesting a page at once.

Added test that verifies that this no longer occurs and counts the number of requests made.

ttc:2
ttf:1.5

So, all we have is ttc:{number of hours to the nearest 0.5}{newline}ttf:{number of hours to the nearest 0.5}. Very short and simple! There are just two fields: ttc is Time To Cause and ttf is Time To Fix. So, TTC is how long it took to work out exactly what the problem was, and TTF how long it took to fix. TTF may include things like writing a test to prevent regressions.

Of course, these things are not hard and fast. There’s little point setting a stopwatch - we’re not pretending that this is a precise science. That’s why we’re only going to a half hour level of precision. It it’s less than 15 minutes, then just put 0, it will all average out.

TTC and TTF are exclusive periods of time - so in this example, the engineer spent 2 hours finding and 1.5 hours fixing, for a total of 3.5 hours spent.

Collating the results

Now that your commit messages are being properly annotated, it’s time to extract the raw data. Fortunately, GitHub makes this very easy by allowing web hooks to be attached to a repository and receive commit data on every push. If you’ve ever tinkered with GitHub’s hooks system, you know the structure of that data looks something like this:

{
  "ref": "refs/heads/master",
  /* ... */
  "commits": [
    {
      "id": "33eb08ab524a6b8dc962aa3a4be62a313730b34f",
      "distinct": true,
      "Fix pagination bug\n\nPagination was causing a buffer..." // ...
      "timestamp": "2014-05-27T18:40:37-04:00",
      "author": {
        "name": "Nate Abele",
        "email": "nate@radify.io",
        "username": "nateabele"
      },
      /* ... */
    }
  ],
  /* ... */
  "repository": {
    /* ... */
  }
}

Once the hook is set up, it can iterate through each push’s commit data, pattern-matching on the message field, like so: /^TT[CF]:?\s*([0-9]*\.?[0-9]+?)$/img. Once parsed, the results may be stored in a database for later analysis.

Difficulties

While straightforward, this technique is not without some caveats. For example, if you’re working out an issue across disparate components in a multi-repository system, it may be difficult to analyse the problem in aggregate.

Further, critical bugs may require immediate fixes, even if only partial, with a final fix to be implemented at a later date. Other bugs, even after identifying the supposed root cause and adding regression tests accordingly, simply refuse to die.

Accurately measuring the impact of these issues, while possible, requires higher-level analysis, involving integration with, and careful tending of an issue-tracking system.

Wrapping up

I always get a bit suspicious of code metrics. Whilst you can have categories of bugs, every problem is very much it’s own thing. It’s vital to understand that any trends and patterns that you observe are in the broadest of strokes only — take every case on its own merits. Be aware that in comparing projects you may well be comparing apples and oranges. And, more critically, if you start using metrics as a tool to beat engineers over the head, then you fail management in the worst way!

Dire warnings against their abuse aside, metrics are the thing that stands between your team and "blind" development. “Gut feel” will only take you so far; if you don’t have any insight into what you’re doing and whether it’s being successful, how do you know if you’re improving?

Do you use metrics in your software projects? Have they been helpful? How do you measure technical debt?

Interfaces - the misunderstood concept

Wed, 28 May 2014 05:05:32 -0400

In a previous post, we talked about the existing relationship between type hinting and interfaces, and the fact that type hinting can lead to overuse of interfaces.

In this post, we're are going to explain where and when interfaces can be useful (outside of the type hinting concerns) in PHP.

In statically typed languages, interfaces are first used to make different classes interchangeable without breaking type safety. But this is not the property we are concerned about because PHP is a dynamically typed language so this issue doesn't apply (well expect if you are using type hinting).

But interfaces are also an alternative to multiple inheritance because, in general, multiple inheritance creates more problems than it solves (e.g the diamond problem). Moreover, when you are looking to multiple inheritance to solve a problem, most of the time the solution is to not use multiple inheritance at all! Indeed, you should always consider composition of features before you think about inheritance. For example, a Car doesn't need to extend Engine and Wheels classes! In this case, having a Car which contains an Engine instance and 4 Wheel instances makes a far more sense.

Languages like Java or C# preferred to use the interface concept instead of multiple inheritance and this is also the way chosen by PHP.

Are interfaces are "equivalent" to multiple inheritance ?

No. They're two different concepts, although both allow attaching "things" to classes, where "things" are:

multiple parent classes (for multiple inheritance)
contracts (for interfaces)

To simplify: an interface is a kind of type (like an integer, a float or whatever) but if it's too abstract for you, just see interfaces as a way to attach "contracts" to classes.

For example if a Car class implements a Vehicle interface, it only means that:

Car is assured to have implemented all Vehicle's methods
you can use instanceof to verify that an instance implements the Vehicle "contract"

What interfaces and multiple inheritance both solves ?

When you are writing business logic, you may need be able to manage some conditional behavior based on the type of an instance.

if ($instance instanceof Countable) {
    return $instance->count();
}

In the above example, if Countable is a class name it can be problematic. Indeed if $instance is an instance of a class A which extends a class B, $instance won't be able to also extend a Countable class. However if Countable is an interface, the problem disappear.

When should I use interfaces?

I'll explain this on piece of code found in the Zend Framework 2 (apologies to the author).

/**
 * Process the select part
 *
 * @param PlatformInterface $platform
 * @param DriverInterface $driver
 * @param ParameterContainer $parameterContainer
 * @return null|array
 */
protected function processSelect(PlatformInterface $platform, DriverInterface $driver = null, ParameterContainer $parameterContainer = null)
{
    ...
        $table = $this->table;
    ...
        // create quoted table name to use in columns processinga        if ($table instanceof TableIdentifier) {
            list($table, $schema) = $table->getTableAndSchema();
        }

        if ($table instanceof Select) {
            $table = '(' . $this->processSubselect($table, $platform, $driver, $parameterContainer) . ')';
        } else {
            $table = $platform->quoteIdentifier($table);
        }
    ...
}

People often use interfaces in method declarations (where they can be of little value), but miss the opportunity to use them in strategic places! In the above piece of code, all instanceof statements should have been applied on interfaces instead of a class name. Indeed, if your class already extends another parent class, you won't be able to use your class implementation with the above piece of code!

In conclusion, interfaces in dynamic languages should not be thrown around indiscriminately. The fact that "type hinting" seems to be a practice adopted by many frameworks and libraries tends to lead to interface abuse, and creates problems which never needed to exist!

So, if you want to make you life easier, just avoid type hinting and use interfaces only when they really add value!

PS: For those who are using some dependency injection system based on a DI container which introspects the code using the PHP reflection layer to extract the "type hint" of methods parameters to automatically inject instances of the correct type, I think simpler approaches need to be considered here.

Type Hinting in PHP - Good or Bad Practice?

Thu, 22 May 2014 03:18:46 -0400

PHP 5 introduced a way to write method declarations which has become somewhat en-vogue. You may have noticed the following syntax in a number of open source projects:

public function myFunction(SomeClass $instance)

This declaration forces the passed parameter to be an instance of SomeClass, and is called type hinting.

The fact that PHP only supports single inheritance can be problematic since with such type hinting, you won't be able to use your own class implementation if it already inherits from another parent class.

A solution to this issue is to apply type hinting on interfaces instead, like the following:

public function myfunction(SomeInterface $instance)

Now your class only needs to implements the required interface and you are good to go. However, there's still a drawback. If you want to override a type hinted method, you won't be able to change the type hint definition, which limits the overriding capabilities. Worst since polymorphism (a.k.a overloading) is not available for PHP things generaly leads to a proliferation of method names for a same task (e.g. stuffWithCollectionInterface()/stuffWithArray() instead of a unique stuff() method). And since array doesn't implement the Traversable interface, using array as a type hint is like type hinting using a class name instead of an interface.

But PHP is a dynamic language, so you can avoid all this issues by simply writing:

public function myfunction($instance)

Pretty confusing, right?

So what does that really mean?

The above example only demonstrates that instead of adding some flexibility to the code, type hinting:

forces developers to create interfaces for almost everything to keep a correct level of interoperability with other developers
limits overriding capabilities
tends to complexify API by introducing a lot of irrevelant method names

The main point to understand here is that type hinting is not really a feature like traits (PHP 5.4) or generators (PHP 5.5) but more like a property of statically-typed languages. Since PHP is a dynamic language, using type hinting makes it less "dynamic". So, to 'neutralize' the type hinting issue, interfaces are mandatory almost everywhere.

Why do so many people use type hinting in PHP?

I've no clear answer to this question but I guess it's probably the result of some Java developpers influence who tends to code in PHP like they did in Java. In Java for example some Dependency Injection techniques are based on code introspection. So the "type hint" is used for being able to automatically inject some instances of the correct type in the called method. But the fact that a technique works for a statically typed languages doesn't means it's the best option for a dynamically typed language too.

Some people using type hinting often argue that it enhances the "degree of confidence that your code works" because a method will only accept parameters which respect a "contract". So type hinting somehow produces some "better" code with "fewer" bugs.

This reminds me of an old post, where the author eventually found that static type checking was redundant with TDD. Please take a moment to read it. Personally, I had a similar life experience on statically-based language so I couldn't agree more with the conclusions of the author.

In my opinion, if you really want to deal with bugs, you'll need to test your application. Type hinting is at the same time limited, inadequate and insufficient for testing an application. And although some claim that both tests & types are required, I can't concur since I'm developing using dynamic languages for the past 10 years and I only encountered type issues a couple of times during this whole period.

Does that mean statically typed languages are bad?

Not at all, Static & Dynamic languages are both valid paradigms but you can't just mix random concepts from both lands and expect to obtain a better paradigm. Dynamic languages helped me to write significantly easier & less cluttered code, and enhanced my productivity. Adding type hinting at this step doesn't make sense for me since it doesn't :

exempt you from writing tests
produce better performance (like static types did in statically typed languages)
make your code easier to play with and share (it's quite the opposite)

I have absolutely no doubt that this affirmation will be largely disagreed with by many in the PHP community. AFAIK, PHP is the only dynamic language who introduced this concept. Ruby, Python or JavaScript just assume than when an object acts like a duck, quacks like a duck, then it's a duck. For the time being I didn't see yet any valid use case for which it would worth the effort of using type hinting in any of my projects. That's why duck typing was the option I choosed for PHP.

In a follow-up post, we will talk about interfaces and why to use them (outside the scope of making type hinting more flexible, of course).

Overachieving Software Projects

Fri, 16 May 2014 11:17:20 -0400

On software projects, we often have clearly defined deliverables. That’s very useful for staying focused, but it can make us a little narrow in our perspective. This post defines the concept of secondary deliverables as a tool for broadening our thinking, sharing knowledge, and raising the profile of an organization within their specific discipline.

What are secondary deliverables?

Secondary deliverables are artifacts produced in the course of your "main work" that can be made to be of value in a broader context – lessons learned from a specific task distilled into a generic, publishable form, or a workflow tool extracted from its original project. To formalise the term, a secondary deliverable is any content (documentation, code, workflow, etc.) that can be derived in a generic form from a specific task.

Secondary deliverables may include:

Blog posts
Podcasts
Video tutorials
Tweets
A talk given at a meetup or conference (which should be video recorded to maximise ROI)
Open source contributions

Let’s take a look at some examples.

Three tales of secondary deliverables

Tale 1: Developer Alex is working on a ticket for a client and writes an end-to-end (E2E) test to verify that the client’s acceptance criteria have been met. In doing so, Alex has to make several decisions about how often to reset the test database from fixtures, and when to roll on to the next test with the same data. In doing so, Alex is making a tradeoff between test execution time and isolation. Alex decides that this is worth doing a blog post about. Alex’s marketing department promote the post and, if the organisation is fortunate, it ends up being featured prominently on an aggregator such as Reddit or Hacker News.

Tale 2: Project manager Charlie trials running sprints from Wednesday to Tuesday rather than Monday to Friday. Charlie keeps a close eye on burndown and velocity and sees if any measurable gains are made. Charlie presents his finding at a local meetup and gets into a conversation with another project manager afterwards, comparing the strengths and weaknesses of various approaches. Charlie capitalises even further on this by recording and releasing a podcast with these PMs discussing the idea.

Tale 3: Operations engineer George has automated deployment of a system’s infrastructure on EC2 using Ansible. George thinks "hey, that wasn’t so hard as I thought" and produces a video tutorial for upload to Youtube. The organization feature the Youtube video on the company’s blog. The video gets spotted by a conference organiser, who invites George to give a talk.

These are just some examples of how adopting this mindset can work. Be creative!

Benefits of secondary deliverables

Enhanced knowledge base. It’s very tempting to solve a problem and simply move on, only to have the issue recur and not to be able to remember how you solved it. If you blogged about it, there’s a much stronger chance that you’ll be able to solve it quickly, and that if you’re not around, your colleagues at least know where to start! Blogging can help an organisation maintain a sense of progression.

Knowledge sharing. A private knowledgebase is one thing, but a public one is even better, as there is more impetus for it to be correct! The entire open source world is built on this ethos. More importantly, software development is a very young industry in the scheme of history, and best practices, such as they are, are in their infancy. Sharing discoveries allows us to coalesce on common patterns and ideas, and your contribution may trigger further insights in others who may be working on similar or parallel tracks.

Organisational perception. If you’ve solved a problem, or done something clever, why not shout about it? Let the world know that you’re smart people! Things like podcasts and tutorial videos can give an organisation an authoritative voice. Conversely, without blog content, technical articles and so forth, an organisation will struggle to position itself as thought leaders. If I’m looking at a tech company website, and there is no tech content on there, I am likely to dismiss that company pretty quickly!

Recruitment. A tech company with strong open source and blogging output, for example, is very attractive to many in the industry. Thought leaders like ThoughtWorks are considered desirable employers partly for this reason.

Greater variety. Frank Herbert may have called fear the mindkiller, but surely boredom is just as dangerous! Spending an afternoon or a week working on writing, blogging and video content derived from one’s primary tasks can be a fantastic way of recovering energy.

Reduction of siloization. Bearing secondary deliverables in mind as a team works on a project can encourage broader thinking. No disrespect intended, but if you leave your company’s profile solely to your marketing team, you can end up looking generic. Without technical blogging and technical content, how are you going to position your organisation in your industry? With secondary deliverables, you are actively supporting your marketing department’s efforts to show your organisation in a positive light. It’s taking your eyes off small, tight "KPIs" (which, in my opinion, are utterly poisonous) and giving an organisation-wide perspective.

Increased ROI. All of the benefits we’ve seen in this list can be summarised as Return On Investment - the secondary deliverables mindset is about getting the most you possibly can out of the hard work that you put in.

Risk mitigation

Secondary deliverables certainly aren’t "free" and do have some risks which should be considered. We’ll take a look at the top three.

Firstly, producing secondary deliverables inevitably exerts a drag on productivity. After all, if you’re producing a video, you’re likely not getting paid directly for it. I would contest, however, that you could consider this time to be coming out of the marketing and R&D budgets.

Secondly, secondary deliverables are the first things to go by the wayside when pressure is on as it’s difficult to justify their value in direct terms. When this happens, it can be frustrating to have a whole list of ideas and no time to implement them. This can be mitigated by planning in some slack time into everyone’s schedule. It won’t always work out, but if you start with some slack, then you stand a fighting chance!

Finally, there is the risk of accidentally leaking sensitive information. Much like code, all secondary deliverables should go through some form of peer review before being released into the wild!

Introducing secondary deliverables to your organisation

Although I’ve not really codified secondary deliverables before, on reflection, it’s long been part of my mindset, and that’s where it has to start. It is not about adding pressure to people to produce things outside of their remit: sometimes, an organisation can get hold of the concept of secondary deliverables and pressure its staff to produce more; that’s a total misrepresentation of what we’re advocating. A secondary deliverable should happen as a byproduct of somebody’s work, not as an additional pressure. Rather, it’s about taking time to maximise the benefits of the work that’s already being done rather than letting that value simply evaporate.

So, with that caution in place, what I recommend is to simply get people thinking about their roles more broadly. Lead by example, whether you are in a position of authority or not. In my experience, that is the most reliable way of effecting change. Start small, and be consistent. An internal podcast or team training lunch is a great place to start!

Wrapping up

The perception of your organisation is everyone’s responsibility, and adopting a mindset of considering "what can I take from this experience and how can I present it?" is a really good way of improving that. The examples I’ve talked about in this post certainly aren’t the only secondary deliverables; it’s much more about thinking “hey, this is a really interesting problem I’ve just solved, is there some way I can share that information with people in my organisation and in the wider community?”

I hope this post has given you some ideas. Why not get in touch and let us know what you think, or share some of your own ideas and post a link below?

Radify - Rapid Application Development

Using front-end build tools

Introduction

Let’s talk about workflow

Basic web development workflow

Creating an asset pipeline

Automated workflows with build tools

Available build tools

The build file

Tasks

Anatomy of a task

Creating your first build file

Explanation

Task clean - cleans out build directory

Task build - calls the subtasks

Task scripts - transforms javascript

Task styles - transforms CSS

Task images - compresses images

Task html - transforms HTML

Plugins

Automating your workflow

Task watch - automating your automation

Task serve - run a local server

Task refresh-browser: refresh your browser

What do we use to write these tasks?

Finished build file

What else can build tools do for us?

Linting

CSS preprocessing

Script preprocessing (transpiling)

Push to server

Best practice

Never edit anything in the build directory

Never let your build tool modify anything in the src directory

Short, sharp, composable tasks

Your normal workflow should be encapsulated by a single task

Have tasks that aren’t part of your main workflow as separate tasks

Don’t store credentials in plain text

Debugging with source maps

Being idiomatic

See also: Browserify

Go forth and build!

Further reading

Thanks

Download as PDF

Picking up the ball

Prevention of and elegant recovery from failure

1. Reduce the likelihood of failure

2. Reduce the cost of failure

3. Respond proactively to failure

Going public? Boundaries in response to failure

Is there an optimal failure rate?

Wrapping up

Rise of the machine users

Common mistakes - overpowered users

Better practise - machine users

Wrapping up

Aspect-Oriented Programming In PHP

Aspect-Oriented programming in PHP

A new kid on the block

The Filter API

Conclusion

Avoiding Bumbling Professionals

Get To Know Yourself

Do Your Own Research

Avoid Billing Conflicts

Be Sensitive to the other Warning Signs

Routing in PHP - a Complete Benchmark

Establishing a Benchmarking Process

The Results

Further Investigations

Conclusion

On Boundaries

On Boundaries

Comic book continuity and Git rebase

Retconning in comic books

Establishing canon

Another artist gets involved

Alex's second story (A2):

Bryce's first story (B1):

Task `clean` - cleans out `build` directory

Task `build` - calls the subtasks

Task `scripts` - transforms javascript

Task `styles` - transforms CSS

Task `images` - compresses images

Task `html` - transforms HTML

Task `watch` - automating your automation

Task `serve` - run a local server

Task `refresh-browser`: refresh your browser

Never edit anything in the `build` directory

Never let your build tool modify anything in the `src` directory

Configuring `kahlan-config.php`

Troubleshooting `node-lambda deploy`