On Process

A lot of people think ‘process’ is a dirty word. Process is a form of bureaucracy, they say. It stifles creativity. Process is rigid and limits our ability to adapt to changing circumstances. No one likes it.

I’m a pretty big fan of process.

What is ‘process’? #

Let’s start by defining what I mean.

A process is a set of steps to be followed by one or more people, triggered by conditions. These steps and conditions are written down, and available to the people who must follow the process.

In this post, I’m assuming that this is in the form of documentation. However, often the best way to write down process is to embed it into the tools and software used by your organization. Such software still normally needs to be supplemented by written process, at least to tell you which tools to use when.

Why write down process? #

What’s the point of this? Surely if we hire smart people and give them a goal they can pursue it in whatever way they think best.

Lots of reasons.

Avoid reinventing the wheel
Speed and confidence
Scaling
Coordination and collaboration
Empowering individuals and teams

Avoid reinventing the wheel #

There are a tonne of mundane things about a working environment that don’t need to be reinvented and rethought every time. You shouldn’t have to poll every person in the company to figure out how to request annual leave, there should just be a process you can follow.

Even for more complex tasks like promoting a person, it really helps to have a process for the more mundane things — who approves? what material must be submitted? when can it happen? who is informed on success / rejection? — so the interested parties can focus on being good enough at their actual jobs to get promoted. In this case, process doesn’t deny individual autonomy and judgement, it enables them.

Speed and confidence #

Some things need to be done quickly and with confidence. If someone has just rolled out a bad release, you need to know how to roll it back, who should do it, and whether they should at all.

Whether it’s written down as a document or written down in code, process makes this possible.

Scaling #

Written process is a great enabler of delegation, and therefore of scaling your organization. If you have a howto for running retrospectives, anyone can run retrospectives.

Coordination and collaboration #

Written process likewise enables interactions across teams. If you have an established team then probably everyone mostly knows how things are done, or at least enough to muddle through if necessary. But people from other teams, or who are new to your team, won’t know what’s expected of them or what to expect of you. Managing expectations is kind of a big deal, and anything you can do to make that less onerous is a win. Having clear, written patterns for interaction can avoid delays, uncertainty, increased effort spent on coordination, dropped tasks, and reduce the need for oversight.

Or, to misquote Alfred North-Whitehead:

Your organization advances by extending the number of important operations which it can perform without talking about them.

Empowering individuals and teams #

If you really award people autonomy, you must grant that one of the ways they might exercise their autonomy is to write down some process.

But also, as outlined above, if you make it possible for people to figure out what to do without first asking for permission or talking to someone, then they will attempt more things and in a more coherent direction than they would otherwise.

You might say that it gives people agency.

And, although it’s too early for me to say this with confidence, the same thing that grants people agency might also make it easier to make useful AI agents. Certainly, the LLM needs something as a prompt, and it might as well be a clear, written version of you do things around here.

Why not process? #

Process sucks when it rules you. It should only ever be a tool (one of many!) in service of teams defining and achieving goals. We are all familiar with times when it has become the opposite, when it hinders the actual work or stifles innovation. Many react to this by dismissing process altogether.

How do we avoid this overmighty process? How do we reliably tell when we are being ruled by process, rather than it being our tool? I don’t know, but it’s something I definitely want to think about.

How to do process well #

This is my favourite way to do process.

Write down what you actually do #

If you see that there’s a thing you are doing regularly, or a set of steps that you refer to by name, write it down

For example, “run a retro”, “announce the release”, “roll back a dodgy release”.

At this stage, you just want to write down what you actually do. This document functions as a how-to, to use the Diátaxis documentation system.

Writing down the steps will almost certainly prompt a whole bunch of good ideas about how to improve the process. This is part of the miracle of writing and one of the big reasons to write it down. Make a note of all of these good ideas, but don’t try to improve the process yet. We’ll get to that bit soon.

This is a lot like abstraction in code. If you see enough repetition, you extract a function and give it a name. If there’s a concept that keeps coming up, you make sure there’s a class or function or type or whatever that represents it.

Follow the process #

Use the doc you have just written to actually do the thing. This tests the doc, and provides a bunch of low-hanging fruit for improving in the quality of the doc. If possible, get someone who is not you, or even from outside your team to do this.

The document isn’t done until it’s been used once.

Suggest an improvement #

What’s something we can do to improve the process? Perhaps a tonne of steps could be eliminated? Perhaps it would be useful to add an extra check? Perhaps there’s a hand-off that just doesn’t need to happen?

Pick one of these, and write it down as an experimental process change.

Decide how to measure the improvement #

For some processes, there are obvious measures. For example, the Four Key Measures from Accelerate are pretty good ones for software.

For other processes, you might just want to take a straw poll of people’s feelings before and after.

However you decide to measure, decide before you try the improvement and write it down.

Try the improvement #

Set a fixed time window for trying the improvement. Update the doc with the improvement and note that it’s experimental.

Try it out for the time window.

Evaluate the improvement #

Did it work? What did we learn?

It is very useful to actually talk about this. Even though documentation is great, ultimately learning mostly happens in people’s heads.

Incorporate or abandon the improvement #

If it worked, make it part of the process. If it didn’t, chuck it away, and if possible write it down in a log of things you have tried.

Repeat #

Keep doing this, and your ways of working will get gradually better and better. To an extent, these improvements will compound, like pushing a flywheel.

Diagram #

Here’s a diagram of the above.

flowchart TD A[Write down what you actually do] -->|Creates a how-to document| B[Follow the process] B -->|Tests the document| C[Suggest an improvement] C -->|Pick one improvement| D[Decide how to measure the improvement] D -->|Set metrics before trying| E[Try the improvement] E -->|Set a fixed time window| F[Evaluate the improvement] F -->|Did it work?| G{Decision} G -->|Yes, it worked| H[Incorporate the improvement] G -->|No, didn't work| I[Abandon the improvement] H --> J[Repeat the cycle] I -->|Document what was tried| J J --> C class A,B,C,D,E,F,H,I process; class G decision; class J repeat;

Important note for managers #

Everyone on your team should be able to suggest improvements, and the decision about whether to keep an improvement should be informed by the metric. If you are the only one doing this, you severely limit the aggregated intelligence available for improving your organization. Your primary job is not to suggest concrete improvements, but instead to guard the meta-process above, to nurture an environment where everyone feels good about contributing, and to provide clear goals and values so your team knows what ‘good’ looks like.

This way of doing process is called “continuous improvement” or “kaizen”.