Burning definitions

Kristian Glass has recently written a few posts that add some much needed nuance to commonly used terms. I thought it would be good to reshare them, as well as the thoughts they provoked in me.

Data informed, not data driven #

You make the decisions, not the data.

Kristian suggests that using “data informed” can clarify that we are accountable for our own decisions, and are not just blind automata following the data. I think this is an excellent point.

Teasing it out a bit further, if we lean too heavily into being data driven, we can forget that we are almost always exercising our judgement based on our own particular view of the world. There is rarely a course of action that is the objective best.

We can also forget that it’s possible to forge a path outside of the metrics¹. l love looking at data and making things better using that data, creativity, experimentation, and iteration, but almost all of the time it’s a hill-climbing approach to optimization. The data is rarely going to “tell” you to break out of that.

Of course, this isn’t what people need to hear a lot of the time. Often, we need to be reminded that before we trust our gut, we should do a quick check against observed reality to see if our instincts are justified.

Mentoring vs coaching #

Lots of people have made this distinction, but it’s worth calling out again. Mentoring is helping someone with your greater domain experience, coaching is helping someone solve their own problems through reflection.

I think there’s a pretty heavy Venn diagram between the two. If I’m mentoring someone, I’m going to do a lot of Socratic questioning², and if I’m coaching someone in an area where I have expertise, I’m likely to give advice. I don’t really see a problem with this, as long as both parties are fine with it and the ontology police don’t know where you live.

Measure vs assess #

This one is new to me, and rhymes well with “data informed, not data driven”. Essentially, the return type of ‘measure’ is an objective value, and the return type of ‘assess’ is a subjective judgement.

Incident report, not post-mortem #

The idea here is that post-mortem should be reserved for when there are actual cadavers, and not doing so both undermines the seriousness of situations that are a matter of life and death, as well as granting our own industry an undeserved self-importance.

In honesty, I don’t have a felt sense of this, and even though I intellectually agree with Kristian’s point, I will often find myself slipping into using “post-mortem” out of habit and a fondness for gallows humour. That said, when there are terms that are more accurate, more precise, and less likely to upset people, we should probably do the work to switch to them.

I think there’s a more pragmatic point here too. People often hesitate to write an incident report because they don’t believe the inciting incident is serious enough to warrant one. When this has been me, it’s mostly been because I’ve had a lot of work to do and have wanted any excuse to avoid taking on yet another task. However, I don’t think the language helps. If I’m asking if something is serious enough to be post-mortem worthy, I’m asking an almost moral question about whether the bad thing that just happened is akin to the work of the great enemy of humanity. Maybe it would be more useful to ask whether writing up this incident would help improve systematic problems that are currently important to our organization. That is, the more neutral language might help frame things in terms of the concrete and real circumstances, rather than in abstractions.

Certainly, if an engineering leader can decide which things are worthy of an incident report, and importantly, which incident reports they actually read and follow up on, then they have a powerful lever for improving their teams. For example, you could decide that every failed rollout needs an incident report, even if there was zero downtime, as a way of driving improvements to the deployment process. That way, you and the team can see the systematic issues, and the team has the tacit permission to actually fix them.³ Using “incident report” instead of “post-mortem” makes the possibility of this play a little more obvious.

Incidentally⁴, check out Kristian’s extensive notes on incidents.