Enhancement: cognitive load and stress

[jdmarshall]

I do a lot of things that are operational that my coworkers don’t understand until we are deep in the shit, and then you can see the light bulbs going off over heads.

Chronic cortisol is documented to cause long term cognitive declines but adrenaline also affects people’s ability to reason it pushes you into D Kahneman‘s “Type 1” thinking. This is why we do fire and tornado drills. Why we have safety training. We want to drill these life saving activities into muscle memory because there’s no time to think clearly when the building is full of smoke. Practice saves lives.

Emergency preparedness and safety gear is all about reducing the number of things you have to juggle in short term memory. But that also applies in the “War Room”. Customers are angry. The CEO and CTO keep interrupting to find out how long it’ll take. The SLAs have wooshed past and now we are in damage control.

Kernighan’s Law gets even tighter when the stakes have escalated. Write code as if you will be debugging it at 2:00 am because sooner or later someone will be.

[zakirullin]

I like your line of thought!

Kernighan’s Law gets even tighter when the stakes have escalated. Write code as if you will be debugging it at 2:00 am because sooner or later someone will be.

This is true! During the incidents the ability to think is shrinking drastically. Under such pressure, people tend to make things even worse by tweaking complex parts of the system that suddenly affect something else (the unknown unknowns). At this state I would much prefer to work with the simplest dumb code possible, and with all the effects of my change clear to me (from comments or code itself). Comments that give "hidden knowledge" make a world of difference in such situations.

Example of such hidden knowledge:
// After deploy, it takes some time for data to catchup due to geo-replication, so don't be afraid of stale output for the first ~5 minutes

// Before deploying the consumer with breaking changes, please drain out the old events completely not to cause infinite retry loops

We can edit the article and add remark about the effects of cortisol and adrenalin on Kernighan’s Law ("examples" chapter)!

This kind of state ("type 1" thinking + induced stress) is a very good litmus test for the system. I was wondering, if we can trigger this check intentionally somehow (in a psychologically safe way though). Maybe some sort of a game, where teams compete and there is a time pressure (not "chaos monkey" though). 🤔

[jdmarshall]

With your examples, I like to put those straight into the runbooks.

Eventually you want any senior employee to be able to handle these things, and give them enough info to know when they need to pull someone out of a meeting or wake someone up.