Skip to main content

Stability

Written Jul 6, 2023, tagged under post,work,software,quality,python,mastodon.xyz

Today I've been thinking about stability. I made a change to some software back in January, and last month a problem came up for users that can be traced back to my earlier alteration. Mistakes happen, but I am thinking about how to improve the odds of stability.

In a slightly different context - security - I have heard about the "swiss cheese" model. We set up layers of protections, but each has a hole in it. If we are unlucky, the holes line up and let bad things get where they shouldn't.

The same actually applies in software. We set up layers of protection: IDE configuration and error highlighting, code quality gates, linting, automated tests, end-to-end smoke tests and so on. The hope is that potential problems will be flagged up by one of these layers. My case managed to find an area where all the holes were aligned, though, which raises the question of what extra layers can be added.

In decreasing order of effectiveness, the layers that could have helped most were:

  • Better existing test coverage around the area in which I made the change. There are lots of tests, but not of the exact area of functionality affected
  • Avoiding unnecessary changes mixed in with other things. My change was 95% fine, but I had refactored something for performance, and that was where the bug came in
  • Before making the change, scrutinising the area and adding missing tests before actually starting work
  • Some better variable/function naming and perhaps some doc strings, to increase the odds of me seeing the change in usage and picking up on the potential issue

These are things I'm going to ponder, as part of writing up what happened. Problems happen, but it's how we respond to them that is most important.

PS in other news, welcome back mastodon.xyz! Looks like the domain name was knocked out due to someone reporting inappropriate content to the registrar, who didn't check with the system administrator before making the name go dark.

Photo by Chun Kit Soo on Unsplash