Today I had a discussion with Michael Bolton on Twitter about software releases. As I felt that my point did not come across correctly in the condensed tweet format, I decided to write an article instead.
This is not about bashing Michael Bolton - in fact I respect and share many of his views a lot and enjoy reading about his thoughts. But this time, I did not agree at all.
The tweet that started it all was
What’s new in the latest version of your app? “Fixed some bugs” isn’t helpful; you might as well say “Things.” Plus you probably introduced some new bugs, didn’t you?
— Michael Bolton (@michaelbolton) May 7, 2020
How about fewer builds and more modest scope, so developers aren’t under such pressure. Fewer bugs that way.
I totally agree with the premise of this tweet. Whenever I see an app update on my phone and read the release notes that say nothing but "Bug fixes and performance improvements", I have the same feeling.
However, what I do not agree with is the implication that this comes from too many builds and pressured developers. And - more importantly - that the "release early, release often" (RERO) approach results in worse quality.
In my opinion, the more releases you have, the better the overall software quality becomes - both in terms of code quality and user experience.
Software is created by humans. This is not a bad thing since a lot of the resulting products are used by humans as well. What we tend to forget is that human beings are not perfect to begin with. Everyone involved in the software creation process (not only developers!) can (and will) introduce bugs. This is natural since the matter we are dealing with - software - is very complex and involves a lot of disciplines working together.
We can try to "automate away" all tasks that a real person might mess up and call it a day. And as a test automation engineer, I am pretty much for that.
Here, the devil lies in the details, though: today, we don't produce software that targets one specific hardware configuration. There can be infinite combinations of mobile devices, operating systems, web browsers, system settings, third party software and hardware combinations that may result in bugs for a fraction of the user base. These cannot be found by software tests, strict build and deploy pipelines and UAT environments. This is something that can only occur "in the wild".
Our goal should always be to produce and release high quality software that works well for our users. A lot of aspects of this can be ensured by rigorous testing, quality gates, highly automated build pipelines, life-like test environments, agile processes that don't stress product teams etc. What we as human beings cannot foresee is everything a user might try to do with our product. We are limited in our perception. I have seen highly trained QA professionals that have a sixth sense for these behaviours but there will always be cases that nobody thought possible.
Less frequent builds and deployments mean that
there is more to test.
More code is being written if you have more time. You could argue that this time could be spent on less features and making them bug-free. Still, I stand by statement that more code means more bugs - independent of how quality-centered the team is.
you have longer feedback loops.
The time and frequency of feedback within the team is longer if a release contains more code. Also the feedback loop from your users concerning the live version of your product is tackled later in the development. Users simply have to wait longer for a feature or a fix for their specific problem.
you might develop the wrong thing.
By developing minimum viable products, it is possible to get a fast market analysis of features that your user base needs. If you have longer release cycles that are spent on fully developed features that turn out to be unwanted.
bugs will remain in the product longer.
Even though releases should never be rushed so we don't risk introducing more severe bugs with a quick fix, having more frequent releases simply means that existing issues are gone faster.
We have to look at the big picture of software development. Quality needs to be ensured by everyone working on a software project and we should automate what makes sense and leave the most important tasks to human beings. "Release often" does not mean that we push garbage to our users but that we give everyone a chance to deliver exactly what is required in the least amount of time.
And this is an important aspect of quality.