Let's stop talking about quality
I don’t like discussions about quality in software.
Don’t get me wrong. I want to build software I can be proud of. I want to be part of teams that build great products. I aspire to craftsmanship. What I dislike is the word “quality”, and how it tends to polarise conversations.
Quality is subjective
A lot of factors go into software quality. Good software is fast. Good software is maintainable, readable, scalable, and well tested. Good software has attractive UI and intuitive interactions. Good software has no bugs, or at least no serious bugs, or at least no bugs that our customer support team can’t work around.
In practice, quality means whatever you want it to mean. To a fan of unit testing, quality means investing in unit testing. To a designer, quality means beautiful screens and careful interaction design. To a customer support manager, quality means all bugs are documented and the serious ones have an ETA.
If you tell people that your estimates are higher than expected because you want to do a quality job, some of them will think you’re spending time refactoring and writing tests, and some will think you’re going pixel-by-pixel in Photoshop. Some of them will end up disappointed.
Quality is boolean
Not only is quality subjective, it’s also impossible to quantify. It’s not clear how much time spent focussing on quality will stop those production issues, or how much less quality you’ll get if you do a rush job. Do you want your products to be 50% better designed, or will 30% do?
That makes it hard to track quality over time, but that isn’t the real problem. The problem is that quality leaves no room for discussion. Debates boil down to “do you want it done properly or not?”
Often this comes up in the form of a decision between quality and speed:
- “We’re a startup - we don’t have time for testing and code review, or we’ll miss the market window.”
- “We’ve been moving too fast and breaking too many things. We need to slow down and be more careful.”
If you frame it this way, you can choose speed or quality, but you can’t have both.
Quality is judgmental
It’s not generally possible to argue against quality. It’s a rhetorical word, loaded with positive connotations. That means discussions about improving quality tend to come across as criticism:
“I can’t do a quality job on this timeline.”
“I can do a bad job, but why would you want that? You must be wrong about when you need this.”
“I’d like us to invest in unit testing - it’s a great way to improve the quality of your code.”
“Our code is terrible right now.”
“We’re having too many production issues, so we need to focus on quality.”
“The team didn’t do a good enough job. You didn’t do a good enough job. The prod issues might be due to product management insisting on impossible requirements, or that reorg we had a month before launch, but can we all agree that a quality product wouldn’t have these problems?”
Quality creates factions
So quality is hard to argue against, hard to nail down, and it’s all or nothing. Not surprisingly, some people respond by rejecting the concept of quality as unhelpful. Quality is a luxury we don’t have time for; better to ship something than polish it forever.
What “we don’t have time for quality” really means is: “I’m not sure what you mean by quality, but I know moving fast is more important than what I mean by quality”. Unfortunately, all too often what gets said is “we don’t have time for quality”. Framed in these terms, there’s no common ground, and we take up sides, each frustrated at the other’s inability to see the big picture.
The pattern repeats at larger scales. An engineering team can split into “pragmatic hackers” (“we just get the job done”) and “serious engineers” (“gotta clean up the mess those cowboys make”). Or engineering can lose trust in the business (“don’t those suits understand how short-sighted they’re being?”) and vice versa (“ignore the nerds grumbling about technical debt, they’re always saying the sky is falling”).
What’s really going on here is a failure of communication. This talk of factions might sound dramatic, but I bet you’ve heard variants of the above at the watercooler or the bar. Every little joke and sarcastic aside serves to undermine trust and communication.
Common ground
The reality is that doing anything in the real world involves difficult decisions in the face of constraints. Everyone has more exposure to some of those constraints than others. The product manager sees customer feedback, feels the urgency of staying relevant in the market. The designer sees limited screen real estate, the need to engage the user quickly. The engineer sees maintenance costs and technical limitations. But you’re all working toward the same goal: ship products, grow the business, delight your customers.
The only way to do that is to communicate. These conversations often seem like a contest between one person’s viewpoint and another. But the best answer for your situation probably has elements of both! Instead of your diverse experiences getting in the way of a productive conversation, why not have them be the conversation?
Help other people understand the constraints you see, and you can work together to decide the best tradeoffs to make. Understand the constraints another person faces, and you can understand how to help them. Instead of negotiating, you can collaborate.
Empathy
Next time you’re frustrated by an unrealistic deadline or an inflated estimate, instead of assuming that person just doesn’t get it - ask how things look from where they are. What does a product manager do every day? What does it mean to an engineer that the code is a mess? What was the designer trying to achieve with that colour change?
Instead of talking about quality, let’s talk about goals. We want to see whether customers engage with this new feature. We want to cut down how often our engineers get paged at 3am. We want the main features to be at most 3 clicks away. We want to communicate the value of the product in the first 5 minutes. Let’s talk about how our different goals interact. Maybe there’s a way to do it that gets everyone closer to their goals?
Because that’s the fun part. That’s why we work in teams - bringing our different strengths together, solving problems, to build something awesome.
So let’s have less talk about quality, and more talk about empathy. Go talk to someone. Learn what makes them tick. And make something cool together.
Update: This post is the first in a series. If you found it interesting or provocative, you may enjoy the subsequent posts:
- Proposing a better way of framing these conversations: Let's talk about confidence.
- An engineering philosophy to get away from false choices between quality and speed: Move fast with confidence.
Thanks to Paul Biggar, Peter van Hardenberg and Dorothy Li for reviewing a draft of this post.
Photo credits: Thomas Hawk ("quality" wall painting), saeru ("quality workmanship" sign, cropped from original), Jeff Eaton (Lego argument), Michael Summers (discussion with notebook), and the Anita Borg Institute (people collaborating around a laptop).
Comments (archived)
Comments are disabled, but please feel free to reach out on social media if you'd like to discuss this post!
Comments from a previous URL for this post (which now redirects here):
Interesting article. I studied "traditional" engineering and then spent 10 years in software engineering, most notably at Airbus. I have always been very interested in the differences between the fields, and which methods can be directly applied and which cannot. But in this instance I wonder if a better solution is to define quality and then measure it, just like traditional engineering and manufacturing does. (As do Google et al, incidentally).
– David at Thu, 25 Feb 2016 17:53:00 +0000
Totally agree - we arse sill having this quality/time discussion based off gut instincts because we don't measure or analyse.
– Merrion (Duncan Jone at Thu, 25 Feb 2016 19:56:12 +0000
Sure, the term quality is subjective like most things. However, to me (and maybe other's will agree) that quality in software means writing maintainable, readable, and documented code. Unit testing is usually involved in order to achieve that. From experience, writing tests from the beginning has proven to increase "quality" of software, in terms of time spent on fixing bugs is reduced. Good article but quality is definitely measurable.
– Miguel Mota at Thu, 25 Feb 2016 17:57:55 +0000
First, maintable means having code that can be read and understood, whether it has documentation or not. In fact you only list those because you feel they would be necessary for a developer to "maintain" a code base. So a bit redundant.
But more importantly, how do you know that writing tests from the beginning increased the quality of code? Did you try creating the same product with the same team members both ways?
Don't get me wrong, I like unit testing but it's a bit presumptuous to say it improved the quality of your code. It's also incorrect to say that it reduces the time spent fixing bugs. The tests exist to be broken, and when they do you have to fix your code. Just because software is telling you to fix your bug, rather than qa or customer support, doesn't mean you're not spending time to fix it.
I'd love to see how a pro unit testing coder would handle all the OCR and Bluetooth device integrations I work on for a living. Good luck mocking those interactions. <.>
– coder.slynk at Fri, 26 Feb 2016 12:27:02 +0000
When you say that "test exist to be broken", you are talking about regression bugs? But in that case, your CI server is giving you a chance to fix them when they are cheapest to fix: long before shipping the new version.
But, fact is, if you tested a certain method "properly" (where "properly" is rather hard to define without lots of experience), it shouldn't have bugs, which basically gives a circular definition of "properly" testing. It also forces you to write as pure (stateless) code as possible, because it's the easiest to prove correct. It doesn't mean your project will be free of bugs, it just means that sneaky bug you will be looking for will rarely turn out to be a trivial typo in a function which doesn't actually do what it should. Again, in most of the cases.
My experience is the same: my bugs always used to come up in places where I didn't give enough (sometimes any) testing coverage.
– Groox at Sat, 27 Feb 2016 07:55:35 +0000
Work on a google car, a space shuttle, a medical equipment, a network,device, an automated financial bot buyer, an encryption algorithm, a pacemaker chip, a network stack, a storage system, a privacy mechanism, a robot... anything that is really ground-breaking and game changer has to have more than just good business trade-offs, it has to be hard to execute. Quality is there when this needs to happen.
– Bat Manson at Thu, 25 Feb 2016 18:26:54 +0000
Sometimes hard things just produce bad business trade-offs, that's why no one wants to do them and that's why they're hard.
– Luiz at Fri, 26 Feb 2016 17:56:14 +0000
I agree with David. It seems to me as if you are writing about quality as a generic and/or abstract concept. In my experience, many of the symptoms you are describing tend to disappear when you start defining relevant quality attributes and expected levels of performance with regard to each attribute.
– Nils Wloka at Thu, 25 Feb 2016 18:33:34 +0000
I use the definition of quality being "meeting ones requirements"; you know, fulfilling your spec.
This definition includes exceeding ones requirements as *not* achieving a quality result.
If you cannot agree on your requirements and spec then you of course cannot achieve a quality result either.
– Paddy3118 at Thu, 25 Feb 2016 20:52:38 +0000
awesome Article really worthwhile and Fruity :) but it depends on the situation to situation Quality itself is having its place in IT industry if it not like that then the QA team must be deleted :)
– Danish at Fri, 26 Feb 2016 07:08:56 +0000
In 1992, I attended a company sponsored course on quality. As per the course, the definition of quality was, "Conformance to requirement." A typical example given was, Hyundai Excel (First Hyundai model sold in the U.S.) was a quality car because for mere $7,000 you got a car for people who put more value to features for money spent.
So, even though at its face value, this definition sounds almost ridiculous, but the last section in this article, somewhat alludes that all external constraints aside, a quality software boils down to conformance to requirement.
– Ash at Fri, 26 Feb 2016 14:14:10 +0000
Love it; and I also love the title of your blog. Good stuff.
– BW at Fri, 26 Feb 2016 14:33:02 +0000
It's true there is no standard industry definition of 'quality' (although ISO and other organizations have put forth sets of definitions), there are quality perspectives and what you have described in your article is basically the product perspective. There's also the manufacturing perspective (e.g. 'conformance to spec'), the user perspective (e.g. 'fitness for purpose'), value-based (e.g. 'you get what you pay for'), and (my favorite) the transcendental perspective (e.g. 'I know it when I see it'). Like any engineering field, being on the same page with terms is essential, but I don't think it means anything we want it to mean. Quality is a complex, multi-faceted concept, and like any requirement, it's important everyone is in agreement with the meaning of the term.
– Corey Brand at Fri, 26 Feb 2016 17:26:12 +0000
It is not logical to think that a one-word specification like "Quality" can describe a stakeholder's needs for a product. Might as well require it to be "Cool". It *is* reasonable to talk about attributes of quality such as defect density or severity, degree of conformance, etc.
Not all aspects of quality are important to all stakeholders. For instance, maintainability may not be important in a product where development stops with initial release. This is a place where developers often trip up; arguing for something that the business stakeholders find unimportant. It is also hard to quantify the cost of poor maintainability, which makes it hard to argue for.
Quality is a continuous variable. Arguing as if it was a binary value is a waste of time, since each participant will pick a value that supports their viewpoint. A more nuanced argument, and one more likely to get by the business stakeholders, would for instance be "Customers are overwhelming our support team with bug reports. We either need to hire X more support staff at cost $C1, or improve our testing, at a cost of $C2 to reduce this load on support, or accept that we can't provide support for this product at our traditional level."
Trading off quality for speed is a false choice. One must only make this choice if the development process is chaotic and lacking in feedback. Improving discipline and feedback in the development process increases both quality *and* speed of delivery. It's sad to say, but most dev processes are chaotic and low on feedback, no matter their claims to be agile.
– Kurt Guntheroth at Fri, 26 Feb 2016 19:51:50 +0000
Funny how this reminded me of "Zen and the art of motorcycle maintenance" about trying to define quality, albeit with a different interpretation. https://en.wikipedia.org/wi...
– FGM at Sat, 27 Feb 2016 17:37:06 +0000