The Problem with Bug Reports

There is an interesting thing I’ve seen. It keeps coming back from time to time. There are ideas around “reporting” that people latch onto that won’t go away.

Organizing information into an easily digestible form is important for any organization. Making sure it can be presented in a way the target audience will understand is the point of any report. If it isn’t understandable, I don’t really think it is a “report” and more a pile of stuff that someone hopes mean something.

To be clear, there are some top-flight tools (Ranorex leaps to mind) that help you make good, concise reports. My concern is not here, but something else.

I am not sure if when finding something that looks “odd” then “create a bug report” and file it and do something else is the best idea. For people interested in “lean” – this might look a lot like “waste.” For others, it looks like a missed learning and understanding opportunity.

What do I mean?

When I am testing a piece of software not familiar to me, I tend to ask a lot of questions. If I see the behavior I don’t understand, I will ask about it. When I see something that looks like it is generally doing the right thing but has something that feels “off,” I ask about it.

In some organizations, these all should be recorded as “bugs” or “incidents” or “tickets” in whatever system is used for tracking. The reasoning is something like “We need to not distract developers from the important work they are doing.”

Let’s Talk About Work

There are many organizations that focus on making sure testers write good, well-formatted bug reports. I agree. It is important to have information presented as clearly as possible for anyone reading the bug report to understand why it is a problem.

I have found that many times developers will finish their “piece” and hand the code off to the testers to test and move on to something else.

I think this is really the second or third in a series of breakdowns. At least, they are problems with communication and team alignment.

I know from my own time developing code that the longer it is since I worked on a function or feature, the longer it will take me to provide an accurate answer to a question or problem.

I have seen this happen with developers I worked with. I am testing a piece of software they worked on. They finished their work two or three (or more) weeks before and have moved on to at least one more project.

My concern is the more this happens the more distracted the developers become. They don’t have the ability to focus on the task they are supposed to be working on, developing. Instead, they are being asked about and investigating code they wrote weeks before. In my experience, when this happens the chances of bugs being introduced in their current work go up.

In development shops that are sub-optimized, the developer often is instructed to work off the bug report itself. If they don’t understand a problem, they are to ask a question as a comment in the report/ticket and send it back to the tester.

We then get a vicious cycle of notes and messages passed back and forth. In theory, it is less of an interruption than an IM conversation or walking over to the developer or tester’s workspace. I understand that time zones and distributed teams can play havoc with the in-person conversation. Still, there are likely more effective means of examining a problem, or potential problem, than messages back and forth in a ticket.

Failure to Communicate

Many departments and even individual teams have no effective communication between developers writing production code and people testing it. There is little to no alignment on the understanding of what the code is intended to do, let alone the business or customer problem it is intended to solve.

The reliance on “fully documented requirements” or “well-crafted user stories” is a contradiction. Anything relying on human language is prone to ambiguity and misunderstanding. I have learned that any time there is the slightest chance of multiple interpretations of a written statement, that is precisely what will happen.

Written documentation, user stories, use cases and every other thing that is written can aid conversation. They can be used to facilitate conversation and record decisions and agreements.

To answer a question with “It is in the documentation, read it” is something less than helpful. It will lead to misunderstanding and confusion at the least. At the worst, statements like that will only hinder any attempt at understanding.

I have learned the way the team interacts amongst themselves will go a long way toward predicting how well the software will behave. In some “teams,” people work on their own projects with little interaction with team members. For these teams, I have found the likelihood of testing finding significant problems goes up.

A team works together, shares notes and advice, tends to have fewer significant defects found in testing.

Of course, that does not always hold up for every project, sprint, or release. I have found how developers interact to be a reasonable predictor for how they will interact with people testing their code.

The better the relationships between everyone on the team, the better the communication.

By having a conversation about the defect, the developer can get better information about what happened. With better information, there will be fewer exchanges when the bug gets fixed. It will not matter if the bug is fixed right then, or sometime in the future.

Working Teams and Teamwork

Each team is different. The needs, expectations and goals for each team are different. Most find coming together as a team and not remaining a collection of individuals who have the same supervisor a challenge.

Developers working side by side can do this easily. They have common challenges and questions, often similar problems. Even if developers don’t encounter them at the same time, obstacles can help a group of developers bond as a team.

In many respects, those common challenges help build the team as they learn to respect each other’s work and insights. It is usually harder for testers to form similar bonds.

Respecting the work done by the developers is a start. You can do that by not proclaiming “this doesn’t work” and dropping the bug report off. You can earn the respect of the developers you work with by providing insight into how you tested the feature. By explaining how you approach the problem you can then ask clarifying questions about the application.

When you have done that, you can highlight the behavior that appears to be incorrect. It is possible it is a problem. It is also possible there is a difference in understanding how the application is supposed to work. By explaining your understanding, you can invite the developers to share their understanding.

The result might be a bug. It might also be a problem in unclear expectations across the team.

Always encourage the team. Avoid placing blame or making definite statements without explaining your work and how you got there. Whenever possible, gain insight and agreement that the problem is real before writing the bug report.

Filing a bug report based on false presumptions or bad understanding can break down the team’s effectiveness. It will also often lower you in the eyes of the team you need to work well with to be an effective tester.

This breakdown in respect and effectiveness is the problem with bug reports.