Let’s rewind a bit back in time and place to when Kipo decided to be a good tester and raise whatever defects he sees in the application. But wait a second, he is still a rookie tester and doesn’t know how to write quality bugs to make the world a better place.
Be not afraid, dear Kipo. After graduating from this tutorial, you will be able to impress everyone just by writing bugs. Is your boss being mean to you? Does your crush see you as a friend? Does your family think you are a failure?
With the tips on this chapter, you will just show the bugs you have written, and they will all bend the knee in front of you!
Breaking down a bug
When we are raising the bugs, we do not write the reports as a whole essay but fill out the required fields.
ID: Bugs are stored in the databases of the defect tracking tools like Jira, so they need to have a unique identifier to be tracked. We usually do not manually set the ID as they are generated automatically by those tools.
Status: It can be open, closed, in progress, etc. We have discussed different status a bug can have in the Bugs 101 tutorial. Go check it out, if you haven’t already. GO!
Current Owner: Can be tester, lead, developer, and whoever position that relates to QA. Also covered in detail in the Bugs 101. Are you still here?
Creator: The tester who wrote the report is known as the “creator” of the bug. This field is also automatically set in most cases when we write the bugs using our accounts in the tracking software.
Knowing the raiser of the defect helps the developers or the other team members to easily contact with the person in case they have any concern about the bug. It is also useful to choose the worst tester of the month by the number of bugs they have entered, so that we can punish the tester by making them sit on the corner and think about how pitiful they are for not entering any bug.
Side note: All jokes aside, some game companies have that policy to let go the play testers (who work on-call basis) if they do not enter the set number of bugs. Brutal, yet true. (Source: Anecdotal info)
Type of Issue: Depending on the project, it is essential to understand what type of bug we are reporting (UI, functionality, performance, etc. We will see various different types of bugs in the following chapter).
Area: The module/section in the application that the defect occurs. Especially if there are multiple development teams working on different parts of the application, knowing the area helps to examine which team to assign the bug for the fix. In addition to that, when we do a bug analysis in the end, we can see which areas are most affected, and should be improved more.
Title: Very brief information about the bug. It should give answer those three questions: “What is the issue?”, “Under what circumstance does it happen” and “Where in the application does it occur?”
Description: Similar to the title, but a bit more elaborate explanation on what is happening. Although it is more detailed than the title, we should still respect the developers’ time and keep it as concise as possible. Nobody wants to read an article on a simple bug, eh?
For example, if a title is “Login button at the sign-in page does nothing after entering the credentials”, the description might say “After entering the credentials and clicking on the login button to sign into the application in the sign-in page, nothing happens”.
Reproduction Steps: Each and every step to see the bug. Unlike the description and the result fields, this is where we can use our energy to be as detailed as possible. It is usual to see a bug is refused due to the developers cannot reproduce the bug.
Actual Result: What are we experiencing after executing the reproduction steps.
Expected Result: What should we be experiencing after executing the reproduction steps.
Severity: How much does the defect impact the business. Severity levels are set prior to the testing, and we as testers only have to pick one of them fitting most the bug.
Priority: When the bug should be fixed. Similar to the severity, there are several different options to choose from.
We are not giving many details for priority and severity because we will have a whole section dedicated to them as they are often confused, and people have hard time judging what level would suit the most to their reports. We gotchu.
Reproduction Rate: Sometimes bugs do not occur every time the reproduction steps are followed. To cover these scenarios, we set this rate.
For instance, when we enter a specific area in the game, there is an out of nowhere popup is displayed covering the screen and does not let us proceed the game. We now have the idea that it might be a bug, but to make sure we reset the game and proceed to there again. Hmm, interesting. The popup is not there anymore. But we should not ignore it thinking “I guess that was a one-time thing”. We should patiently (and in pain) try to reproduce the popup over and over again (10 times or more) to see if it reappears. Let’s say that it appeared exactly in 5 of our attempts. In this case, we know that it is definitely a bug, and we set the reproduction rate as 50%.
Hardware/Environment: Usually, in a QA team, testers will deliberately have different devices, or different environments (e.g., browsers). This adds more variety to testing since issues may also be device related.
Build Number: As we know, we are likely not testing the end product, and do the testing by the builds. It is important to include the build number for tracking purposes.
Labels: In larger projects where there are thousands of bugs in the databases, it is not easy to search the bug reports, and adding preset labels make the searching or filtering process faster and easier.
Attachments: We always need to include a visual representation of the bug either as a screen capture or as a video record.
It is not because developers think we are inferior lifeforms that need to prove everything we do, but because to provide a reference for the issue if they are not clear from the description. Sometimes they don’t even look at the description and can understand the issue merely from the attachments and that also makes the workflow go faster.
Additional Notes: This is where we can finally scold the developers and tell them they suck at their job. Again, and again do not!!! But any additional information or idea that we want to tell them is added here, such as the prediction of the reason of the bug, and suggestions on how to fix it.
We will keep our promise, and now investigate the confusing severity and priority fields in depth.
Severity vs Priority
Severity is on his corner, has been playing defensive, waiting for his opponent, Priority, to make a mistake. Priority suddenly attempts a sloppy right hook and loses his form, and bam! Priority finds himself on the ground, all his life is sliding across his eyes, and in the background, the victorious smile of Severity…
Okay, we left the author in his imaginary box match, and now we can talk about some serious stuff.
Severity and priority are arguably the topics that the rookie testers struggle the most. Even more experienced folks may do misjudgments on setting the levels for them. But we are here, and we will make it crystal clear for you all.
Severity
Severity describes how bad the issue affects the business. Since the QA team is fully aware of the requirements and functional specifications, it is always our responsibility to assess the severity. Following are the types of severities a bug can have in most projects.
Critical
When a bug is critical, the entire life of the system is in danger. These and major bugs are also called “blockers” as they “block” us from testing a certain feature. As an example, if the application shuts down when we click on a button, it is a critical defect. Underlying issue is usually in the code or in the backend for these bugs.
Major
Not the entire system shuts down, yet there is an issue that affects a huge part of the application, perhaps a whole module. For instance, in a website, the login functionality does not work at all and even though the system is partially working, we cannot proceed to the parts that a member of the website can use due to the defect.
Moderate
Now we get down to the level of more humble ones and starting with moderate or normal bugs. When a bug definitely affects the requirements but does not block us from testing or severely affects the functionality, it is a moderate bug. For example, if there are multiple ways to proceed to the login page, say an icon and a tab, but one of them does not redirect us to the login page, there we have a moderate bug.
Minor: When the functionality is not affected, and we are unsure about the impact of the issue, we often set it as minor. These defects are usually related to the type of testing we are doing. Like, if our main scope is verifying the functionality, but we are seeing cosmetic or spelling issues on the UI, we can call them minor bugs.
Peeps, this is important to notice that we cannot use minor bugs and cosmetic issues interchangeably. If the QA team is hired to perform a UI testing, then different interface problems can even fall into one of the categories above.
Proposed: This severity may not exist in a lot of projects, but sometimes, especially for the ones who need subject-matter expertise from the testers. To illustrate, if the QA team is made up of bunch UX design chads, they may be given the right to make suggestions on the features, instead of merely reporting the defects.
Priority
What is priority then? Shouldn’t correlate with the severity? Well, maybe. But hold your horses, because it is not that simple.
Yes, usually the critical issues have the highest priority, but an issue can have a high severity, yet a low priority as well. It may sound strange, but it will make a lot of sense when we explain how it can be possible later on. Now, let’s see the priority ranking.
Note: Not in all the cases testers are the ones who set the priority, and it may be left to the judgement of the product owner or the developers. But mostly we need to set it, so there is no escape from learning it. Why would you want to escape anyway, it’s fun, init?
Immediate (P0): Stop whatever you are doing and start working on this bug! This is the message we are sending through the P0 level bugs.
High (P1): This bug should be fixed as soon as you are done with the critical ones.
Medium (P2): You can fix this one whenever possible as soon as it is fixed for this release.
Low (P3): It is not really necessary to fix it soon. It can be fixed on the later releases, or maybe in the next century? Whenever your highness wants.
Famous Severity X Priority Crossover
Honk, honk. Pay close attention here as it is a very common question in job interviews to explain and give example about the different combinations of severity and priority for the defects.
We will go through each of them through the same application scenario. We will use a multilevel game, named “Funking Tonight”.
High Severity X High Priority
These types of bugs have a huge impact on the software where there will certainly be a user interaction. Since the users “have to” proceed in the application, these bugs have to be immediately fixed as well.
In the game, right when the user attempts to enter the game by clicking on the PLAY button, nothing happens, and the user cannot actually play the game. In this case, it is a severe bug as it blocks us from proceeding, and also has a high priority because, well, otherwise the whole game is useless, and we cannot even perform testing.
High Severity X Low Priority
When there is still a big impact (Probably not in the critical level) on the area that the defect occurs, but there is a chance that users may not even encounter the issue, the bug will be in this category.
At one of the levels in the game, when the player reaches to the limits of the world, the game unexpectedly crashes. That crash severely affects the game, but what are the chances that users will reach that particular level and the world limit? Since it is quite an unlikely situation, the bug can be fixed later on after the high priority ones.
Low Severity X High Priority
Bugs who fall into this type have a little impact on the system, yet they will surely impact the business as the users will realize them in any cases.
When the game is opened, there is a little typo on the game name, and it displays “Fu*king Tonight”. Technically the application is not in danger, yet without an exception all the players will see the typo and it will create a sensation in the gaming community. Unless the product owner decided to do that deliberately as a marketing strategy, this defect should immediately be fixed.
Low Severity X Low Priority
Supposedly these are the silent little ants of the testing dimension. This type of bug can also be a typo on the interface. But not as serious as the previous one.
Let’s say on one specific dialog in the game, the character starts talking with “hillo” instead of hello. Players will likely not even care or realize about the typo as it does not affect their gameplay. It is still a defect and has to be fixed to make the game perfectly bug-free yet is the last bug to be fixed.
If you have realized, we do the severity/priority distinction by how the system and the user experience affected to how the business will be affected. By internalizing this core knowledge and grasping both the functional and business requirements, we can be as accurate as possible while assessing those features for a defect.
Best snitching practices
There are 7 things we should be keeping in mind while reporting bugs and make sure that they are going to be smashed. Why 7? Cuz rainbow, and bugs hate rainbow! (What are we talking about?)
1-) Be concise
First point is very simple and straight to the point. Just as how our bugs should look like.
It is always important to think from the developers’ point of view while writing the report. “If I was the developer reading this bug, would I be annoyed by the number of unnecessary details?”, “Would I fully understand what the bug is about?” are two questions we can think of, and in the end, we should be able to convince the developers that the issue is a bug and not a feature.
So, include just enough explanation on the title, description, actual and expected result fields. But don’t be lazy or try hard to shorten the text either and make sure to add all the necessary details.
Note that this principle may exclude the reproduction steps, as we said that we should be as detailed as possible to make sure the bug will be reproduced without any problem.
Now, it’s the time to write an example. Kipo, over here!
We were testing the
2-) #everyfieldmatters
Did you know that #everyfieldmatters is the most common hashtag among the test leads on Twitter?
Okay, that was a made-up info, but it is a fact that every field matters. So, fill out all the attributes of the bug thoroughly before raising it. Sometimes, again especially in large projects, when testers are the bold of a big machine, they may not see why the accuracy in each field is crucial.
It is less likely to make mistakes on the ones like the build number or the device, but sometimes the area, type of issue or the priority may be a bit vague to the tester. If we are unsure about what to put for those fields, we should not play the guessing game, and discuss with our colleagues or even with the leads. Believe me, leads will get more upset if you assess the area of issue than if you simply ask them what would suit the best.
Also, considering the attachments section as the last field, always add the proof of the bug. If it is explainable by a screenshot, there is no need to bother recording a video, but otherwise be a good tester and add a recording if the issue is a more complicated one. If you can, in addition to the proof of the issue, also attach the console log, so that the developers can spot the issues more quickly.
3-) Always double check
Nah, double ain’t enough, let’s triple check.
There are several things to triple check before submitting the bug ticket.
The bug itself
Yes, first of first, we need to make sure that what we have found is actually a bug. That’s where it comes the importance of understanding the functionality and the business very well.
Especially in agile projects, there will be parts that are still under development and not yet in the testing scope. Or something that looks like an issue, in reality, may be something that the developers have done to work around another bug.
To ensure that your bug is valid, always recheck the functionality and the testing scope.
Dupes
Once you are confident that what you have seen is a defect, make a search in the database to see if it has been already entered. For that, learn your way around the tracking system very well. Usually, it is slow to search only by keywords in the description, so try to use the fields. We don’t have various attributes for no reason.
Fields
We said that it is less likely to make mistake on the obvious fields, but they are the ones that are overlooked the most. You think you have prepared a perfect report with all accurate and concise information, but you apparently forget to include the affected device. Imagine how demoralizing to be called out by your lead or by the developers to be more careful while writing bugs due to a small mistake. But it is cold and brutal out here. If you want to perfection the software you are testing, you have to perfection your reports as well.
As another piece of advice, if you are new to QA, or to your project, feel free to seek a peer review from your more experienced fellows. No, they won’t be bothered, 90% of the time they will feel even happy to be acknowledged as a point of reference. This way you can prove yourself as an excellent tester even when you just start testing 😎.
4-) Be a diplomat
Here we came to the most vital point. This point adds a soft skill to the art of bug writing.
By being a diplomat, we mean just watch the tone of your communication with the developers. Again, NEVER blame the developers. You are nobody to tell them that they made a mistake, as we don’t know what is going on during the development process. Even if it is an obvious issue, only let them know about the defect and do not use any wording indicating it occurs due to a developer’s mistake.
Also, NEVER prompt them to fix the issue. We have to know our place, folks. We do not have any hierarchy with the developers, and our sole purpose should be to let them know about the bug. By assessing the priority, we are already letting them know when it is better to be fixed, so there are no other words we should be using to make them irritated.
Lastly, try to avoid using negative tone. This may be a bit tricky in practice especially you are working in your second language. But over the time, when you observe the way that the experienced testers write, you will be understanding how to use a neutral and constructive tone.
5-) Don’t form a relationship with your bug
Always keep in mind the reason of raising defects. Try to avoid thinking the bugs as your children, and don’t cry when they are not accepted by the developers. This is a bit tragicomic, but some testers misunderstand how to take ownership, and instead they get attached to their bugs.
Always remember, that while working, we are testers and team members, and if we don’t have a logical explanation from the project perspective on why the issue is a bug, and why it should be fixed in the timeline that we want, we should not be harassing the developers with unnecessary comments, like “This bug is serious and should immediately be fixed” without giving any further proper explanation.
After the work, you can perhaps grab some ice cream and take your bug up to a park to socialize with other bugs. Nobody will tell you not to do so, but remember during work, they are just reports.
6-) One bug – One issue – One Piece – One dream
As the rule of thumb, one report should be written to explain only one issue. This point has two sides.
Refrain from cramming more than one separate issue into one bug.
7-) Use the additional comments effectively
Additional comments or the advance information is the cherry on a bug. Although it is normally an “optional” field, try to use them whenever you think there is a useful piece of information that can be provided.
For instance, if you have an idea about the underlying reason of the defect, don’t hesitate to include it even if you are 100% sure. If it is true, developers will benefit from it, if not, nobody will judge you as long as your prediction is based on logic.
Other than the cause, we can give a more detailed explanation on impact of the defect, as we cannot really give the details in the description since it should be kept concise.
Funky QA 