Skip to main content

What to log in a bug?

Bugs are a testers bread and butter, it's what we live on and thrive on raising.

 However, my pet hate are bugs that just aren't very descriptive. I've seen bugs in the past with the simple message "Search is not working"... This wastes time of a developer who will then have to debug and find out for themselves when/where search isn't working, why it isn't working and then fix the issue. It also reflects badly on QA, which in previous posts is something we have to fight against!

 This does tie in when should you raise a defect (more on this in a future post), as I like to go by the principle of if it will take longer to raise a bug than to fix it then don't raise it and just speak to a developer and explain the situation, giving them a walk through if needed, and then wait for them to fix it and test it again. Obviously this isn't always the route that is chosen, and when a fix for the issue will take a lot of investigation or time to fix, then a bug is to be raised.

 What to log in the bug?

 I find any information that you, as a QA feel would help the Developer fix it is a good start.

 This can be (but not limited to) the following:
  • Title
  • Description - Usually a descriptive summary of what the bug is, how it is reproduced, the effects of the bug, and areas that are affected by said bug
  • Severity - How severe is the bug? Often this can be given as a 1-5 number, with 1 being the most critical and absolutely blocking anyone from doing any testing on the system, to 5 which can often be a minor cosmetic issue. Often the company you are at will have a defined Severity list... if not, I think it's best you create one ASAP!
  • System Specification - This can vary depending on the system under test, if a website is under test, useful information here would be things like Browser type, Browser version, however if a windows app is under test, then obviously windows version with the version number, RAM, CPU etc would be useful for a developer
  • Associated Test Cases - List here any test cases that are failing as a result of this test, this ensures there is traceability and also helps when the bug is fixed to identify the minimum tests that need to be run
  • Build Number - Was a specific build responsible for the bug? If so raise it here. 
  • Environment - Which environment the bug was found in, was it a UAT environment? A development environment? etc.
  • Attachments - Not always a necessity, but things like a screenshot are often good here.
  • Status - This is important for following the progress of the bug throughout it's life. 
Often, people will say to put in a Business Priority, this to me isn't something that should come from a QA, and thus isn't always included in a bug template, if you wish to include this field (which can be useful in deciding what bugs to fix) then speak to the Product Owner, explain the bug, explain the issues and try and come to a conclusion together on the Business Priority in getting this bug fixed.

To me, the more information you can include in a bug the easier it will be for a developer to recreate, investigate and hopefully fix the issue. 

There are tools you can get that can record the interactions with the browser (I will discuss useful addons in later posts for browsers) and thus a video can be produced which will aid the developer in viewing the steps required to recreate the issue.

 So taking the bug above, which was labelled as "Search isn't working", we can apply the above template to it and achieve the following.
  • Title : When Searching an "Oops Something has gone wrong page is shown"
  • Description - When the user searches for a search term on any page, then an Oops Something has gone wrong page is displayed. The Server logs show the following error in the Event Viewer:

    " ERROR - Failed to establish connection to XXX "

    The values in the web.config for the search Endpoint are:

    " XXXXX "

    Calling the search API directly also does not generate any results.

  • Severity - 2
  • System Specification - The bug occurs under any browser.  
  • Associated Test Cases - 52342, 234234
  • Build Number - 0.8742
  • Environment - http://ajh-web-dev-23
  • Status - New
(The screenshot in this instance doesn't really add much to the bug report, but it can help the developer in other cases, most notably on browser specific issues where elements are misplaced etc)

From the above bug the developer will know exactly what has already been checked, they can see immediately if the endpoint is correct, and even that it's not an issue with the website as the API directly isn't generating any results.

I think you'll agree that the above bug is far more informative than "Search is not working" and will save time from the developer and even the QA in having to play a game of information ping pong with the developer in getting the correct information out of the QA! This will be the first impression the developer gets of the bug, so make it as informative and easy to follow as possible.



Labels:

Comments

  1. Unknown26 February 2013 at 00:58

    Nice post. I've had lots of discussions about this over the years, and come to the conclusion that you should always assume the person reading the bug report doesn't have any intrinsic knowledge of the system. Therefore, don't use any special terms/abbreviations that are only known internally, and add a step-by-step guide to reproducing the bug. Consider it a "how to recreate this issue for dummies". :)

    ReplyDelete
  2. Amar G Deogirikar30 April 2013 at 06:08

    Apart from the ones mentioned above, I usually also like to include the following details in a bug report:

    Title: Dev's/Product guys in my current company like to know priority and severity at the start of the title, starts with P0;SH (sounds like posh but this is showing that this is a blocker bug. Example: P0;S-H When Searching an "Oops Something has gone wrong page is shown"

    Severity - have 3 levels, High - blocking user from doing something, Medium - Nothing obviously noticeable, Low - Affects nobody and no harm is done.

    Priority - signifies how quickly a bug needs to be fixed. And I appreciate your comments on this one as I have also noted that testers usually tend to mark bugs with the highest possible priority to any bug they log, but this can cause issues with Dev's because they will not take this field seriously in the future and lose respect. I think testers should put priority on bugs anyway, and able to justify this because they know the bug so well and are in a position to guess this field. This always be triaged later with the Product owner and scrum master.

    Bug Type: Functional/UI offset or Type/ Data issue

    Reproducibility: Always/intermittent use of this field is to specify the degree of reproducibility of the issue. Does the bug reproduce consistently every time that some steps are followed? or if is it a random bug?


    what are your thoughts on this one?

    ReplyDelete

Post a Comment

Popular posts from this blog

Coding something simple.... or not! Taking a screenshot on error using Selenium WebDriver

I recently wrote a little function that takes a screenshot at the end of a test if it has errored. What sounded very simple at the start turned out to be quite a bit of work, and quite a few lines of code to handle certain scenarios! It's now over 50 lines of code! I'll start with what I had at the beginning, this was to simply take a screenshot in the working directory, we are using SpecFlow and Selenium to run the tests, so we are going to check if the ScenarioContext.Current.TestError isn't null, if it is, then using Selenium, take a screenshot (note the below code is a simplified version of what I had at the beginning). [AfterScenario]         public static void TakeScreenShotOnError()         {             if (ScenarioContext.Current.TestError == null) return;             var screenshotDriver = Driver as ITakesScreenshot;             if (screenshotD...
15 comments
Read more

How to manage resources within new teams?

Working where I work we are constantly spinning up new teams to take on new workloads as business come up with new demands and new features they want developed and tested. The problem with this is how do we ensure the work of the newly spun up team is of sufficient quality. One method is by taking people from other established teams and placing them on the new team. This works great for the new team, but unfortunately it will oftenl eave the established team lacking in a resource whilst they try and fill the gap left by the person who has left. We are seeing this often with our offshore teams, it can be damaging to the team structure and the teams velocity, but try as I might, I can't think of another way around it. It's far easier to take 1 person from a team that is established than it is to build a whole new team from scratch. At least by leaving the core of a team in place, you should be guaranteeing that the new team are aware of any coding standards or any QA standard...
Post a Comment
Read more

Considerations when creating automated tests

We recently released to a number of teams our automated regression pack that has been worked on over the past few months. This regression pack tests legacy code, but contains a large number of tests.  As a bit of background, a number of teams are working on new solutions whilst some are still working on legacy code. With this in mind we constructed an email with a list of guidelines when creating new tests that need to be added to this regression pack.  I figured that these can be quite broad so should apply for any organisation, so thought it would make an interesting blog post...  So here goes,  when creating automated tests, it's important to consider and adhere to the following: - Think about data . The tests need to retrieve or set the data they need without any manual intervention - This should help them be more robust and easier to run without manual intervention. - The tests need to be idempotent - By making it so that each test is standalone and does...
Post a Comment
Read more