Sunday, April 19, 2009

Bug reporting skills, Good testers and "I entered something and it crashed" syndrome

I have roamed around India speaking to testers from different cities. I notice common patterns and region specific patterns about how testers think and or communicate.

I know there would be a never ending argument of what are the attributes of good testers because everyone seem to have a different opinion. However, I think there exists a set of attributes that any school of thought would agree upon.

Let me discuss one such and you might have guessed it from the title. I have never seen a good tester say, "I entered something and something and it crashed". But why do they not say?

I have heard the above sentence from a lot of testers and most of them have been clueless about software testing. I shall explain to you my reasoning of the same.

Assume you are the cockroach catcher and that's what you do for a living. I hire you to catch a cockroach that I saw in my home and you are going to ask me, "Where did you see the cockroach?" and if I dont have an answer for that how helpful am I being to myself in getting the cockroach out of my home?

If I am staying in a 2 acre bungalow, your job is all the more complicated. Instead, I say, "I last spotted a couple in the kitchen and the bathroom", I am helping myself and you.

Look into your organization's bug tracking system and you'd find testers reporting problem like, "I entered some input and application crashed". Now, that's like reporting, there is a cockroach in this country find it.

Unless you are specific as a tester, how do you suppose a developer to fix the problem. Testers are finding a bug to get it resolved or help the management take a better decision. Reporting it in such a way that it is of no help to anyone is an extreme bad job.

Why would a developer ever want to respect such a tester? Why would the developer community want to respect such community of testers?

So, if you have been doing it and want to be better, here is a way:

You may want to look into what the Quantified String Generator can do to solve this problem of yours or any other toolkit at Common Test Data Generator in Testers Desk

Here is an output:

2^4^6^8^11^14^17^20^23^26^29^32^35^38^41^44^47^50^53^56^59^62^65^68^71^74^77^80^83^86^89^92^95^98^102^106^110^114^118^122^126^130^134^138^142^146^150^154^158^162^166^170^174^178^182^186^190^194^198^202^206^210^214^218^222^226^230^234^238^242^246^250^254^258^262^266^270^274^278^282^286^290^294^298^302^306^310^314^318^322^326^330^334^338^342^346^350^354^358^362^366^370^374^378^382^386^390^394^398^402^406^410^414^418^422^426^430^434^438^442^446^450^454^458^462^466^470^474^478^482^486^490^494^498^502^506^510^514^518^522^526^530^534^538^542^546^550^554^558^562^566^570^574^578^582^586^590^594^598^602^606^610^614^618^622^626^630^634^638^642^646^650^654^658^662^666^670^674^678^682^686^690^694^698^702^706^710^714^718^722^726^730^734^738^742^746^750^754^758^762^766^770^774^778^782^786^790^794^798^802^806^810^814^818^822^826^830^834^838^842^846^850^854^858^862^866^870^874^878^882^886^890^894^898^902^906^910^914^918^922^926^930^934^938^942^946^950^954^958^962^966^970^974^978^982^9


Now, whether the developer or specification talks about Boundary or not, you might be able to find it out using this Quantified String Generator. For instance, let me assume you copy paste the above on to your input field and it accepts 983 characters and it then appears to crash as you hit the submit button.

Now, investigating this bug is an interesting challenge: Typical bad testers would directly report the problem however here is a pattern that I observe in most of the good testers I have met - They would want to know from which number does this crash occur.

Is it 983 characters or is it a number below that?

So, we go for a smaller sample than 983 characters: And here is a sample output from Quantified String Generator:

2^4^6^8^11^14^17^20^23^26^29^32^35^38^41^44^47^50^53^56^59^62^65^68^71^74^77^80^83^86^89^92^95^98^102^106^110^114^118^122^126^130^134^138^142^146^150^154^158^162^166^170^174^178^182^186^190^194^198^202^206^210^214^218^222^226^230^234^238^242^246^250^254^258^262^266^270^274^278^282^286^290^294^298^30

Tip: Did you notice the beauty that you dont need to count the characters and it counts itself?

That's 300 characters.

Now, in case we see that it does not crash for 300 characters, we have learned that it is somewhere between 300 and 983. And as we zero down, we might find that great number from which the application starts crashing.

Imagine you reporting a problem like, "I used the Quantified String Generator and discovered that 432 characters ( and thereafter ) appears to crash the application" and ask a striking question to the developers, "Why would it crash at 432?", "What is so magical about 432?"

Note: If you'd want to see that in demonstration, you may consider watching this video.

That's when the developer would start respecting a tester for his skills of bug reporting, bug investigation and communication.

I leave the choice to you. You need to do lots of things and change the way you do things to become a good testers and you just need to do whatever you are doing right now to be a bad tester.

1 comments:

Virtualkey said...

Hi Pradeep,

I liked the way you integrated explaining a feature of Testersdesk with Bug reporting techinques.Scenario which yor considered looked pretty obvious to me.By the way can you let us know what are all the other possible areas where 'Quantified String Generator' can be used.
I am not sure wether you did it in your video. I could not see it, for around 30 minutes, time being one of the reasons.I think a 3-5 minute video giving me snap shot of different scenarios for specific Feature in TD, and a detalied followup post for that video would keep newbies and enthusiasts glued to your posts with more intrest and facination.

Happy Testdesking !

Virtualkey

Post a Comment