Validate and Verify

As software testers, we have to find faults in our own creations. Even though its lot of fun and a lot more sadistic pleasure to send a RED report every week to management on how buggy our own software is,  we can’t be holding on to  it till eternity. This is more of a problem for software managers like me then what I was few years back, a Test engineer. Somehow it never occurred to me as a Test Engineer that all my RED reports are actually not a good news to my manager and nor to company in general.

So as we like it or not, there is a time when there are very few bugs which deem a fix and we have to ship the product. Once a version is shipped, the whole game reverses. The same shortcomings which looked so much on the face are now ‘Known Issues’ or ‘Cosmetic’ and suddenly the average test engineer Mr. Joe starts to take full ownership, the same feature which he scoffed at since it was sort of developed by someone else is now something which he blessed. Any fault which gets reported later is looked at with suspicion and an explanation is tried then gladly logging them in the ‘Bug Reporting System’. Over time, the whole problem of maths-n-science and of finding logical bugs becomes a OB (Organizational Behavior) and psychology thing. What is now happening is that a tester has started to love his product too much and not able to see faults.

Read More

A good ‘Test Plan’ is often that well bound book on your glossy shelf which you never read. I must have created many test plans in my stint at Newgen Software, Legato and lately at Adobe and probably never consulted them again during the course of the testing. But its a well needed document. I was fortunate to not work in a highly process oriented company but we still need to make a plan, at least in the beginning of the project. Most of times I have seen people scouting for websites and what not to get that golden test plan template which they can just fill in and sort of get away.

I think we all do it in our youth and as I look back it all looks not-worth it :), the template hunting part. So let me take you through a decent or a good-enough test plan format w/o binding it in any template.

1. Think about the project for which you are going to create the test plan and explain it in few lines. Name this section as ‘Backround’ or ‘Context’ and have it as one of the first things. If you are testing a web-portal specializing in travel-experiences like www.ghumakkar.com (sneaking some publicity for this site) then mention something like “ghumakkar is a travel blog platform for traveler community and is primarily meant for sharing travel stories and to inspire the world……” and so on.

Read More

While testing software, we do come across with cases where some kind of input is needed. Whether this is a manual input, e.g. user name for a login screen or a machine input, e.g. screen resolution or some kind of DB input, e.g. db triggers (do this if its a sunday, send SMS if there is a credit in a account), we need to worry about a large number of inputs. When we plan test cases for these scenarios, the nightmarish task is to identify a small set of these input values, since inputting each one of them is impractical and non-feasible solution.

For example, if we have to test Login Screen, I would probably start with normal alpha names, then probably alpha-numeric name and then the ones with special characters and so on. While for a Login screen, the problem is not very complex, it does tend to get complex if we have to identify inputs for a photo-share program, specifically a photo-uploader part. In this case probably it would be

1. low res files

2. high res files

3. files of a particular format

4. invalid files

and so on. The big question is that how do we decide on that golden set of inputs which we think would sort of solve our problem. Read More

Few days back, I tried to explain in my last post about SCM. If you are not a regular reader here, then I would encourage you to read that and then come back to this one. Click here to read about whats SCM.

Having understood SCM, the other key question for test practioners would be to dive into its relevance vis-a-vis software testing. Going back to basics, SCM is management of a configuration of various modules (which as a whole is user-facing software). For Software testing connection, lets try to figure out the ‘configurations’ and ‘modules’ which are related to software testing.

‘Test Plan’ is a good module to start with. A test plan is virtually like the bible or the constitution, its a set of rules, methods, priorities which help you to make day-to-day decisions. Well thats sounds very theorish. But it is. It may happen that you dont really go back to that document to confirm your action but its that way. To give you an example, a ‘Test Plan’ mentions that if you find a defect in a third-party component then you would follow a process and you follow it all the while. Or for that matter, a ‘Test Plan’ mentions the tools which you are going to utilize or whether you would have a ‘Bug Hunt’ for all major features or not. It also mentions some more key decisions like “what all you wont be testing”, whats your assumptions,

Read More

Software Configuration Management or SCM in short is one of those terms which either you know or you dont. Most of the folks who know it would actually believe it to be a source code control system or just a set of processes and documents or worse a change control procedure. While all of this is true, I am not sure how many of us have actually tried to think about its true meaning, its need and why its called SCM.

The truth is that I never thought about SCM as well and in my all test plans, I would invariably have a section on SCM and that in turn would mean that how we would keep code, documents, how the revisions would be stored but it never bothered me to brood on why its called ‘Software Configuration Management”. So after reading about it at places like http://en.wikipedia.org/wiki/Software_configuration_management and thinking about it for a while, here’s my take on this.

Lets separate the words, so we have “Software”, then we have “Configuration” and finally “Mangement”. I thought that if I could somehow understand “Configuration” well enough and then if I could link that with first and last word, then I am done.

Read More

Software testing is about executing a set of test cases which are well documented and probably anyone after a few rounds of rehearsal can do it well. While practitioners would not agree with this since in practice, you dont find all the bugs from documented test cases. At most you document the basic workflows or the core cases but its not realistic to assume that one would be able to document all the test cases. Typically you would document the ones which are most important, so the big questions is that ‘how much to document’.

I am sure all of us have been into a situation where we found a bug at some crucial time and then everyone would want to look at test cases and while the witch hunting is going on, you as a test manager wonder that we never document everything. So after testing for many years and then managing a small team and complete products, here are some tips on test-case documentation. Read More

Even though this industry is looked at by other folks as being infested by geeks and people who prefer to keep long hairs and not worry about being too tidy, who only drink black coffee, are at odds on social gathering and speak in C or Java rather then roman or french, the truth is that this industry indeed has lots of noble men.This is what you would be thinking, if you are a developer.

“………………..Wow. thats like real ego massage. A software tester claiming, or is proclaiming a better word, to be noble. What else could a man do ?. Especially when he is not writing code, or a better way to say would be ‘when he has not written a piece of code ever’………………..”

Read More

So many times, I go out and try searching for some thing or the other and wonder that why I can’t find a good directory of websites which provide information on software testing. So this time, I did few searches and identified few of them and thought of sharing them with all. I have not done any extensive research so I would definitely be missing lot of them but I used Google and if you dont get listed on Google then you have it.

If you do want me to include some more then comment back and I will add it here. I could not find a better way to order them so I used Google’s Page Rank for the home pages of these websites.

1. http://www.stickyminds.com/
2. http://www.testingfaqs.org/
3. http://www.testing.com/
4. http://www.softwareqatest.com/
5. http://www.sqatester.com/

Read More

Software development is evolving, its not too old a science where practitioner have found the magic bullet and converted that into neat and working processes and manuals. Its not yet reached to the level of predictability as that of an automobile manufacturing plant and it would take so much more time to get it as defined as “Tax” or ‘Accounts’, both of which Indian Software Engineers dont love too much.

While anything is evolving fast, new and new definition are added, new acronyms gets introduced and virtually anyone who is someone gets active. Such is the state of Software Development currently.

OK, so far so good, whats the context and why such a title if we are going to talk about the evolution of Software Development. Well, I wanted to briefly touch upon this before we go about de-mystifying three kinds of testing which is so often interchangeably used, more so because we try to generalize things. So to be short, let me start with the first.

1. Performance Testing - Measure the performance (startup time, time to save a db commit, time to get results from db etc) under controlled conditions. Repeat. Read More

As part of our testing, maximum time is spent around bugs, whether its writing a new bug, verifying a fix, giving more information, contesting the validity, linking similar bugs and so on. All these times we are looking at various pieces of information which are in the bug report.

My experience in testing has made me firmly believe that a good Bug Report is key to high quality software. My personal hypothesis is that if we can tackled this activity well then we are bringing in more efficiency. So lets look at a good bug report format.

Contents of a good bug report

Read More