Validate and Verify

A good ‘Test Plan’ is often that well bound book on your glossy shelf which you never read. I must have created many test plans in my stint at Newgen Software, Legato and lately at Adobe and probably never consulted them again during the course of the testing. But its a well needed document. I was fortunate to not work in a highly process oriented company but we still need to make a plan, at least in the beginning of the project. Most of times I have seen people scouting for websites and what not to get that golden test plan template which they can just fill in and sort of get away.

I think we all do it in our youth and as I look back it all looks not-worth it :), the template hunting part. So let me take you through a decent or a good-enough test plan format w/o binding it in any template.

1. Think about the project for which you are going to create the test plan and explain it in few lines. Name this section as ‘Backround’ or ‘Context’ and have it as one of the first things. If you are testing a web-portal specializing in travel-experiences like www.ghumakkar.com (sneaking some publicity for this site) then mention something like “ghumakkar is a travel blog platform for traveler community and is primarily meant for sharing travel stories and to inspire the world……” and so on.

Read More

While testing software, we do come across with cases where some kind of input is needed. Whether this is a manual input, e.g. user name for a login screen or a machine input, e.g. screen resolution or some kind of DB input, e.g. db triggers (do this if its a sunday, send SMS if there is a credit in a account), we need to worry about a large number of inputs. When we plan test cases for these scenarios, the nightmarish task is to identify a small set of these input values, since inputting each one of them is impractical and non-feasible solution.

For example, if we have to test Login Screen, I would probably start with normal alpha names, then probably alpha-numeric name and then the ones with special characters and so on. While for a Login screen, the problem is not very complex, it does tend to get complex if we have to identify inputs for a photo-share program, specifically a photo-uploader part. In this case probably it would be

1. low res files

2. high res files

3. files of a particular format

4. invalid files

and so on. The big question is that how do we decide on that golden set of inputs which we think would sort of solve our problem. Read More

Few days back, I tried to explain in my last post about SCM. If you are not a regular reader here, then I would encourage you to read that and then come back to this one. Click here to read about whats SCM.

Having understood SCM, the other key question for test practioners would be to dive into its relevance vis-a-vis software testing. Going back to basics, SCM is management of a configuration of various modules (which as a whole is user-facing software). For Software testing connection, lets try to figure out the ‘configurations’ and ‘modules’ which are related to software testing.

‘Test Plan’ is a good module to start with. A test plan is virtually like the bible or the constitution, its a set of rules, methods, priorities which help you to make day-to-day decisions. Well thats sounds very theorish. But it is. It may happen that you dont really go back to that document to confirm your action but its that way. To give you an example, a ‘Test Plan’ mentions that if you find a defect in a third-party component then you would follow a process and you follow it all the while. Or for that matter, a ‘Test Plan’ mentions the tools which you are going to utilize or whether you would have a ‘Bug Hunt’ for all major features or not. It also mentions some more key decisions like “what all you wont be testing”, whats your assumptions,

Read More

Software Configuration Management or SCM in short is one of those terms which either you know or you dont. Most of the folks who know it would actually believe it to be a source code control system or just a set of processes and documents or worse a change control procedure. While all of this is true, I am not sure how many of us have actually tried to think about its true meaning, its need and why its called SCM.

The truth is that I never thought about SCM as well and in my all test plans, I would invariably have a section on SCM and that in turn would mean that how we would keep code, documents, how the revisions would be stored but it never bothered me to brood on why its called ‘Software Configuration Management”. So after reading about it at places like http://en.wikipedia.org/wiki/Software_configuration_management and thinking about it for a while, here’s my take on this.

Lets separate the words, so we have “Software”, then we have “Configuration” and finally “Mangement”. I thought that if I could somehow understand “Configuration” well enough and then if I could link that with first and last word, then I am done.

Read More

Software testing is about executing a set of test cases which are well documented and probably anyone after a few rounds of rehearsal can do it well. While practitioners would not agree with this since in practice, you dont find all the bugs from documented test cases. At most you document the basic workflows or the core cases but its not realistic to assume that one would be able to document all the test cases. Typically you would document the ones which are most important, so the big questions is that ‘how much to document’.

I am sure all of us have been into a situation where we found a bug at some crucial time and then everyone would want to look at test cases and while the witch hunting is going on, you as a test manager wonder that we never document everything. So after testing for many years and then managing a small team and complete products, here are some tips on test-case documentation. Read More

Software development is evolving, its not too old a science where practitioner have found the magic bullet and converted that into neat and working processes and manuals. Its not yet reached to the level of predictability as that of an automobile manufacturing plant and it would take so much more time to get it as defined as “Tax” or ‘Accounts’, both of which Indian Software Engineers dont love too much.

While anything is evolving fast, new and new definition are added, new acronyms gets introduced and virtually anyone who is someone gets active. Such is the state of Software Development currently.

OK, so far so good, whats the context and why such a title if we are going to talk about the evolution of Software Development. Well, I wanted to briefly touch upon this before we go about de-mystifying three kinds of testing which is so often interchangeably used, more so because we try to generalize things. So to be short, let me start with the first.

1. Performance Testing - Measure the performance (startup time, time to save a db commit, time to get results from db etc) under controlled conditions. Repeat. Read More

As part of our testing, maximum time is spent around bugs, whether its writing a new bug, verifying a fix, giving more information, contesting the validity, linking similar bugs and so on. All these times we are looking at various pieces of information which are in the bug report.

My experience in testing has made me firmly believe that a good Bug Report is key to high quality software. My personal hypothesis is that if we can tackled this activity well then we are bringing in more efficiency. So lets look at a good bug report format.

Contents of a good bug report

Read More

Whats the life cycle of a bug or a defect ?

Even though this seems like a simple problem and may not need lot of thinking but its very important to understand it well. Most of the tester spend lots of time around bugs. They report bugs, regress bugs, verify bug fixes, bounce them and so on. From the same yard stick a developer spends lots of them on and around bugs as well, tries to reproduce them, asks and gets more info to repro them, fixes them, tests the fix, comments on the bug for testing recommendation and so on.

Amid all this, dev and tester also fights a lot over bugs and possibly all for a good cause viz. Quality. Lot of this fight can be saved or better understood if we really understand the life cycle of a bug. That way if a bug gets marked back to tester for adding more information, that would then look like routine or normal to him rather then taking that as an offensive.

Read More

Some days back I went to watch Die Hard 4.0, please notice that its 4.0 and not 4. The movie is about our old cop Bruce and the bad boys who want to destroy the world, in this case they were more interested to demo or show a POC (Proof Of Concept) that given the right funding they can save the world from destruction. They actually did some destruction to drive home the point that destruction is possible.

The movie moves very smoothly with few people getting killed every now and then. Special effects are awesome and good wins over bad, albeit the good (in this case our very own Bruce) also wins over a F16 bomber, in-numerable bullets, n kill attempts, speeding oncoming cars and what not. Read More

If you are software test engineer like me then almost every one in your office would have at least told you once that we must do automation and more if we are already doing it. Most of these people would be from non-testing function and probably they dont know enough and hence their feedback on doing-automation or doing-more-automation makes lot of sense, these guys are not biased and must be genuinely interested in seeing more automation.

To most of them, you would have replied with a sort of sheepish smile (like the one you just did), or with a mono-syllable answer, mostly Yes, or may be few lines, incase its an executive who is asking/suggesting.

Read More