Validate and Verify

While testing software, we do come across with cases where some kind of input is needed. Whether this is a manual input, e.g. user name for a login screen or a machine input, e.g. screen resolution or some kind of DB input, e.g. db triggers (do this if its a sunday, send SMS if there is a credit in a account), we need to worry about a large number of inputs. When we plan test cases for these scenarios, the nightmarish task is to identify a small set of these input values, since inputting each one of them is impractical and non-feasible solution.

For example, if we have to test Login Screen, I would probably start with normal alpha names, then probably alpha-numeric name and then the ones with special characters and so on. While for a Login screen, the problem is not very complex, it does tend to get complex if we have to identify inputs for a photo-share program, specifically a photo-uploader part. In this case probably it would be

1. low res files

2. high res files

3. files of a particular format

4. invalid files

and so on. The big question is that how do we decide on that golden set of inputs which we think would sort of solve our problem. Read More

Few days back, I tried to explain in my last post about SCM. If you are not a regular reader here, then I would encourage you to read that and then come back to this one. Click here to read about whats SCM.

Having understood SCM, the other key question for test practioners would be to dive into its relevance vis-a-vis software testing. Going back to basics, SCM is management of a configuration of various modules (which as a whole is user-facing software). For Software testing connection, lets try to figure out the ‘configurations’ and ‘modules’ which are related to software testing.

‘Test Plan’ is a good module to start with. A test plan is virtually like the bible or the constitution, its a set of rules, methods, priorities which help you to make day-to-day decisions. Well thats sounds very theorish. But it is. It may happen that you dont really go back to that document to confirm your action but its that way. To give you an example, a ‘Test Plan’ mentions that if you find a defect in a third-party component then you would follow a process and you follow it all the while. Or for that matter, a ‘Test Plan’ mentions the tools which you are going to utilize or whether you would have a ‘Bug Hunt’ for all major features or not. It also mentions some more key decisions like “what all you wont be testing”, whats your assumptions,

Read More

Software Configuration Management or SCM in short is one of those terms which either you know or you dont. Most of the folks who know it would actually believe it to be a source code control system or just a set of processes and documents or worse a change control procedure. While all of this is true, I am not sure how many of us have actually tried to think about its true meaning, its need and why its called SCM.

The truth is that I never thought about SCM as well and in my all test plans, I would invariably have a section on SCM and that in turn would mean that how we would keep code, documents, how the revisions would be stored but it never bothered me to brood on why its called ‘Software Configuration Management”. So after reading about it at places like http://en.wikipedia.org/wiki/Software_configuration_management and thinking about it for a while, here’s my take on this.

Lets separate the words, so we have “Software”, then we have “Configuration” and finally “Mangement”. I thought that if I could somehow understand “Configuration” well enough and then if I could link that with first and last word, then I am done.

Read More

Software testing is about executing a set of test cases which are well documented and probably anyone after a few rounds of rehearsal can do it well. While practitioners would not agree with this since in practice, you dont find all the bugs from documented test cases. At most you document the basic workflows or the core cases but its not realistic to assume that one would be able to document all the test cases. Typically you would document the ones which are most important, so the big questions is that ‘how much to document’.

I am sure all of us have been into a situation where we found a bug at some crucial time and then everyone would want to look at test cases and while the witch hunting is going on, you as a test manager wonder that we never document everything. So after testing for many years and then managing a small team and complete products, here are some tips on test-case documentation. Read More

Even though this industry is looked at by other folks as being infested by geeks and people who prefer to keep long hairs and not worry about being too tidy, who only drink black coffee, are at odds on social gathering and speak in C or Java rather then roman or french, the truth is that this industry indeed has lots of noble men.This is what you would be thinking, if you are a developer.

“………………..Wow. thats like real ego massage. A software tester claiming, or is proclaiming a better word, to be noble. What else could a man do ?. Especially when he is not writing code, or a better way to say would be ‘when he has not written a piece of code ever’………………..”

Read More

So many times, I go out and try searching for some thing or the other and wonder that why I can’t find a good directory of websites which provide information on software testing. So this time, I did few searches and identified few of them and thought of sharing them with all. I have not done any extensive research so I would definitely be missing lot of them but I used Google and if you dont get listed on Google then you have it.

If you do want me to include some more then comment back and I will add it here. I could not find a better way to order them so I used Google’s Page Rank for the home pages of these websites.

1. http://www.stickyminds.com/
2. http://www.testingfaqs.org/
3. http://www.testing.com/
4. http://www.softwareqatest.com/
5. http://www.sqatester.com/

Read More

Software development is evolving, its not too old a science where practitioner have found the magic bullet and converted that into neat and working processes and manuals. Its not yet reached to the level of predictability as that of an automobile manufacturing plant and it would take so much more time to get it as defined as “Tax” or ‘Accounts’, both of which Indian Software Engineers dont love too much.

While anything is evolving fast, new and new definition are added, new acronyms gets introduced and virtually anyone who is someone gets active. Such is the state of Software Development currently.

OK, so far so good, whats the context and why such a title if we are going to talk about the evolution of Software Development. Well, I wanted to briefly touch upon this before we go about de-mystifying three kinds of testing which is so often interchangeably used, more so because we try to generalize things. So to be short, let me start with the first.

1. Performance Testing - Measure the performance (startup time, time to save a db commit, time to get results from db etc) under controlled conditions. Repeat. Read More

As part of our testing, maximum time is spent around bugs, whether its writing a new bug, verifying a fix, giving more information, contesting the validity, linking similar bugs and so on. All these times we are looking at various pieces of information which are in the bug report.

My experience in testing has made me firmly believe that a good Bug Report is key to high quality software. My personal hypothesis is that if we can tackled this activity well then we are bringing in more efficiency. So lets look at a good bug report format.

Contents of a good bug report

Read More

Software testing is gradually coming off age and with every new day everyone is starting to realize the importance of having a dedicated testing effort for any software project. In a company like Adobe where I manage small to medium sized test teams, there is now a much greater focus towards testing. Most of the times we have a 1:1 ratio between developer and testers which is very good but still as the project starts to wind and testing team has to test almost all the s/w every day, it starts to get tight and amid this one big question which every tester keeps asking is that

“why the hell developers can’t write a new piece of code without bugs”

which in itself is a very un-reasonable query but a more reasonable one is

“when a developer fixes a bug, why can’t he ensure that there is no side effect.”

Why ? why and more whys ??

Read More

Whats the life cycle of a bug or a defect ?

Even though this seems like a simple problem and may not need lot of thinking but its very important to understand it well. Most of the tester spend lots of time around bugs. They report bugs, regress bugs, verify bug fixes, bounce them and so on. From the same yard stick a developer spends lots of them on and around bugs as well, tries to reproduce them, asks and gets more info to repro them, fixes them, tests the fix, comments on the bug for testing recommendation and so on.

Amid all this, dev and tester also fights a lot over bugs and possibly all for a good cause viz. Quality. Lot of this fight can be saved or better understood if we really understand the life cycle of a bug. That way if a bug gets marked back to tester for adding more information, that would then look like routine or normal to him rather then taking that as an offensive.

Read More