Validate and Verify

The million dollar question which every Manager asks to testing teams is that ‘Whether they have executed all the test cases ? “. If you are a software test practitioner, you would be feeling really frustrated. How on earth, can you ever execute all the ‘Test Cases’ for a particular feature.

Well, there is no silver bullet but there is a way which you can use to get a good enough answer for the above question. Its called ‘Code Coverage’ analysis.

How does it work
There are certain software tools in the market which can be used to do ‘Code Coverage’. One of the popular software is ‘Bullseye’. The process is simple. While building binaries for your software program, configure ‘Bullseye’ with the IDE. When binaries are made, bullseye inserts some of its own signatures. This is called ‘Instrumentation’ of the code.

Read More

As Software testers, one of the big responsibility we all share is being acting like a user. While most of us love it and use this for our and user’s advantage its not a very natural thing to do. If it had been, then probably you would be that user and not really a software tester. The only way, we can most truly act like a user when we are testing either a bug-reporting application or a test-case-automation system or like wise. If we happen to be testing these applications then we are in a better situation but In real world, we end up testing banking and insurance application, graphics and document applications, storage management systems and so on. I work in a product based company and my area of testing falls under ‘Consumer Photo Applications’ so even though I click lots of photos and I spend time on them, I am not really someone who would buy a USD 100 software to manage my ever growing library of photos, correct them, share them over e-mail or cut DVDs and so on. So I am not really a true user.

Read More

This post is mostly a result of a comment from Rahul. He made the comment on a post on Validation and Verification and I think it was a great idea, so here I am with an attempt to distinguish these two closely related and often wrongly used acronyms viz QA (Quality Assurance) and QC (Quality Control).

First the definitions
Quality Assurance - Quality assurance, or QA for short, is the activity of providing evidence needed to establish quality in work, and that activities that require good quality are being performed effectively. All those planned or systematic actions necessary to provide enough confidence that a product or service will satisfy the given requirements for quality. (As per Wikipedia )

Quality Control - In engineering and manufacturing, quality control and quality engineering are involved in developing systems to ensure products or services are designed and produced to meet or exceed customer requirements. These systems are often developed in conjunction with other business and engineering disciplines using a cross-functional approach. (As per Wikipedia)

And here’s a simple version of mine.
QA - Set of processes to ensure that we do a high quality work. These processess are at all level. In s/w world, this could mean a good template to capture requirements, a check-list for testing team which needs to be ticked before we deliver the s/w, code-commenting procedures so that if someone looks at a code which is done by someone else then it makes sense and so on.

QC - Test the desired behavior. Its about testing and finding faults. QC is not about putting a right process but about exercising the process.

Simple and over. Well, mostly but let me go a little deeper and illustrate this difference with few examples
Read More

Large software organizations have the luxury to invest in labs, or test labs to help their test engineer better test a software. These labs could range from a setup having racks and racks of machines of various configs or could be just a big room with 10-20 machines. Having worked in a company (Legato Systems, now EMC) which creates Enterprise Backup Solutions (Legato Networker) I have been fortunate to see a lab which really had these large cupboard sized machines, fiber-optic powered SAN (Storage Area Network), really cool racks which support multiple machines with same keyboard, a display which would slide out and then go vertical. Probably it was done since it was needed. At the same time, out here at Adobe, we usually have a set up where a bunch of standard desktop machines are present running all kinds of operating systems, various locales and so on. Probably the reason we do not have racks is that our users do not have racks. Though I am sure that having racks might be more space efficient but thats for another day.

Some of these labs are powered by Image servers, the ones which can spit a OS image and do a raw-copy on local disk by booting the machine through network, while others may rely on OS installation through shiny disks. At some places, you have a check-in/check-out register for each of the machine, so as you get in you can work on a machine which is available and then as you go, you let go of the machine. I am sure that during your work experience you might have seen a different kind of lab as well. Also, some of you must have spent long nights in one of these labs trying to isolate a bug or just getting done with your part of test coverage. Sometimes these are also fertile grounds of new associations, more so since these are not too crowded, fall in a neutral zone (its neither harry’s office nor sally’s cubicle) and there are always many reasons to ask for help, there would always be something which is not working. Before your mind takes off and before you get nostalgic , let me come back to my intention of writing this post.

My intention here is to try to capture some of the best practices, best methods and general good housekeeping tips to harness most from a test lab. A happy test lab is key to success of test-case-execution and here are some things which you can employ to make your lab happy. These are not in any order but feel free to ask back incase it appears fragmented.

Five Things towards a Happy Test Lab

Read More

I was previously blogging at a blogspot site and over time I realized that I am getting limited in terms of tools there so I opted to do it on my own. Apart from other things which are needed to host and run a website, one of the basic initial things is to identify a name. And after being unsuccessful at getting a domain for the more common names, I was lost and dejected when suddenly I thought of validateverify, checked it and found that this is available.

Over time, I also realized that there are still a lot of people who are googling on these terms, to find their meanings, there differences and so on. While as a software testing practitioner and as a interview-holic person, ‘Validation and Verification’ are ingrained on my mind-slate, I can imagine that there would be lot of people who might just get confused. More so in places where English is not their first (and probably preferred) language.

So lets try to understand these terms and then see their relation vis-a-vis software testing.

Validation - Act of finding whether something is ‘right’ or not. The ‘Right’ is from the perspective of overall need and desire. Whether something is valid or not. 

Verification - Act of finding whether something does whats its supposed to do in defined (or may be undefined at the time of testing) condition. Verification is for built behavior, its for smaller nuances and not for the initial big goal.

Let me now take couple of examples to drive these points home.

Read More

With software getting larger and larger, the interaction between engineers who work on various parts of a software has been getting more challenging. While this is not really a big problem for development engineers, the ones who primarily write code, but its indeed a growing problem for ‘Test Engineers’, the ones who test a software and ensure that it does what its supposed to do for the end-user.

Let me take a couple of examples to explain this better. In first case, we would take the case of ‘MS Word’ which is a part of ‘MS Office’ family of products. In second case, we will take a much smaller product like ‘Adobe Reader’ (more popularly knows by its former name ‘Adobe Acrobat Reader’).

MS Word has tons of modules or components or ’section of codes’ which would have their own small teams. These may be File IO (Open, Save, SaveAs), Fonts, Print, Tables, Mail Merge, Formatting, Bookmarking, Help and so on. Some of these could be common across products, e.g. Fonts handling, Help etc. Each of these small teams will have their own test engineers and most of them test ‘their’ piece within a certain ‘Entry and Exit’ ideal boundary condition. Its always assumed that the guy who is before this node in the value chain would behave well and the guy who is next in chain would also behave well. Thats a wishful thinking.

Lets take an example

Enter Text -> Save File -> Select Text -> bold the text -> Undo bold -> Save -> Close.

In this workflow (a logical sequence of actions performed for a task), lets take the case of the team which is testing ‘Undo and Redo’ function.

Read More

Software Testing is still more in realm of ‘Arts’ than Science, especially when we are talking about testing software products,  something like MS-Office or  Mercury  WinRunner. Unless you are a part of a highly evolved and organized form of testing, in other words, a defined set of test cases on a defined set of test-bed to be executed, you can safely assume that you wont be able to find all the bugs. Being paid to find bugs and still not able to find all is an oxymoron arrangement and as Test Engineers and Test Managers, we need to live in the reality (of not being able to find all bugs) and still do business (find all bugs).

In this post, I am going to talk about a exercise which lots of organizations are started to get into their test-process as an effective tool for finding bugs, which typically act elusive to the ‘Test Engineer’ who is looking at a specific area. This exercise is called by different names like Bug Hunt, Bug Bash, Focus Day, Bug Harvest and so on. We would refer it as ‘Bug Hunt’, just a personal preference, they all work same / similar.

What is a Bug Hunt ?

A Bug Hunt is an intense bug finding mission over a small period of time, from couple of hours to a day, done by people who are not testing that area or that product as normal day-to-day work or may not be ‘testers’ at all, allows all kinds of bugs (some bugs are feature-requests) to be reported and is done to sort of give one big shake to an area or a product before its being shipped.

Read More

A good ‘Test Plan’ is often that well bound book on your glossy shelf which you never read. I must have created many test plans in my stint at Newgen Software, Legato and lately at Adobe and probably never consulted them again during the course of the testing. But its a well needed document. I was fortunate to not work in a highly process oriented company but we still need to make a plan, at least in the beginning of the project. Most of times I have seen people scouting for websites and what not to get that golden test plan template which they can just fill in and sort of get away.

I think we all do it in our youth and as I look back it all looks not-worth it :), the template hunting part. So let me take you through a decent or a good-enough test plan format w/o binding it in any template.

1. Think about the project for which you are going to create the test plan and explain it in few lines. Name this section as ‘Backround’ or ‘Context’ and have it as one of the first things. If you are testing a web-portal specializing in travel-experiences like www.ghumakkar.com (sneaking some publicity for this site) then mention something like “ghumakkar is a travel blog platform for traveler community and is primarily meant for sharing travel stories and to inspire the world……” and so on.

Read More

While testing software, we do come across with cases where some kind of input is needed. Whether this is a manual input, e.g. user name for a login screen or a machine input, e.g. screen resolution or some kind of DB input, e.g. db triggers (do this if its a sunday, send SMS if there is a credit in a account), we need to worry about a large number of inputs. When we plan test cases for these scenarios, the nightmarish task is to identify a small set of these input values, since inputting each one of them is impractical and non-feasible solution.

For example, if we have to test Login Screen, I would probably start with normal alpha names, then probably alpha-numeric name and then the ones with special characters and so on. While for a Login screen, the problem is not very complex, it does tend to get complex if we have to identify inputs for a photo-share program, specifically a photo-uploader part. In this case probably it would be

1. low res files

2. high res files

3. files of a particular format

4. invalid files

and so on. The big question is that how do we decide on that golden set of inputs which we think would sort of solve our problem. Read More

Few days back, I tried to explain in my last post about SCM. If you are not a regular reader here, then I would encourage you to read that and then come back to this one. Click here to read about whats SCM.

Having understood SCM, the other key question for test practioners would be to dive into its relevance vis-a-vis software testing. Going back to basics, SCM is management of a configuration of various modules (which as a whole is user-facing software). For Software testing connection, lets try to figure out the ‘configurations’ and ‘modules’ which are related to software testing.

‘Test Plan’ is a good module to start with. A test plan is virtually like the bible or the constitution, its a set of rules, methods, priorities which help you to make day-to-day decisions. Well thats sounds very theorish. But it is. It may happen that you dont really go back to that document to confirm your action but its that way. To give you an example, a ‘Test Plan’ mentions that if you find a defect in a third-party component then you would follow a process and you follow it all the while. Or for that matter, a ‘Test Plan’ mentions the tools which you are going to utilize or whether you would have a ‘Bug Hunt’ for all major features or not. It also mentions some more key decisions like “what all you wont be testing”, whats your assumptions,

Read More