Validate and Verify

The other day I found myself in front of a bunch of fresh from the oven college grads looking up to me for some words of wisdom on Software Testing. I have been taking this class , once a year, to young recruits for some years and I am always in a fix on what to teach. I have my own share of tricks in my bag which I use for these classes but I did something new for the current batch.

It all started with a simple question i.e. What is Software Testing and I got a number of good responses but that led to a thought on what would be a first step to engage with software testing and the answer was very obvious, comprehension. Comprehension of the problem, of the object in scrutiny, of the principle which requires validation, of the hypothesis which would undergo various tests and rounds of rigorous observations before its blessed as a theory and so on. Its comprehension, comprehension and more comprehension.

The goal of Software Testing is to be able to certify a software after identifying and resolving all the defects which you encounter during your engagement with the software as a tester. That engagements would be an enriching one only if you know the subject. Its like knowing the intricacies of a specific musical instrument to play it well rather than just knowing how to play a particular kind of instrument. If you would have noticed that for all rock shows, concerts a lot of time goes in tuning the equipments. Even the masters need to tune the instrument before they begin their performance so as a tester , while its utmost important to know about testing methodologies, practices, tools etc, its far more important to know the software in question.
Read More

I am sure using the word QA would make you think of processes we go by standard definition. But you all would certainly agree that testing and QA are quiet interchangeably used. Working in Software services since start of my career and mostly with onsite and offshore model I have worked in both SDLC and typical third party QA practices where the SDLC is being conducted in a distributed framework (Requirement and Design is done by one company, Coding and Unit testing by another, QA by another and final support is done by some other company). 10 years back I felt a dearth need for an end to end test management tool which would reduce confusion and chaos during the QA\ Testing process. To fulfill this I feel Quality Center is a perfect tool. I have implemented and used Quality Center (earlier called Test Director) in various projects across different organizations.

HP Quality Center is basically a web based test management tool which if used and implemented correctly would help you right from specifying testing requirements, planning tests, executing tests, and tracking defects.
Read More

When we code a software, we choose an IDE which typically runs on an OS of a specific locale, having some Service Pack, co-existing with some software and so on. We compile the code and choose relevant targets, the compiler spits a binary and your job as a coder is done. In a complex world, there would be a build system which would get all the relevant files from relevant branches , from a source code control system , compile them and trigger the installer scripts. Installer script would create a package out of it. And its done.

When the package reaches the software tester, he has a big problem to solve. The first question which he has to answer that whether he should install the s/w on a fresh clean OS which has been installed just now, warm and inviting. OR he should rather install it on a dirty OS which has a plethora of software and would be more close to end-user. The answer is simple, do both.

The task doesn’t finish there, now comes the question about

* Which operating system, since it runs on all Windows. So should I install and make a clone of myself so that all of my clones can do simultaneous testing. To give you an example, MS Word runs on Windows and MAC. Within windows, it runs on Windows XP, Windows Vista and Windows 2000 and some other less popular flavors as well. Within Windows XP, it runs on all flavors of XP viz. SE, MCE, Professional. Win 2K has many more flavors and with Win Vista you need to do a MS certification to really say with surety on how many flavors it has. I am yet to talk about ‘Service Packs’.

You ask this question to a non-software-tester group head and he would say ‘Do it on All’.
Read More

The million dollar question which every Manager asks to testing teams is that ‘Whether they have executed all the test cases ? “. If you are a software test practitioner, you would be feeling really frustrated. How on earth, can you ever execute all the ‘Test Cases’ for a particular feature.

Well, there is no silver bullet but there is a way which you can use to get a good enough answer for the above question. Its called ‘Code Coverage’ analysis.

How does it work
There are certain software tools in the market which can be used to do ‘Code Coverage’. One of the popular software is ‘Bullseye’. The process is simple. While building binaries for your software program, configure ‘Bullseye’ with the IDE. When binaries are made, bullseye inserts some of its own signatures. This is called ‘Instrumentation’ of the code.

Read More

As Software testers, one of the big responsibility we all share is being acting like a user. While most of us love it and use this for our and user’s advantage its not a very natural thing to do. If it had been, then probably you would be that user and not really a software tester. The only way, we can most truly act like a user when we are testing either a bug-reporting application or a test-case-automation system or like wise. If we happen to be testing these applications then we are in a better situation but In real world, we end up testing banking and insurance application, graphics and document applications, storage management systems and so on. I work in a product based company and my area of testing falls under ‘Consumer Photo Applications’ so even though I click lots of photos and I spend time on them, I am not really someone who would buy a USD 100 software to manage my ever growing library of photos, correct them, share them over e-mail or cut DVDs and so on. So I am not really a true user.

Read More

This post is mostly a result of a comment from Rahul. He made the comment on a post on Validation and Verification and I think it was a great idea, so here I am with an attempt to distinguish these two closely related and often wrongly used acronyms viz QA (Quality Assurance) and QC (Quality Control).

First the definitions
Quality Assurance - Quality assurance, or QA for short, is the activity of providing evidence needed to establish quality in work, and that activities that require good quality are being performed effectively. All those planned or systematic actions necessary to provide enough confidence that a product or service will satisfy the given requirements for quality. (As per Wikipedia )

Quality Control - In engineering and manufacturing, quality control and quality engineering are involved in developing systems to ensure products or services are designed and produced to meet or exceed customer requirements. These systems are often developed in conjunction with other business and engineering disciplines using a cross-functional approach. (As per Wikipedia)

And here’s a simple version of mine.
QA - Set of processes to ensure that we do a high quality work. These processess are at all level. In s/w world, this could mean a good template to capture requirements, a check-list for testing team which needs to be ticked before we deliver the s/w, code-commenting procedures so that if someone looks at a code which is done by someone else then it makes sense and so on.

QC - Test the desired behavior. Its about testing and finding faults. QC is not about putting a right process but about exercising the process.

Simple and over. Well, mostly but let me go a little deeper and illustrate this difference with few examples
Read More

Large software organizations have the luxury to invest in labs, or test labs to help their test engineer better test a software. These labs could range from a setup having racks and racks of machines of various configs or could be just a big room with 10-20 machines. Having worked in a company (Legato Systems, now EMC) which creates Enterprise Backup Solutions (Legato Networker) I have been fortunate to see a lab which really had these large cupboard sized machines, fiber-optic powered SAN (Storage Area Network), really cool racks which support multiple machines with same keyboard, a display which would slide out and then go vertical. Probably it was done since it was needed. At the same time, out here at Adobe, we usually have a set up where a bunch of standard desktop machines are present running all kinds of operating systems, various locales and so on. Probably the reason we do not have racks is that our users do not have racks. Though I am sure that having racks might be more space efficient but thats for another day.

Some of these labs are powered by Image servers, the ones which can spit a OS image and do a raw-copy on local disk by booting the machine through network, while others may rely on OS installation through shiny disks. At some places, you have a check-in/check-out register for each of the machine, so as you get in you can work on a machine which is available and then as you go, you let go of the machine. I am sure that during your work experience you might have seen a different kind of lab as well. Also, some of you must have spent long nights in one of these labs trying to isolate a bug or just getting done with your part of test coverage. Sometimes these are also fertile grounds of new associations, more so since these are not too crowded, fall in a neutral zone (its neither harry’s office nor sally’s cubicle) and there are always many reasons to ask for help, there would always be something which is not working. Before your mind takes off and before you get nostalgic , let me come back to my intention of writing this post.

My intention here is to try to capture some of the best practices, best methods and general good housekeeping tips to harness most from a test lab. A happy test lab is key to success of test-case-execution and here are some things which you can employ to make your lab happy. These are not in any order but feel free to ask back incase it appears fragmented.

Five Things towards a Happy Test Lab

Read More

I was previously blogging at a blogspot site and over time I realized that I am getting limited in terms of tools there so I opted to do it on my own. Apart from other things which are needed to host and run a website, one of the basic initial things is to identify a name. And after being unsuccessful at getting a domain for the more common names, I was lost and dejected when suddenly I thought of validateverify, checked it and found that this is available.

Over time, I also realized that there are still a lot of people who are googling on these terms, to find their meanings, there differences and so on. While as a software testing practitioner and as a interview-holic person, ‘Validation and Verification’ are ingrained on my mind-slate, I can imagine that there would be lot of people who might just get confused. More so in places where English is not their first (and probably preferred) language.

So lets try to understand these terms and then see their relation vis-a-vis software testing.

Validation - Act of finding whether something is ‘right’ or not. The ‘Right’ is from the perspective of overall need and desire. Whether something is valid or not. 

Verification - Act of finding whether something does whats its supposed to do in defined (or may be undefined at the time of testing) condition. Verification is for built behavior, its for smaller nuances and not for the initial big goal.

Let me now take couple of examples to drive these points home.

Read More

With software getting larger and larger, the interaction between engineers who work on various parts of a software has been getting more challenging. While this is not really a big problem for development engineers, the ones who primarily write code, but its indeed a growing problem for ‘Test Engineers’, the ones who test a software and ensure that it does what its supposed to do for the end-user.

Let me take a couple of examples to explain this better. In first case, we would take the case of ‘MS Word’ which is a part of ‘MS Office’ family of products. In second case, we will take a much smaller product like ‘Adobe Reader’ (more popularly knows by its former name ‘Adobe Acrobat Reader’).

MS Word has tons of modules or components or ’section of codes’ which would have their own small teams. These may be File IO (Open, Save, SaveAs), Fonts, Print, Tables, Mail Merge, Formatting, Bookmarking, Help and so on. Some of these could be common across products, e.g. Fonts handling, Help etc. Each of these small teams will have their own test engineers and most of them test ‘their’ piece within a certain ‘Entry and Exit’ ideal boundary condition. Its always assumed that the guy who is before this node in the value chain would behave well and the guy who is next in chain would also behave well. Thats a wishful thinking.

Lets take an example

Enter Text -> Save File -> Select Text -> bold the text -> Undo bold -> Save -> Close.

In this workflow (a logical sequence of actions performed for a task), lets take the case of the team which is testing ‘Undo and Redo’ function.

Read More

Software Testing is still more in realm of ‘Arts’ than Science, especially when we are talking about testing software products,  something like MS-Office or  Mercury  WinRunner. Unless you are a part of a highly evolved and organized form of testing, in other words, a defined set of test cases on a defined set of test-bed to be executed, you can safely assume that you wont be able to find all the bugs. Being paid to find bugs and still not able to find all is an oxymoron arrangement and as Test Engineers and Test Managers, we need to live in the reality (of not being able to find all bugs) and still do business (find all bugs).

In this post, I am going to talk about a exercise which lots of organizations are started to get into their test-process as an effective tool for finding bugs, which typically act elusive to the ‘Test Engineer’ who is looking at a specific area. This exercise is called by different names like Bug Hunt, Bug Bash, Focus Day, Bug Harvest and so on. We would refer it as ‘Bug Hunt’, just a personal preference, they all work same / similar.

What is a Bug Hunt ?

A Bug Hunt is an intense bug finding mission over a small period of time, from couple of hours to a day, done by people who are not testing that area or that product as normal day-to-day work or may not be ‘testers’ at all, allows all kinds of bugs (some bugs are feature-requests) to be reported and is done to sort of give one big shake to an area or a product before its being shipped.

Read More