Comments on: How to create a optimized testbed-configuration matrix http://validateverify.com/2008/08/how-to-create-a-optimized-testbed-configuration-matrix/ There is always one more bug Tue, 06 Jan 2009 00:43:47 +0000 http://wordpress.org/?v=2.5.1 By: Nandan Jha http://validateverify.com/2008/08/how-to-create-a-optimized-testbed-configuration-matrix/#comment-134 Nandan Jha Mon, 11 Aug 2008 06:01:38 +0000 http://validateverify.com/?p=28#comment-134 Thanks Avinash for reading and sharing your thoughts on this. Probably my post was not very clear but if I have to test a s/w as you mentioned and if Vista+IE8 doesn't fall under 1 and 2 (default and 2nd default) then I would spend my testing time on this combination through 4 and 5. As you could imagine, a mass market desktop software probably would end up running on every possible combination and in reality you wont have time and resources to test on every thing so you start optimizing. I understand that in case of Enterprises , one would need to certify different configs because it may happen that a big client is using one of the configs and he wont change his h/w because of you. About 8 years back, I was working on an Enterprise Backup software and we had same issues. We did use to fall back on standards to save ourselves from certifying every thing. Essentially if a server supports HTTP then I wont bother too much on whether its a IIS or Apache. I would do most of my testing on most common and 2nd most common and then utilize 3 and 4 activities for certifying other configs. Thanks again, probably your comment is the exact reason, I wrote this up. In my personal opinion, we still have to perfect this art and we are still learning. Before we reach there, we would end up doing much more then necessary to be safe. Thanks Avinash for reading and sharing your thoughts on this.

Probably my post was not very clear but if I have to test a s/w as you mentioned and if Vista+IE8 doesn’t fall under 1 and 2 (default and 2nd default) then I would spend my testing time on this combination through 4 and 5.

As you could imagine, a mass market desktop software probably would end up running on every possible combination and in reality you wont have time and resources to test on every thing so you start optimizing.

I understand that in case of Enterprises , one would need to certify different configs because it may happen that a big client is using one of the configs and he wont change his h/w because of you. About 8 years back, I was working on an Enterprise Backup software and we had same issues. We did use to fall back on standards to save ourselves from certifying every thing. Essentially if a server supports HTTP then I wont bother too much on whether its a IIS or Apache. I would do most of my testing on most common and 2nd most common and then utilize 3 and 4 activities for certifying other configs.

Thanks again, probably your comment is the exact reason, I wrote this up. In my personal opinion, we still have to perfect this art and we are still learning. Before we reach there, we would end up doing much more then necessary to be safe.

]]> By: Avinash Kumar Gupta http://validateverify.com/2008/08/how-to-create-a-optimized-testbed-configuration-matrix/#comment-133 Avinash Kumar Gupta Mon, 11 Aug 2008 05:41:48 +0000 http://validateverify.com/?p=28#comment-133 I agree, but to an extent only. Most product companies need to print on the box - what all does this software support. Taking your example of photo site might not be the best here but let me take example of a photo sharing software which resides on the desktop. A company who builds this product needs to put the list of supported OS, browsers, locales etc on the box. If Testing dept of that company hasn't tested on say Vista (or IE8), this company can't afford to put Vista or IE8 on the box as supported environment. A customer who is using IE8 and Vista, when walks into the store to buy this software, would first read abt the supported software and would return the box back to the shelf not finding both of them there. Not to mention, this whole scenario gets magnified to a different level when it comes to enterprise software. Million dollar deals could be lost just because your software didn't support a particular version of browser or application server. While 80-20 works for bugs, but then there are other criteria in the business world which bind testers to follow the matrix market defines... I agree, but to an extent only. Most product companies need to print on the box - what all does this software support.
Taking your example of photo site might not be the best here but let me take example of a photo sharing software which resides on the desktop. A company who builds this product needs to put the list of supported OS, browsers, locales etc on the box. If Testing dept of that company hasn’t tested on say Vista (or IE8), this company can’t afford to put Vista or IE8 on the box as supported environment.
A customer who is using IE8 and Vista, when walks into the store to buy this software, would first read abt the supported software and would return the box back to the shelf not finding both of them there.

Not to mention, this whole scenario gets magnified to a different level when it comes to enterprise software. Million dollar deals could be lost just because your software didn’t support a particular version of browser or application server.

While 80-20 works for bugs, but then there are other criteria in the business world which bind testers to follow the matrix market defines…

]]>