MyTechFinds.com

01.Ability to run more than one test. What does this really mean? The tool you are about to select should have some in-built facility to let you run a bunch of test cases together (one-after-another to be specific). If there is no such inbuilt facility in the tool (which is rare), there should be some freedom to let you code your automation in such fashion that you achieve this goal.

02.Ability run subset of test cases. The tool should provide a facility by which you can classify different type of tests, may be based on type, feature or whatever you like. Again, once you have done the classification creating small bunches of tests, it is obvious that you want to run them together satisfying the first requirement.

03.Unattended execution. Sometimes I feel this is asking for too much from an UI driven automation tool. But it depends how you want to read it. I read it as some way to schedule my test runs with minimal configuration, plus once kicked-off, it should be self contained to proceed on different situations it might encounter during that run. Now, the later part is actually highly dependent on "how" your automated testing is designed and how well the coder has implemented it. Yeah, it will need some of your intelligence to be smart! Just record-playback is not going to give you a robust system.

04.Ability to capture execution results. A good automation tool should always be able to tell you what just happened. As we are expecting it to be unattended, we want the tool to tell us what happened at the end of each test case (at least) if not log results of each step of execution. So, when the pass is over, the tester can look at the logs to figure out which tests passed and which failed and then take appropirate action. Most of off the shelf tools come with good logging facilities, but again, you will have to be smart to design and code your tests to extend the tool's logging capabilities.

05.Ability to compare "expected" vs "actual" results. So we want logging capabilities in the tool and we also want the tool to be able to compare results to a baseline so that we can make a decision on calling a particular step or even a test case to be passing or failing. This is probably the most important "ask" that you should have when looking out for a tool. You also have to ask deeper question of what all it can compare - text?, numbers?, images? etc. You will find these specific questions looking at the software under test. It is needless to say even if the tool can compare two values, it will be your responsibility to code it right to make it able to take the right decision!