Data Driven Automated Testing

By: Michael Grater

There are times in software testing when you have a lot scenarios or features to validate but not a lot of time for testing. This may also involve running the same tests repeatedly over a period of time. In cases like this, I recommend using a method of test automation known as data driven testing.
For example, most site login forms have two fields and a button:

  • User ID field

  • Password field

  • Login button

Even with this simple configuration there are multiple variations that must be tested, such as:

  • Entering a valid user ID and invalid password

  • Entering an invalid user ID and valid password

  • Leaving the user ID field blank but entering a password

  • Entering a user ID but leaving the password field blank

A data driven test harness would feed pairs of user IDs and passwords (some valid, some not) to a testing script that would iterate through each set and capture the results for review. The user IDs and passwords could retrieved from a spreadsheet, flat file, XML file, or a database. Over time you could add specific data that have been known to cause problems or prove that code hasn’t broken. In this way your simple automated test transforms into an incredibly valuable site stability tool. Another clear benefit of data driven automated testing is that it frees up a tester so she can focus on other areas of the project that might not be easily tested with automated tests. The concept can be applied to any automation tool. At R/GA we’ve reaped great efficiencies from this approach with both Selenium and JMeter for a diverse set of our clients, including Nike. While it takes a little more time to configure, the long term payoff can be tremendous.