How to build custom redistributable test data generation solution?

DTM soft offers two solutions to build test data generation solution by the customer. The first, DTM Test Data Generation SDK, is suitable for professional developers how to use a high-level programming language like C++, C# or Visual Basic or scripting language like Java Script. The second solution designed for advanced users and does not required third party compilers at all. It is DTM Data Generation Script Complier. This article introduces and compares these solutions.

So, the SDK is a set of API functions grouped in three blocks: low level, fill methods level, and project level. The first level produces simply test values like a random date between two dates in the required format. The second layer offers middle-level functions like Value Library access. At the highest level located functions those works with whole projects. The programmer allowed mixing the function calls to any complex solution with own shell or GUI.

The script compiler operates with own language. It is easy to use descriptive language designed for data generation purposes. It allows users to describe database connection and how to populate any item of the database: tables and columns. The data generation language does not contain complex flow control instructions, user interface definition or input and output.

The important difference between mentioned solutions is initial overhead. DTM Data Generator can export ready to use data generation script. The user enabled to modify it before use as script compiler input. That means the script developer can use built in DTM Data Generator's Wizard to create an initial script within a few seconds.

At the other hand, Data Generation SDK usage requires manual infrastructure creation: the programmer should write all required library calls wrapped into language-dependent code. Also, low and middle-level functions use requires the manual creation of INSERT or UPDATE statements for the target database.

Let's analyze code fragment for generation elementary data item. We'll deal with random integer values. For the function call it is:

int iValue = IntRandom(10,100);	// generate integer between 10 and 100.

For the script language the user specifies pattern for data filling out:

Pattern="$RInt(10,100)"

You can't see important differences in this scope of view. However, if you up to "table" level we can see easy to understand script like:

Rule Type="data"
  Table Name="Test-Table"
  Rowcount Records="10000"
  TransactionSize="500"
  InsMode="replace"
  Fields
   Field Name="ID"
    Pattern="$RInt(10,100)"

For the C++ we have to:

• Establish database connection
• Remove all data from the table
• Generate value by mentioned API call
• Create INSERT statement for "Test-Table" based on this value
• Repeat it 10000 times with run COMMIT after each 500 rows.

The data generation script has no direct access to complete set of fill methods provided by DTM Data Generator. Each data generation rule should be described as a pattern of the pattern engine. For example, $File function should be used instead of "By File" fill method, $Query instead of "By SQL Statement", etc. In other words, it has only one level and it is "By pattern" fill method for all columns. When the user exports project file to data generation script DTM Data Generator converts all "native" fill methods and project options to patterns and script items. For example, a number of data rows to be generated will be converted to "Rowcount" item like in the sample script above.

But the flat structure of the script language creates a limitation. The user can't populate test database by one call as SDK function "RunProject". The script has a complete set of project properties and at least one data generation rule. It is complete description without references to the external project file.

By default, the compiler generates binary executable file only. However, the important feature of DTM Data Generation Script Compiler is C# source file output. This file can be used as a base for the complex solution with a user interface or additional functionality.

Data generation solutions comparison

In closing we created comparison table for both introduced technologies and tools:

^* - the compiler can produce C# source file that can be partially integrated into another application.