Large Test Data Files Generation

Even the software uses database as a data storage it can use dummy text files for a few goals: import and export, configurations, logs, etc. Typically we deal with two kind of random text files: tabular and non-tabular.

The tabular file has columns and rows without any extra information. In other words it is a simple text copy of the database table. It is a way of information exchange between legacy applications. Also, it is right way to have human-editable content without custom applications like database editor.

DTM Flat File Generator was designed to generate extra-large test data files. It offers great performance and easy to understand due to built-in and predefined data generators for most useful data: numbers, dates, names, cities, addresses, etc.

However, DTM Flat File Generator is not suitable for non-tabular structures or structures with extra items, nested items or complex structures. DTM File Factory was designed for similar cases. It allows users to define template that defines target document structure. To create test file the generator changes template's items to random values, includes optional parts, repeats information blocks. Also, this test file generator has limited binary-files support.

From time to time we recommend our customers to use the file generator for database population instead of full-featured DTM Data Generator. Why?
There are main reasons:

• There is no direct access to database by technical or business reasons: no required connector, security, etc. In this case generated data can be loaded locally by native import tool of the database system.
• Performance for single extra large table. For single table sequential execution of flat file generator and native import can work faster than direct data loading due to transaction and triggers disabling.
• Customer's database is tool simply and has no relationships and dependencies for automatically resolution and filling in.
• Manual preparation is required before data loading, e.g. custom transformation, a few file combination.

See Also

• DTM Data Generator, the tool for test database generation.
• DTM Data Generator for Excel, the software for test Excel spreadsheet creation.
• DTM Data Generator for JSON, the utility for test JSON or BSON document population.