In our real case study, we needed a system that could create test data. This data would be used for different types of application testing. The system requirements stated that we needed to create a test data set introducing different types of analytical and numerical errors. Twelve different scenarios must be tested, and the data files must contain or be able to contain data that will allow these 12 tests to be performed. Additionally, the system needs to create different files that mimic the data sets or files submitted by clients. There can be up to eight different data sets or files. Each record in each file must have a correlation ID or primary/foreign key value to match and relate the records in the files. These correlation IDs can be persisted in a text file which the system will read and assign with the created output.

Next, the system must be able to create different quantities of records per file to mimic the number of transactions in the source system. The output of the system should be capable of stressing the end-user application by producing test files of varying sizes. The requirement for output is to be able to create files of 1,000, 10,000, 100,000 and 1,000,000,000 records.

Finally, the system should keep track of the number of records in each file, the time taken to create the output, the time taken to process, the number of errors created per output test file by the 12 different test types , the number of errors correctly captured by automated tests, and other business-specific metrics. Some of these data points will come from the agentic AI system and others will be generated by the automation testing system.