Preparing Data Subsets

Data warehouses contain a large number of records. It is computationally expensive to process all the data. The data should be organized into batches or samples. W. Edwards Deming wrote a book entitled "Sample Design in Business Research". Just grabing an arbitrary collection of data can lead to incorrect results.

Two batches of data are selected for special interest. One batch is selected as the "preliminary batch". The preliminary batch of data will be used to develop the starting values for the model. Another batch will be specified as the "test batch". The test batch will be used to check the model for accuracy. The remaining batches are used to refine the model.

JSM Software can help your business properly subset your data into batches.