Data Exploration

After the data has been validated and cleaned the process of checking the integrity of the database is called exploring. For example, the state field is only two letters. Validation would check the state field was populated with letters. Cleaning would remove any record where the state fields contained a sequence that did not match a state name, e.g. AA. Exploring looks at all the values to see if there is a problem. One example is that the data contained only records from CA, PA, and GA. Exploring is basically checking to see that for each field with an enumerated value that all the possible enumerations are used, and that the numbers of records with each enumerated value match the expected distribution. For example, if 90% of the records were from CA and the other 10% contained records from the other states then there is probably a problem.

JSM Software can help your business explore your data.