A Bibliographical Study on Importance of Data Profiling and Data Mining for Effective Business Transactions; a Techno-Business Leadership Perspective

Data profiling is performed several times and with varying intensity throughout the data warehouse developing process. (Kimball et al). The light profiling assessment undertaken immediately after candidate source systems is identified and DW/BI business requirements are assessed till it is satisfactory. The purpose of this initial analysis is to clarify at an early stage, on the correctness of the data, and the detail are appropriate to the level required and that anomalies are comfortable enough to be handled subsequently, if it is otherwise, the project may be terminated. In addition, more in-depth profiling is done prior to the dimensional modeling process in order to assess what is required to convert data into a dimensional model. Detailed profiling extends into the ETL system design process in order to determine the appropriate data to extract and which filters to apply to the data set. Additionally, data may be conducted in the data warehouse development process after data has been loaded into staging, the data marts, etc. Conducting data at these stages helps ensure that data cleaning and transformations have been done correctly and in compliance of requirements.