Information On Line Transaction Processing

989 Words4 Pages
ETL Overview Within an enterprise there are various different applications and data sources which have to be integrated together to enable Data Warehouse to provide strategic information to support decision-making. On-line transaction processing (OLTP) and data warehouses cannot coexist efficiently in the same database environment since the OLTP databases maintain current data in great detail whereas data warehouses deal with lightly aggregated and historical data. Extraction, Transformation, and Loading (ETL) processes are responsible for data integration from heterogeneous sources into multidimensional schemata which are optimized for data access that comes natural to human analyst. In an ETL process, first, the data are extracted from…show more content…
2. Incremental extraction: In this type of extraction only the changes made to the source systems will be extracted with respect to the previous extraction. Change data capture (CDC) is mechanism that uses incremental extraction. There are two physical methods of extraction: Online extraction and Offline extraction. Online extraction process of ETL connects to source system to extract the source tables or store them in a preconfigured format in intermediary systems e.g., log tables. In Offline extraction the data extracted is staged outside the source systems. Transformation The transform stage applies a series of rules or filters to the extracted data from to derive the data for loading into the end target. An important function of transformation is the cleaning of data, which process aims to pass only "proper" data to the target. one or more of the following transformation types: 1. Selecting only certain columns to load. 2. Translating coded values and encoding free-form values. 4. Deriving a new calculated value. 5. Sorting. 6. Joining data from multiple sources and duplicating the data. 7. Aggregation and disaggregation. 9.Turning multiple columns into multiple rows or vice versa. 10. Splitting a column into multiple columns. 12. Lookup and validate the relevant data from tables or referential files for slowly
Open Document