Enterprise Data Warehouse (EDW) should deliver unified, consistent information but usually doesn’t, due to conflicts in master data and a lack of common understanding of data sense (metadata). So, the typical complaints from IT and business users are:
“We have implemented the enterprise data warehouse and don’t need metadata management. Why does EDW deliver information of improper quality?”
“We have an enterprise Master Data Management (MDM) system in production. Why can’t we agree on data sense and terminology?”
“We have developed our company’s business glossary. Why do our business users still receive contradictive reports?”.
Strategy
These problems can be solved by the parallel execution of three strategic projects:
Metadata Integration
Enterprise metadata integration establishes a common understanding of data and master data sense.
Master Data Integration
Master data integration eliminates data and metadata coding conflicts in various information systems.
Data Integration
Data integration provides end users with data as a single version of the truth based on consistent metadata and master data.
This triple strategy provides effective interaction of data, metadata, and master data management systems, eliminates modules with similar functionality, lowers the total cost of ownership, and increases user confidence in EDW data.
The triple strategy allows the implementation of the agreed architecture, environment, life cycles, and key capabilities for data, metadata, and master data management systems.
As a rule, developers need to demonstrate at least an insignificant success in data integration quickly. Creating data, metadata, and a master data management environment is a high-priority task. But business users do not see immediate benefits from that environment. Therefore, the first phase should choose two or three pilot projects.
Process
Projects should provide minimum acceptable functionality of the future EDW. The project team needs to analyze pilot project results, adjust the data metadata tasks, and master data integration.
The second step is to choose new pilot projects to reach the basic functionality of the EDW. Data, metadata, and master data management environment must be developed enough to meet the requirements of basic EDW functionality. Project results should be re-examined after the completion of pilot projects of the second phase.
The next step should be the development of a fully functional EDW, which is impossible without comprehensive support by data, metadata, and a master data management environment. The EDW development project is not completed when EDW is delivered into production. If new systems can provide information important for data analysis across the enterprise, these new systems must be connected to the EDW.
To avoid integration issues, it is desirable to create new systems based on the capabilities of data, metadata, and master data management environment. Therefore, data, metadata, and master data management environment must evolve if the company and its IT systems exist, as indicated in the illustration by the arrows that go beyond the schedule. In turn, a data, metadata, and master data management environment should be changed according to the needs of new systems.
This article was contributed by our expert Sabir Asadullaev
Frequently Asked Questions Answered by Sabir Asadullaev
Q1. How can pilot projects be used to achieve the basic functionality of an EDW, and what is the process for analyzing and adjusting the results of these projects?
As a rule, developers must quickly demonstrate at least an insignificant success in data integration. The creation of data, metadata, and a master data management environment is a high-priority task. But business users do not see immediate benefits from that environment. Therefore, the first phase should choose two or three pilot projects. Projects should provide minimum acceptable functionality of the future EDW.
The project team needs to analyze pilot project results, adjust the data metadata tasks, and master data integration.
The second step is to choose new pilot projects to reach the basic functionality of the EDW. Data, metadata, and master data management environment must be developed enough to meet the requirements of basic EDW functionality. Project results should be reexamined after the completion of pilot projects of the second phase.
The next step should be the development of a fully functional EDW, which is impossible without comprehensive support by data, metadata, and a master data management environment.
Q2. What is the role of technology and infrastructure in developing a future EDW, and how can they be optimized for performance and scalability?
Many EDW models meet specific requirements: access speed, high data quality, reliable data protection, storage of large amounts of data, work with streaming, structured and unstructured data, performance, scalability, etc.
Many of these requirements cannot be achieved simultaneously. For example, high data quality requires collecting and analyzing all data, which slows down access to data. Therefore, choosing an EDW model is a compromise between different requirements.
There are many different technological and infrastructure solutions, the right choice of which will allow the implementation of the required EDW model within the allocated budget, resources, and timeframes.
Q3. What are the potential challenges or obstacles that may arise during the implementation of a future EDW, and how can they be addressed?
The EDW development project is not completed when EDW is delivered into production. To avoid integration issues, it is desirable to create new systems based on the capabilities of data, metadata, and master data management environment. If new systems can provide information important for data analysis across the enterprise, these new systems must be connected to the EDW.
In turn, a data, metadata, and master data management environment should be changed according to the needs of new systems. Therefore, data, metadata, and master data management environments must evolve as long as the company and its IT systems exist.
