1、 Processes of Data Integration XMLs use in Data Integration Table of ContentsTable of Contents 2I. Introduction 3A. Differentiating Data Integration from Application Integration (EAI) 31. Each Technology solves different business problems 32. Each Technology requires different tools 4B. Factors In D
2、etermining Which Technology To Use 41. Limitations of Application Integration (EAI) 42. Strengths of Data Integration 5C. Guidelines for Selection 6II. “Actual” Data Integration (ODS Based) 7A. Data Integration using an Operational Data Store (ODS) 7B. ODS Components & Processes 81. Components 82. P
3、rocesses 9III. “Virtual” Data Integration (XML-Based) 11I. IntroductionIntegrating applications is simple from a communication perspective, but it can be extremely challenging from a data perspective. Multiple applications can be made to send and receive messages and transactions among each other; b
4、ut if they do not have a common understanding of the context and meaning of the data involved, the result will be incomplete, or inaccurate, information in one, or more, of the applications. Therefore application integration (EAI) should not be implemented as an isolated technology but rather as par
5、t of a broader integration strategy that evaluates which type of integration (data or application) is appropriate for which task. Even though application integration is getting more attention today, the need for data integration is also growing rapidly, driven primarily by e-business and portal requ
6、irements. This paper will explore the critical data integration issues that need to be evaluated as part of a middleware-based application integration strategy, and guidelines for when migration to a centralized data repository for housing key elements of enterprise information is preferred, rather
7、than an EAI middleware solution, as well as the tools and standards to do so.A. Differentiating Data Integration from Application Integration (EAI) The key difference between application integration and data integration is that application integration enables real-time sharing of data between differ
8、ent systems and/ applications. Data integration requires the combining of data from disparate sources into a new, consolidated data resource. Application integration, which is the creation of new strategic business solutions by reusing the functionality of existing applications, involves the use of
9、EAI middleware to connect disparate systems and/or applications. EAI middleware enables one application, or database, to communicate with another application, or database, but does not require any change to the existing data in the underlying databases. Data integration involves the use of ETL middl
10、eware to reduce data redundancy by collecting and reorganizing disparate data into one physical or logical place. Integrated data repositories are the alternative to EAI, which present a centralized and logical approaches to integrating information. The physical implementation of data integration co
11、uld be within a centralized enterprise data warehouse or several logical data repositories (ODS). 1. Each Technology solves different business problems There are many requirements for EAI, from business-to-business (B2B) to internal application integration scenarios, however different integration sc
12、enarios require different integration technologies. Application Integration (EAI)The business driver behind application integration, (the creation of new strategic business solutions by integrating the functionality of existing enterprises applications) is the recognition that stove-piped applicatio
13、ns, typically transaction processing (OLTP), automate individual steps in a larger business process. By capturing output from one system and routing it into a receiving application, a broader business process is automated. Three critical attributes of application integration are:1) participating sys
14、tems are tightly dependent 2) integration is done at the application level to preserve transactional integrity 3) both systems are process centric in their design. Data Integration The goal of data integration is to get redundant data, stored in multiple independent systems, to agree on the facts, s
15、ince there is no practical way to re-engineer the applications and eliminate the redundancy. In todays environment, data consistency is important for both OLTP and DSS architectures. When data consistency is the sole motivation for integrating previously independent applications, then data integrati
16、on technology, rather than EAI technology should be used. Three critical concepts are:1) participating systems are loosely coupled 2) integration is done at the database level, and 3) the receiving system is data centric in its design 2. Each Technology requires different toolsEAI tools are the most
17、 appropriate integration technology for application integration. Within EAI tools, logic and business rules are developed that transform the syntax and semantics of the sending applications data, messages or transactions into inputs that are semantically and syntactically consistent with those of th
18、e receiving systems. ETL tools are more appropriate for data integration. Data must be complete, consistent, timely and relevant to create information quality. Therefore, data originating in two or more applications must be reconciled into a semantically consistent format. ETL tools integrate data f
19、rom multiple systems, reconciling redundant and overlapping data into a single, consistent integrated data structure and enabling syntax and semantic transformations to be applied to the data. Although ETL tools and EAI tools have legitimate functional overlap, they feature different technical stren
20、gths specific to the problems they solve. Following are the key differentiating features between them. 1. The first is the time requirement. - If real-time, or near-time, update is required, then an EAI tool will satisfy the requirement better than an ETL one. 2. The second is the volume of data to
21、be moved. - An ETL batch process will always be more efficient at moving a large volume of data than individual messages. 3. The third is the level at which integration needs to occur. - Messaging brokers tend to have a programming API which results in system integration at the application programmi
22、ng level. 4. The fourth is the need for intelligent routing. - ETL vendors do not currently provide intelligent, or dynamic, routing of transactions. 5. The fifth is the level of metadata support - Both EAI and ETL make use of metadata however, EAI vendors have not offered as rich an approach to met
23、adata as the ETL vendors. B. Factors In Determining Which Technology To UseUsing EAI middleware to link disparate information sources for certain types of applications should not be viewed as an “quick fix” alternative to building and maintaining an integrated data repository. In many cases, adding
24、new applications to installed applications introduces inefficiencies or disconnected islands of information1. Limitations of Application Integration (EAI) Using EAI middleware to link disparate information sources can have an impact on data consistency and/or quality, which can create the following
25、problems: Data Redundancy. Separate databases often maintain duplicate or near-duplicate data, incurring increased storage and management costs. Process Redundancy. The middleware approach requires that data be transformed into the requesting applications format each time it is accessed. . Synchroni
26、zation and Ownership Issues. These problems may result from varying source system availability, as well as from the logical repositories receiving new and updated data from a broad range of sources . Data Inconsistency. Serious inconsistencies may result from the source systems employing different s
27、emantics, formats, periods of applicability and cycles of update and refreshment. Limited Repeatability and Auditability. Changes in the source systems operational data may cause the same action to deliver different results at different times. 2. Strengths of Data Integration There are several types
28、 of data integration scenarios, where building an integrated data resource may be preferred over an EAI solution: To Solve Data Integrity Issues Between SystemsEAI enables integration in a broad, business context and can serve as a foundation for integrating legacy, as well as, new applications, how
29、ever it is unlikely that an application integration project will succeed, no matter how sophisticated the EAI tools, if the underlying systems are a mess (i.e. applications with eccentric data, unique business rules, and convoluted processes) these systems may never be usefully integrated. Not all a
30、pplication integration projects will require integrating such disparate heterogeneous data sources to require a data integration solution. However, for integrating portals, commerce servers or custom applications to two or more relatively complex and disparate, heterogeneous, data sources, a data re
31、pository should be considered as a way of not only reducing initial effort, but also creating leveraged reusable data access components. To Transform & Cleanse Disparate Data Most organizations are faced with managing large quantities of disparate data. This disparate data severely impacts both the organizations ability to perform its business activities
copyright@ 2008-2022 冰豆网网站版权所有
经营许可证编号:鄂ICP备2022015515号-1