Master Data Management: Consensus-Driven Data Definitions for Cross-Application Consistency (Report Excerpt)

Many data-management professionals and their business counterparts are asking: "What is master data management? Why should I care? Why is it imperative now?" This report seeks to answer these and related questions.

Overview of Master Data Management

For many people in IT, this decade is all about integration. It’s about integrating customer data, integrating application silos, integrating data for BI purposes, integrating with partners, integrating with governments, and integrating through new interfaces like Web services. As if the list weren’t long enough, the practice known as master data management (MDM) has become a candidate for even more integration work.

Hence, a lot of data-management professionals and their business counterparts are asking: “What is master data management? Why should I care? Why is it imperative now?” This report seeks to answer these and related questions.

What Is Master Data Management?

We need to build up to an answer to this question by describing (in procedural order) the technical components and best practices that are subsets of master data management:

• Master Data. Master data (sometimes called reference data) consists of facts that define a business entity, facts that may be used to model one or more definitions or views of an entity. Entity definitions based on master data provide business consistency and data integrity when multiple IT systems across an organization (or beyond) identify the same entity differently.
• Business Entity Definitions. Today, the business entities modeled via master data are usually customer, product, or finances. But master data can define any entity, like employee, supplier, location, asset, claim, policy, patient, citizen, chart of accounts, and so on.
• System of Record. Common across the many approaches to MDM is the creation (or selection) of a system of record (sometimes called a trusted source). The point is to establish a central, authenticated master copy from which entity definitions (and perhaps physical data, too) are propagated among all IT systems integrated via the MDM solution.
• MDM Hubs, Master Files, and Application Databases. The system of record can take many forms. Many users build a central database (like a data warehouse or operational data store) as a hub through which master data, metadata, and physical data are synchronized. And some hubs are simply master files or master tables that collect and collate records. Sometimes a pre-existing application (typically for ERP or CRM) has the needed definitions already, so it’s selected as the system of record and the basis for entity modeling elsewhere.
• Master Data Integration. Regardless of the technology approach, the goal of the system of record is to provide a central mechanism for collecting and sharing consistent definitions, usually across otherwise unrelated IT systems. Obviously, this requires technologies and best practices for system integration, data integration, and application integration. Hence, many technical users consider MDM to be an integration practice, enabled by integration tools and techniques for ETL, EAI, EII, and replication. When the system of record is a hub that connects many diverse systems, multiple integration technologies may be required, including newer ones like Web services and service oriented architecture (SOA).
• Master Data Management. Drawing from the above definitions, we can now say that MDM is the practice of defining and maintaining consistent definitions of business entities, then sharing them via integration techniques across multiple IT systems within an enterprise and sometimes beyond to partnering companies or customers. More simply put: MDM is the practice of acquiring, improving, and sharing master data.

MDM Solution Types. MDM solutions in most organizations are built into or closely associated with a larger application, though some span multiple applications. Due to these associations, MDM solutions fall into three broad categories (see Figure 1):

• Operational MDM. Upstream in the general flow of data, one or more MDM solutions are built into and/or used to integrate operational applications for enterprise resource planning (ERP), customer relationship management (CRM), supply chain management (SCM), financials, and so on. Since the applications don’t change frequently, master data and entity definitions don’t either. Furthermore, these applications and their business processes support transactions and sometimes operate in real time, so operational MDM must, too.
• Analytic MDM. Downstream in the data flow, data warehousing has long involved some form of MDM, because of the balance between tracking data lineage (to ensure you have the right data) and repurposing data to create new structures (like aggregates and time series). Entity definitions change often due to data discovery, analytic business modeling, and other iterative practices. Analytic MDM is also seen in practices that resemble data warehousing, like customer data integration (CDI) and financial performance management (FPM).
• Enterprise MDM. Today, MDM is practiced mostly in isolated silos or with a short list of applications that don’t step beyond the bounds of either operational MDM or analytic MDM. However, some organizations have moved to the next level with enterprise MDM, which is an autonomous infrastructure that can integrate master data across multiple IT systems and businesses. Spanning the whole data flow is daunting, because enterprise MDM must satisfy the diverse requirements of both operational MDM and analytic MDM. Yet, enterprise MDM is a worthy goal, because it extends beyond IT silos and organizational boundaries the general benefits of MDM, namely: well-designed entity definitions applied consistently.

Enterprise MDM and data flow

Figure 1. MDM solutions can be upstream or downstream relative to the flow of data.

Why Should We Care about Master Data Management?

The main reason that technical and business people should care about MDM is the long list of problems that occur when it’s ignored. Somewhat less compelling is the short list of benefits that come from improving master data and its integration across IT systems.

The Problems of Poor Master Data. In an Internet-based survey that TDWI ran in mid-2006, a whopping 83% of respondents reported that their organizations have suffered problems due to poor master data. (See Figure 2.) Hence, the vast majority of users attests that MDM problems are real, numerous, and severe.

Figure 2. Based on 741 respondents.

Poor-quality master data creates problems mostly within data warehousing and BI—but also outside it. (See Figure 3.) The top three problems relate directly to data warehousing, namely inaccurate reporting (81%), arguments over which data is appropriate (78%), and bad decisions based on incorrect definitions (54%). Others are general data management problems that sometimes impact data warehousing, like data governance and stewardship limitations (53%), limited visibility for data lineage (52%), and no understanding of master data homonyms (46%). Other problems cited by survey participants are business problems, like poor customer service (35%), inefficient marketing (32%) or purchasing (18%), and new products delayed (17%).

Figure 3. Based on 2,921 responses from 615 respondents.

The Benefits of High-Quality Data. Though a vast majority of users have suffered problems from poor master data, a much slimmer majority (54%) claims to have derived benefits from good master data. (See Figure 4.) Still, this indicates that benefits are possible and can be identified.

Figure 4. Based on 741 respondents.

As with master data problems, the benefits likewise relate most strongly to data warehousing and related data management practices, followed by general business issues. (See Figure 5.) Near the top of the list are data warehousing and BI issues, like accurate reporting (75%), better decision making (69%), and easier auditing of information’s origins (47%). Mixed with these are general data management issues, like data quality (76%), consistent definitions (65%), and data governance (39%). Low-ranked benefits are mostly general business issues, like risk reduction (33%), superior customer service (21%), and supply chain optimization (15%).

Figure 5. Based on 2,224 responses from 402 respondents.

Good MDM Yields Good Reports. Users can clearly see the impact of MDM—both negative and positive—on visible products of information management, especially reports of various types. For example, in TDWI’s survey, inaccurate reporting is the leading problem and accurate reporting is the second leading benefit. The survey aside, most of the users that TDWI interviewed pointed to reports as an area where both technical and business people seek improvement via MDM.

Though all reports may benefit from improved MDM, regulatory and financial reports are a hot spot, because they are scrutinized carefully today and can result in dire consequences when discrepancies are found. In fact, many interviewees admitted that they and their peers live in fear of an audit, and that they feel MDM can help them avert or prepare for such an event. For example, the consistently applied definitions of MDM ensure that reports are populated with correct data, and the data lineage of MDM answers questions in the event of an audit. Similar concerns are seen in the survey, where roughly half of respondents reported “limited visibility for data lineage” as a problem and “easier auditing of information’s origins” as a benefit. Despite its negative impetus, “audit paranoia” is a significant driver for increased efforts in master data management, as well as related areas like data quality, data warehousing, and report design.

Why Is Master Data Management Imperative Now?

Trends unique to this first decade of the new millennium have brought MDM to the forefront:

• Technology Trends. As an integration practice, MDM is late to the data integration frenzy of this decade, yet still part of it. For many technical users, MDM is an extension of other growing practices, like data warehousing, BI, data quality, and metadata management. In some cases, MDM’s central definitions are part of a general trend toward centralized IT.
• Business Trends. After Enron and WorldCom, many corporations have entered an age of accountability, where executive actions are driven by audit paranoia, which MDM helps alleviate. Market consolidation is rampant in some industries (banking, high tech, telco), and MDM can be a useful component in the quick-but-critical integration of customers, financials, and products demanded of a merger, acquisition, or corporate reorganization.
• MDM Trends. Although MDM has long existed in a few isolated silo applications, companies now practice it in more silos and with more connections among silos. Forward-looking companies even practice enterprise MDM, with a broad scope across many applications and businesses. As with many data management practices, the general trend in MDM is toward broader and better integrated enterprise scope.

This article was excerpted from the full October 2006 report. TDWI appreciates the sponsorship of Actuate Software, ASG, Business Objects, DataFlux, Informatica, SAP, Sunopsis, and Teradata.

To download the full report, visit www.tdwi.org/research/reportseries.