Optimizing the Benefits of EDM and SOA by Coordinating Strategies

Organizations often pursue strategies for enterprise data management (EDM) and service-oriented architecture (SOA) as separate programs and initiatives. However, there are important overlapping and interdependent components, processes, and quality checkpoints in which coordination is necessary to ensure the success of either strategy.

Building on the main points discussed in part one of this article, "The Case for Coordinated EDM and SOA Strategies" in October's SOA World Magazine, we'll introduce the Coordinated Service-Oriented Data Architecture (C-SODA) framework as an effective and flexible tool for assessing/driving coordination between EDM and SOA strategies and initiatives. The C-SODA toolkit also includes an EDM/SOA capability maturity model (CMM) toward progressive organizational maturity in gaining benefits from coordinated strategies.

While this framework and CMM do not preclude the need for more specific EDM/SOA programs, it does:

Organizations should consider:

For many organizations these days, regardless of industry, the answer is usually an emphatic "Yes." If so, the transactional and services data and metadata associated with an SOA strategy should be considered for data governance and management, and possibly as master data, in a broader EDM strategy.

Information quality management is a major component of SOA.

Key EDM and SOA Strategy Coordination Points
To help different organizational stakeholders better understand the strategy, EDM/SOA coordination points are presented separately by EDM and SOA perspectives. If a particular group is coordinating these from either perspective, they can use the appropriate list to guide EDM/SOA initiatives.

From the EDM Perspective
Key coordination points for an EDM - SOA strategy program include:

  1. Data governance and stewardship with SOA governance and services stewardship
  2. Organizational roles, processes, and strategic program/project decisions for SOA-related master data and metadata
  3. IT roadmap releases for interrelated services and data
  4. Master data with SOA services or the Enterprise Services Bus (ESB) that use/distribute this
  5. Enterprise Data Model with SOA services portfolio and releases
  6. Enterprise Information Architecture (EIA) releases with SOA services releases
  7. MDM and metadata management with SOA services' initiatives architecture and design
  8. EIA, MDM, and metadata management processes/tools with SOA services' initiatives design processes/tools
  9. Organizational processes and roles for EIA, MDM, and metadata management with SOA services architecture processes/roles
  10. EIA, MDM, and metadata management with SOA initiatives' enterprise services development teams
  11. MDM releases with SOA services portfolio/releases
  12. Metadata management with SOA Services Model and SOA-specific metadata (e.g., Web Services Description Language [WSDL], ESB configuration, and security settings)

From the SOA Perspective
This perspective is intended to help SOA-focused stakeholders better understand the SOA-EDM coordination points, including:

  1. SOA governance and services stewardship with data governance and stewardship
  2. IT roadmap releases for interrelated services/data
  3. SOA services portfolio and release management with EIA and Enterprise Data Model
  4. SOA services releases with EIA releases
  5. SOA services' designs with master data and MDM
  6. SOA service initiatives' designs with MDM and metadata management
  7. SOA services with metadata and metadata management
  8. SOA components' metadata (e.g., services' WSDL, ESB configuration, and security settings) with metadata management
  9. SOA services' initiatives design processes and tools with EIA, MDM, and metadata management processes/tools
  10. Enterprise architecture releases, including technical architecture supporting ESB and integration points to data stores and consumers as well as to EIA and its database references
  11. SOA initiatives' services architecture/development with EIA, MDM, and metadata management
  12. Initiative-level SOA services architectural processes and roles with those for EIA, MDM, and metadata management

Additional COE or ICC Coordination Points
In mature EDM/SOA environments that have Center of Excellence (COE) or Integration Competency Center (ICC) capabilities, additional coordination points for EDM/SOA programs are:

With a more enlightened SOA approach that is consistent with EDM objectives, services and data are reused. Hence, another reason to coordinate these strategies is to facilitate these natural synergies.

Coordinated Data - SOA Governance as a Linchpin
Besides strong dependencies between EDM/SOA strategies that alone justify coordination, there are significant opportunities for economies and the ability to manage each strategy more effectively when jointly governed.

SOA governance is the foundation for key coordination points with EDM at both the strategic (e.g., MDM and EIA) and tactical (e.g., service-data stewardship, metadata management) levels. Governance of both services and data is the primary linchpin for coordinating both strategies and underlying processes, quality checkpoints, and artifacts.

Early in establishing coordinated data-SOA governance, both services/data should have stewards designated through their respective governance programs. Service-data relationships form virtual stewardship "communities." Service stewards will employ the appropriate data stewards (i.e., for data utilized by services) in requirements, designs, testing, and services evolution. Coordinated data-SOA governance processes/roles manage these relationships.

Significant MDM & SOA Dependencies & Synergies
Master data can help break down operational silos, which is a natural synergy with the SOA mission to do the same via cross-domain services and their reusability.

Master data is the consistent set of identifiers (data) and extended attributes (metadata) that describe core enterprise entities used across business processes/services. Master data is not all your data, just the subset required for sharing and standardization. By definition, it's changed infrequently and is referenced by business processes, transactions, and reusable services. This is the undeniable linkage between master data and MDM to SOA strategies/activities.

The scope of master data includes core subject-area elements, maintained in a referenceable repository. Additionally, standard (e.g., SOA) services for manipulating/reusing data objects are developed as part of MDM change management and governance processes.

When organizations use master data at an enterprise level, the need for separate departmentally maintained "versions of the truth" are alleviated. Although users often say they want a "single view of data," what they really want are multiple views for stakeholders. This is where metadata relates directly to MDM, where referenceable interpretations of master data to the "gold standard" source can be managed for stakeholder views/services.

There are also related impacts on real-time transformations between data stores and users, generally requiring metadata transformations of "in-flight" data. Hence, progressive SOA capabilities raise new data management issues beyond those addressed by traditional EDM, MDM, and data governance for "static" data.

SOA naturally exposes data issues to more people, processes, and integrated systems. A focus on EDM is needed when in support of an SOA. Through EDM, and primarily MDM, organizations achieve consistency, accuracy, and integrity of information assets in support of key strategic initiatives, such as the successful migration toward an SOA environment.

What MDM Can Learn from SOA
MDM and SOA share similar design principles. For example:

Additional SOA paradigms can contribute to the maturity of MDM. For example, extending MDM with "loose coupling" (i.e., a federated approach) allows support for SOA's semantic conformance needs and can create an agile MDM system. Such a service-oriented MDM system provides data quality, conformance, and other MDM functionality as business or data services, enabled for human, application, or business service consumption.

MDM systems should also be configured to handle extensible data types (XML, HTML, PDF, etc.) to expose the Master Data Model for services consumption. Architects should develop unifying schemas for merging content types; with XML's increasing adoption as the standard for information exchange, barriers to content convergence (e.g., via business or presentation services) can be mitigated.

EDM and SOA are integral parts of the same EA puzzle and neither can mature successfully without the other. As with the many dependencies and synergies for simultaneous implementation of both these strategies, a common framework and maturity model can lend itself to the evaluation of organizational readiness as well as to the planning of roadmap initiatives for these strategies in a coordinated fashion.

Why SOA Needs MDM
While SOA enables integration and data exchange through services, this is only marginally useful without a common vocabulary of data content/structure. MDM defines how an enterprise establishes and maintains such a vocabulary, so to fully adopt/implement a SOA program an organization must first address MDM.

Before creating service-oriented applications, align master data and metadata for SOA designs. Without a focus on MDM, it becomes impossible to communicate information about transactions because there's no common understanding of the basic business objects to which the services refer. Services don't know where to access "gold standard" information, which has to be the same (structure/content) between producers and consumers of services.

As applications interact with data through multiple levels of services, impacts of even small data structure changes may be significant. Hence, coordinated EDM with SOA should first be instituted as coordinated data-SOA governance with MDM processes. The technical intersection of MDM and SOA occurs in the EIA. MDM is a key component of the EIA, providing semantic integration of services for master data. For services to provide consistent information to consumers across multiple data providers, it's imperative that data inconsistencies/redundancies are addressed.

C-SODA Framework & CMM Overview
Both the framework and CMM introduced here are based on a Coordinated Service-Oriented Data Architecture (C-SODA). As implied, the C-SODA is built on data architecture; in this case, the data architecture aspects that support a service-oriented environment.

The C-SODA complements full-fledged EDM/SOA frameworks and maturity models by specifically identifying dependencies and synergies as well as evaluation criteria and maturity phases of these coordinated strategies. The C-SODA is used to evaluate and drive an organization's strategic readiness for coordination along the framework's dimensions in both EDM and SOA domains.

To be successful, EDM and SOA programs require careful planning and execution along several interdependent dimensions. Further, when these programs are to be executed in parallel, coordination between EDM and SOA concerns is required to promote success.

We utilize a proven framework to evaluate seven critical dimensions (see Figure 1) that determine the strategic readiness of an EDM/SOA or C-SODA program. Key questions to ask about each dimension area are:

These dimensions provide the framework for which coordinated EDM/SOA strategies can be evaluated and/or a roadmap of initiatives can be formulated to improve organizational capabilities and maturity. Determination of an organization's strategic readiness for combined EDM/SOA capabilities along each dimension will help gauge overall maturity. This includes some standalone EDM and SOA capabilities, but further focuses on the synergistic nature of these strategies and their dependencies and coordination points.

When considering the C-SODA framework, use the following guidelines:

  1. Not all dimensions carry the same priority for coordinated EDM/SOA strategies. To promote maturity along necessary coordination/integration points of the strategies, process, data, services/applications, and architecture are considered primary dimensions; strategy, metrics, and people are considered secondary dimensions.
  2. Primary dimensions directly impact decisions and approaches for coordinating strategies, while secondary dimensions are used for context and in support of coordinating primary dimensions.
  3. Primary dimensions are addressed in greater detail and emphasis for evaluation and/or formulating a roadmap of initiatives to achieve greater maturity.
  4. Primary/secondary designations for dimensions (and components) may be adjusted for an organization's needs

For example, the strategy dimension, meaning "business strategy," generally only has indirect secondary impacts on services/data. By addressing process (e.g., data-SOA governance, MDM, metadata management, services-data stewardship), data (e.g. master data, metadata, reference data), services/applications (e.g., SOA services portfolio, related metadata, service initiatives' designs/tools), and architecture (e.g., SOA and data infrastructure/tools) in detail during evaluation or roadmap development, we have a sufficiently complete picture of how to establish/evolve the organization's coordinated EDM/SOA capabilities.

Utilizing the C-SODA Framework
Fine-tuning the framework for your organization, initially and as your EDM and SOA strategies mature, involves:

  1. Determining primary/secondary dimensions and dimensional components for consideration (see Figure 2). This will be your organization's/program's C-SODA.
  2. Applying C-SODA to evaluate current state of coordinated EDM/SOA strategies. It must be:
    a. Detailed enough to define granularity of capabilities and potential initiatives for improved maturity
    b. Able to rate each dimension/component for current state capabilities relative to desired current state (not future state, but how well EDM/SOA processes, architecture, and data meet current needs)
  3. Inventorying initiatives either currently underway or that will begin shortly for which ratings will change in the current state
  4. Determining the future vision of the enterprise based on the C-SODA used to evaluate the current state:
    a. For short (six months), intermediate (year), or long-term (18 months+) goals as needed
    b. (Optionally) granulize into intermediate steps/goals
  5. Determining gaps between C-SODA's current state (plus known initiatives) and C-SODA's future vision. Gaps can be further granulized into intermediate steps/goals
  6. Developing initiative definitions for filling gaps
  7. Prioritizing/scheduling gap-filling initiatives to achieve desired C-SODA capabilities

Most C-SODA dimensions have business and IT components (see Figure 2). There will potentially be both (and combined) initiatives executed in a coordinated fashion to drive improved capabilities. In developing a specific C-SODA for your organization, you may adjust/emphasize key components for your business or initiatives.

The C-SODA Capability Maturity Model
The C-SODA CMM supports the framework for evaluating or driving increased organizational capabilities/maturity. Let's describe what a Service-Oriented Data Architecture (SODA) looks like, since increasing organizational maturity and EDM/SOA coordination will result in increased SODA capabilities.

Figure 3 shows a conceptual view of a SODA with an EAI/Data Integration layer at the center representing the foundational data and data platform services that will be reused in the other services layers. There are multiple layers and domains of SOA services represented in Figure 3 as two (or more) "SOA/ESB" domains addressing different levels of service abstractions and functional domains in the organization.

The SOA/ESB layers abstract "gold standard" data sources from data/service consumers whether end users at the User Experience/Presentation Layer or (Operational/Third-Party) systems. This abstraction lets us institute workflow and orchestrate/automate the services. Workflow management capabilities, coupled with business rules (i.e., new metadata) for decision making about which services to invoke with what parameters, how to notify users/systems throughout workflow progress, how to route messages, and how to manage SLAs further provides a powerful architecture to optimize data and services assets in a coordinated fashion.

Figure 4 shows building blocks for C-SODA strategy phases along the maturity path, ultimately leading to data and services managed as corporate assets (optimized phase). Each building block at each level of the CMM has implications for either or both EDM or SOA strategies/capabilities.

Each building block on each strategy level should be fulfilled before considering that level achieved and moving to the next. Many building blocks at lower strategy levels are prerequisites for higher-level ones, so lower building blocks should generally be fulfilled to achieve higher maturity levels. Data governance is a prerequisite to MDM, which is a prerequisite to EDM. Some aspects of these building blocks are prerequisites for Enterprise Business Services, which are prerequisites to maturing the enterprise as a SODA. Thus, in building a roadmap to achieve greater maturity along the C-SODA CMM, pay particular attention to unfulfilled prerequisite building blocks and address them in early prioritized initiatives.

Color-coding guidelines per strategy level, shown in Figure 4, can be utilized in the C-SODA ratings during an assessment of organizational maturity. More detailed C-SODA CMM views are shown in Figures 5 and 6 to address EDM/SOA perspectives, depending on the stakeholders. Either view can be utilized, but one may become the primary focus among an organization, program, or project.

Most organizations have reached some degree of centralized data sources, but still have inconsistent management of common data and services. This generally means they are centralized or managed but not optimized.

Figure 5 displays more detail than the building blocks diagram and further shows the appropriate evolutionary steps for each primary dimension of the C-SODA framework. We see how each of the primary dimensions progress in capabilities as the organization matures to more advanced C-SODA strategy phases. This roadmap of progressive capabilities can be adapted when defining the organization's future vision, overall and for each framework dimension/component.

Also notable in this view is that most organizations attempting to evolve their EDM/SOA strategies currently fall within centralized or managed strategy phases (deployment major phase) in maturity. To optimize these (coordinated) strategies, organizations must proceed further to enabled or optimized strategy phases (agility major phase).

Figure 6 shows an alternative SOA-centric view of the C-SODA CMM strategy levels. It is consistent with the building blocks and EDM view, but is intended to show capability maturity in terms of SOA stakeholders/implementers. As we develop coordinated EDM/SOA strategies and programs, these views may be utilized in parallel by appropriate groups, and the C-SODA CMM building blocks diagram (see Figure 4) would be utilized by all stakeholders/implementers as a shared vision for EDM/SOA capabilities further detailed in each view.

Next Steps
Organizations should develop an appropriate data-SOA governance program and C-SODA CMM coordinated at the highest levels to enable the optimal value of services and data. This is the highest priority in developing coordinated EDM/SOA capabilities.

As an organization develops its (data and/or SOA) governance, it should coordinate processes, checkpoints, and ownership for services/data. Because we rarely develop these strategies as coordinated entities initially, it's important to adapt appropriate processes between them and to (re-)define roles/responsibilities to support coordinated data-SOA governance.

If the organization has an overarching program/project management organization (PMO) that plans and funds initiatives, especially enterprise-level initiatives spanning services/data, coordination should be taken into account as prioritized initiatives laying the foundation for achieving advanced C-SODA capabilities.

The organization should scale progressive EDM/SOA initiatives, including data-SOA governance responsibilities for coordinated processes and communications. Internal education should inform EDM/SOA resources/stakeholders how to effectively leverage each other during joint activities. Last, a properly chartered COE/ICC can facilitate bringing diverse stakeholders together to drive the coordination during all stages of coordinated EDM/SOA initiatives.

© 2008 SYS-CON Media