Data Quality Management with SAP Master Data Governance on SAP S 4HANA

SAP Master Data
Governance
on SAP S/4HANA
Master Data Quality
Management
SAP Master Data Governance
October 2023
Public
Agenda
Overview Data Quality Management
Define and Apply Validation Rules
Data Quality Analysis and Remediation
Define and Apply Derivation Scenarios
Use Reference Data for Business Partners with Data Provider Integration
Add Custom Fields
Public
2
Overview Data Quality Management
Typical approaches to master data management
Central governance and distribution / de-central ownership and consolidation
LoB customer
LoB procurement
LoB finance
LoB production
Other LoB
qu
alit
y
Ent
er
qu
alit
y
e
ov
Consolidate master data into a single view
for accurate analytics and operational
insight (continuously or on request, such as
for initial load or mergers and acquisitions)
Im pr
SAP Master Data Governance
– consolidation
Enterprise
master data
ality
qu
y
alit
qu
De
fin
e
Systems under central governance
r
ito
n
o
M
SAP Master Data Governance
– central governance
Create master data in-line with business
processes (such as integrated product
development or supplier management)
SAP Master Data Governance
– data quality management
Manage master data quality by defining,
enforcing, monitoring, and improving quality
Systems not under central governance
Legacy systems
Reporting and analytics
Business networks
Cloud
All typical approaches are supported: central governance with distribution, decentralized ownership with consolidation, data quality monitoring with remediation
Public
4
SAP Master Data Governance, Data Quality Management
Process flow
Define Quality
Requirements are defined based on
your company’s business processes
Priorities are set according to value,
impact, and quality evolution
► Experts collaborate to define
needed quality level and required
checks
► The system helps to identify
additional meaningful rules
Public
Enter Quality
Ensure quality at point of entry
Consider all entry-points: single
changes, mass changes, load
scenarios, in daily business,
projects, …
► Rule-based checks in all
processes of SAP MDG
► Automation with derivations
and external reference data
Monitor Quality
Improve Quality
Operational motivation: detect
issues before processes fail
Correct data and drive the
correction process
Tactical: ensure progress and
performance of current activities
Fix data entry processes
Strategic: enable achievements,
define new initiatives
Evolve the definition of quality
► Tools to fix data and to improve
checks at point of entry
► Easy to consume monitoring
and trend reporting
5
Data Quality Management
Define, measure, and improve the quality of your master data for sustained process excellence
In a Nutshell …
Collaboratively define, implement, and govern validation rules for a consistent usage in master data processes.
Business Value
Comprehensive repository of well-defined and agreed
validation rules, providing transparency on business aspects,
the implementation, and the usage in master data processes.
Benefit from a consistent definition of data quality, and
continuously evaluate and monitor the quality of your master
data by applying the validation rules.
• Business partner and product master data are supported.
Additionally, you can use the apps as a platform for custom-defined
objects.
• Collaboratively describe, catalog, and implement validation rules.
• Use the same rule across all MDG processes to ensure correct data
entry.
• Schedule quality evaluations, analyse evaluation results and initiate
the correction of erroneous data.
• Get an overview of the current data quality status and KPIs.
Public
6
Data Quality Management
Automate creating and updating master data for even higher data quality with less efforts
In a Nutshell …
Collaboratively define, implement, and govern derivation scenarios for a consistent usage in master data processes.
Business Value
Central place for derivation scenarios, which can be used
in all MDG processes. Get transparency on business
aspects and technical implementations of derivation
scenarios. Use derivation scenarios to accelerate the
automation in the master data management process.
• Use the repository to catalog and define derivation scenarios.
• Business partner and product master data are supported.
• Define a comprehensive description of derivation scenarios
including business aspects, ownership, and implementations.
• Collaboratively manage derivations scenarios and their lifecycle
from creation to sunset.
Public
7
Data Quality Management
Leverage reference data from external providers to create and update your business partner master data
In a Nutshell …
Reduce manual efforts for creating and maintaining up-to-date business partner master data with sustained quality
• Update existing business partners by enriching with missing
and more recent information
qu
alit
y
qu
alit
y
e
ov
• Expedite the creation of business partners with an
integrated lookup in data pools
Ent
er
Data Provider
Reference Data
Im pr
• Gateway to access external master data content and
services provided by data providers
ality
qu
y
alit
qu
Utilize external services for reference data with the
integration in master data management to automate
and optimize processes and to achieve correct,
validated and current business partner master data.
SAP Master Data Governance
data quality management
De
fin
e
Business Value
Services
r
ito
n
o
M
Enterprise master data
• Engage with your data provider* for a best fit to your usage
* Currently supported: CDQ AG, see SAP Store.
Public
8
Master Data Quality Management with MDG on SAP S/4HANA
Functional Scope 1/2
Rule Management
Data Quality Remediation
• Collaboratively describe, catalog, and implement
rules for data quality using a central rule repository
• Efficiently delegate or perform the remediation of
data quality issues
Rule Mining
Check of Data in Change Requests
• Discover new data quality rules utilizing machine
learning, analyzing your existing data.
• Apply validation rules at the point of data entry
with central governance
Data Quality Evaluation
Check of Data in Consolidation and Mass
Processing
• Assess data quality by applying validation rules on
existing data
Data Quality Analysis
• Apply validation rules in the validation step of a
mass process or a consolidation process
• Monitor progress of data quality initiatives and get
insights for improvement
• Integrated with SAP Analytics Cloud
Public
9
Master Data Quality Management with MDG on SAP S/4HANA
Functional Scope 2/2
Define Automation Rules
Manage system landscapes
• Collaboratively define derivations and enable
business users to control the lifecycle
• Reuse validation rules and derivation scenarios in
different systems by using the apps for export and
import
Apply Automation Rules
• Extend your master data according to the defined
rules in mass processes, consolidation processes
and central governance change requests
Data Provider Integration
• Leverage reference data from external providers
to create and update your business partner master
data*
Domains
• Product master
• Business partner
• Custom Object
• Asset management (Solution Extension)
*available in cloud-ready mode only
Public
10
Master Data Quality Management in classic mode and cloud-ready mode
As of SAP S/4HANA 2023, SAP Master Data Governance
on SAP S/4HANA (SAP MDG) offers two different modes:
•
The standard delivery option of SAP MDG, known
as “SAP Master Data Governance”, is now referred to
as “classic mode in SAP Master Data Governance”.
•
The “cloud-ready mode in SAP Master Data Governance” is an alternative to classic mode in SAP
MDG. As of today, cloud-ready mode is only applicable for business partner data.
This presentation outlines the scope of master data quality management and relates both to cloud-ready
mode and classic mode (if not marked otherwise)
Cloud-ready mode is only applicable for business partner data and supports all classic mode features plus a
few enhancements that are detailed in the following slides.
Please note that you can continue using the validation rules and derivation scenarios that you created in
classic mode in cloud-ready mode.
Public
11
Define and Apply Validation Rules
Managing Validation Rules
High-Level Scope and Features
You can manage your validation rules for master data
quality in one single place
• Repository to catalog and define validation rules
• Comprehensive description of rules, including
business aspects, ownership, and rule
implementations
• Collaboration and status handling during the
lifecycle of rules, from creation to sunset
Business Value
• Central place for master data quality rules
• Transparency on business aspects, usages and
technical implementations of rules
Public
13
Properties of Validation Rules
General
• Rule name, ID, and Status
• Checked field – field that is the subject of the rule (see later)
• Base table – basis for the evaluation of the rule (see later)
Business Details
• Description – Detailed description what the rule checks
• Reason – Why is the rule needed? What is the impact if data does not
comply to the rule?
• Scope – description of the data set to which the rule is applicable
• Link – URL link to further information that is stored elsewhere
Contacts
• Rule Owner – A user who is responsible for the rule
• Implementation Expert – A user who will do the implementation
• Business Contact – Free text, e.g. the requester(s) of this rule
• Data Owner – Free text: who must and can correct the data? e.g.
responsible purchasing department
Public
14
Rule Status
Status of a Validation Rule
• Represents its life-cycle status
• Determines if the rule is considered
during data quality evaluation and MDG
New
processes
Send for Implementation
Status
Purpose
New
Recently created rule, still in definition
To be Implemented
Definition of rule is completed,
implementation requested or in progress
To be Tested
Implementation of rule completed and to
be verified
Approved
Rule is valid from business perspective
and its implementation is tested
Disabled
Rule is excluded from data quality
evaluations
New & Save
Status Transitions
• Pre-conditions are checked, e.g. all
BRFplus expressions must be active
before a rule can be set to Approved
status.
• Authorizations are required
Flexible Configuration, e.g.
• One user does all
• Strict segregation of duties
Public
To be
Implemented
Send for Testing
To be Tested
Activate
Approved
Disable
Possible statuses of a validation rule with the
corresponding UI actions to set the status.
Disabled
Delete
Note:
Only straight-forward status transitions shown,
further transitions are possible.
15
Rule Implementation with BRFplus
Implementation with BRFplus
Business Rules Framework plus is used for
implementing and executing rules
Powerful, versatile, assisted
• Implementation with BRFplus from simple “field is
not initial” to complex calculations
• Powerful and “context-aware” BRFplus workbench
• Robust and proven rule engine with code
generation
• Scaffolding of BRFplus performed automatically
with Prepare, once a usage is added
Public
16
Data Provisioning
Supported Constellations
During evaluation, data of the base table, the root
table (MARA/BUT000) and the parent table of the
base table are automatically supplied.
Other constellations are supported with extra data
provisioning (e.g. BRFplus Database Lookup
Expression, model extension, creation of custom
views) – see examples for BP and products
Note: Consider the performance impact when using
own data provisioning!
Public
17
Validation Rules in MDG Processes – Overview
Consistent Re-Use of Rules
• Usage of rule for
– data quality evaluation
– check in change request
– check in consolidation
– check in mass processing
• Re-use of the same rule implementation
• Flexibility to independently enable rules for
evaluation and check
• Possibility to assign predefined messages
Business Value
• Efficiency in the definition process of rules
• Consistency in application and implementation
Public
18
Validation Rules in Change Request Processing of Central Governance
Change Request Processing
• Validation rules will bring up errors or warnings
as defined by the rule.
• Execution can be defined per change request
step
• Open for custom implementation in the access
class of MDG
Public
19
Validation Rules in Mass Processing and Consolidation
Validation and Activation Steps
Validation rules will bring up errors or
warnings as defined by the rule.
Covered Process Variants
• Consolidation of Source Records
• Consolidation of Active Records
• Mass Processing
• Mass Maintenance
• Mass Load
Public
20
Assign Predefined Messages
Messages for Check Usages
• Maintain predefined messages for validation
rules
• Use message variables from rule context and
master data context
• Define message severity
• Will be shown during execution of checks on
the UI
Public
21
Enhancements for cloud-ready mode in SAP MDG – Validation Rules
The usages in the Validation Rules for
Business Partners change as follows in the
cloud-ready mode in SAP MDG:
Check in Mass Processing is replaced
by Check in Central Governance, which can be
used for checks that are executed during
governance processes for business partners.
Check in Inbound Processes is available for
checks that can be applied to inbound
processes in central governance for business
partners.
All other usages remain unchanged.
Public
22
Export and Import Validation Rules
Synchronize Rules across systems
Export Rules
• Select them in the rule repository
• Download them in open office XML format
Import Rules
• Validate the import file
• Import validation rules including BRFplus
implementation
• Import validation rules that already exist in
the system to update existing validation
rules in SAP Master Data Governance
(as of Feature Pack Stack 01)
Public
23
Simulate Validation Rules
Create Simulation Process
• Test rule implementation before it is used
productively
• Simulate both active version and inactive version
of the validation rule implementation
• Preview the evaluation results so you can see
the potential impact on your data quality
• Drill down to result details to check the results
for individual master data records
Public
24
Rule Mining – Apply Machine Learning to Discover Rules in Master Data
High-Level Scope and Features
You can use rule mining as a complementary process
to manual rule definition to discover rules in existing
master data
• Use mining runs to analyze existing master data
• Collaboratively decide on business relevancy of
proposed rules from rule mining
• Create and link validation rule from accepted rules
• See information from rule mining in implementation of
validation rule
Business Value
• Profit from easier and quicker discovery of rules with
machine learning
• Efficiently qualify and implement discovered rules as
validation rules
Public
25
Process of Master Data Rule Mining
Rule Mining
Define
MINING RUN
Start
MINING RUN
Review
MINED RULES
Usage of rule in MDG processes
Accept and Link
MINED RULES
Business
User
Public
Define Data
Quality Rule
Monitoring
Improvement
Master
System
Data
Stew ard
26
Data Quality Analysis and Remediation
Configure Score Calculation
Aggregation of rules scores to
data quality dimensions
Dimensions
• Multiple dimensions supported
• Score: weighted average of rule scores
• Thresholds: Target, Warning, Critical
Rules
• Assigned to dimension with Impact
• Multiple assignments possible, but only
once per category
Categories
• Multiple categories supported
• Special: Global category used for data
quality overview
• Score: Average of dimension scores
• Thresholds: Target, Warning, Critical
Public
Steven Olsen
Note: The score-related cards of the app Data Quality Overview are bound to display the category Global only.
28
Data Quality Evaluation Overview – Available Cards
Latest Data Quality Score
• Data quality score by dimensions of
category Global
• Drill-down into the results to analyze the
score by dimension, rule, … down to the
actual rule outcomes
Data Quality Trend
• Trend of the dimension scores of the
category Global
• Drill-down …
Incorrect Data
• Various views (e.g. by product status, by
plant)
• Drill-down …
Drill-down from KPI
to detailed results
Latest Data Quality Evaluations
Public
29
Analyze Data Quality Using the Data Quality Dashboard in SAP Analytics Cloud
• Provide an easy-to-use dashboard
for end users, from chief data
officers to master data specialists
• Get a real-time overview of master
data quality and historical trends
• Manage your own charts
• Benefit from a cloud-based central
analytics platform
• Integrated with S/4 via Live Data
connection and direct navigation
Public
30
Data Quality Evaluation – Overview
Data quality evaluation applies all rules
to the active data in the system.
The results of the evaluation are stored
for later analysis.
Manage Evaluations
• Initiate ad-hoc evaluations
• Schedule evaluations e.g. at weekly
intervals
• Delete entire evaluations
Public
31
Analyze Evaluation Results – Overview
Use
• Analyze data quality evaluation results
1
• Find out which master data have errors, and which
don’t
• Start the correction of data
• Use specific predefined apps for products and
business partners
2
3
Public
32
Analyze Evaluation Results – Products
Evaluation Results Apps for Products
Example: Evaluation Results for Plant Data
• Products
– all rules, from any base table
– fields from basic data (MARA) for drill-down
• Plant Data
– rules with base table Plant (MARC) and below
– fields from basic data (MARA) and plant (MARC)
for drill-down
• Sales Data
– rules with base table Sales Data (MVKE) and
below
– fields from basic data (MARA) and sales line
(MVKE) for drill-down
Public
33
Analyze Evaluation Results – Business Partner / Customer / Supplier
Evaluation Results Apps for Business Partners
Example: Evaluation Results for Supplier Company Code Data
• Business Partner
– all rules, from any base table
– fields from general data (BUT000) for drill-down
• Sales Data of Customer
– rules with base table Sales Org Data (KNVV) and below
– fields from basic data (BUT000) and Sales Org Data
(KNVV) for drill-down
• Company Code Data of Customer
– rules with base table CC Customer (KNB1) and below
– fields from basic data (BUT000) and CC Customer (KNB1)
for drill-down
• Purchasing Data of Suppliers
– rules with base table Purchasing Org Data (LFM1) and
below
– fields from basic data (BUT000) and Purchasing Org Data
(LFM1) for drill-down
• Company Code Data of Suppliers
– rules with base table CC Supplier (LFB1) and below
– fields from basic data (BUT000) and CC Supplier (LFB1) for
drill-down
Public
34
Correct Erroneous Data
1. Delegate work (share page)
Send link to page including filters to delegate
correction work
1
2. Export of evaluation items
Export table with evaluation results in open office
(XLSX) format
3. Single products or business partners
Navigation to all apps in the role of the user
Examples: open fact sheet, change with change
request, …
4. Multiple products or business partners
Process selected objects with mass processing
3
4
5
2
5. Export products or business partners
Export selected objects in open office (XLSX)
format for offline editing and later import in mass
processing app
Public
35
Define and Apply Derivation Scenarios
Managing Derivations
High-Level Scope and Features
You can derive the values of fields or create and
update dependent entities (for example, create a
storage location if a plant is created).
• Repository to catalog and define derivations
• Comprehensive description of derivations,
including business aspects, ownership, and
implementations
• Collaboration and status handling during the
lifecycle of derivations, from creation to sunset
Business Value
• Central place for derivations that can be used in a
change request, in mass processing and
consolidation.
• Transparency on business aspects and technical
implementations of derivations
Public
37
Define the Structure of a Derivation Scenario
Business users define the structure of a derivation
scenario in a user-friendly SAP Fiori UI
• Describe derivation scenario and rules
• Define responsibilities by assigning scenario
owner, implementation expert, rule owner
and data owner
• Collaborate via status handling
• Manage execution of derivation rules
Public
38
Manage Derivation Rules
• Define the fields for a derivation rule on the
SAP Fiori UI
• Use the system-generated BRFplus tables to
maintain the values to be derived
• Use the built-in value help to maintain values
• Use the full set of BRFplus capabilities for
complicated use cases
Public
39
Derivation Scenarios in Mass Processing and Consolidation
Apply derivation scenarios using
a process template with a step
for derivations.
Public
40
Derivation Scenarios in Change Requests
Apply derivation scenarios in a
change request, based on a
background step in the rule-based
workflow, a custom BAdI
implementation, a new API
(CL_MDG_MDQ_RBWF_DERIVE)
and corresponding customizing.
Public
41
Enhancements for cloud-ready mode in SAP MDG – Derivation Scenarios
There are no changes to the definition of
derivation scenarios in cloud-ready
mode in SAP MDG.
When a derivation scenario is executed
in federated master data governance in
an inbound process with goal Inbound
Process Approve or Inbound Process
Merge, the data ownership is checked. If
you want to derive data that is not
owned by the current system, you get a
notification, and the derivation scenario
is not executed.
Public
42
Export Derivation Scenarios
Download derivation scenarios and their rules
• Start the export process for selected
derivation scenarios
• Display the logs to check and correct any
issues found during validation
• Download the file with the derivation
scenarios and derivations rules, which can
then be imported using the Import
Derivation Scenarios app
Public
43
Import Derivation Scenarios
Upload and import derivation scenarios and
their rules
• Validate derivation scenarios and their
rules before they are imported
• Display the logs to check and correct any
issues found during validation
• Import the validated derivation scenarios
and enable them for execution
Public
44
Use Reference Data for Business Partners
with Data Provider Integration
Data Provider Integration – SAP MDG on SAP S/4HANA 2023
Create and enrich business partner from reference data provided by CDQ AG
➢
➢
Public
Users can look up reference data to create and
enrich business partners
Requires central governance in cloud-ready
mode
✓
CDQ provides access to numerous data sources.
✓
The availability of the CDQ platform, its services, and
data sources is subject to your agreement with CDQ
AG and independent of contracts with SAP SE.
✓
A description of the CDQ Business Partner LookUp
service and its configuration is provided by CDQ
✓
Requires a contract with CDQ AG, see SAP Store.
46
Create Business Partner from Reference
With External Data Provider CDQ - User Interface and Flow
1. Search in the Manage Business
Partners app reveals that a
needed business partner is not
available in MDG
1
2. With Look Up and Create you
can search for reference data
from CDQ *
2
3. Having chosen the matching
reference record, Create
Business Partner transfers
information to the Manage
Business Partners app
4. Reference data is available for
creating the business partner
without the need of manual
data entry
3
4
*Requires a contract with CDQ AG, see SAP Store.
Public
47
Enrich Business Partner from Reference
1
1. Use Look up and Enrich to
search for additional reference
data for a Business Partner.
2. Existing information of the
Business Partner is used to
search for reference records
from CDQ *.
Search criteria can be manually
adjusted.
3. Select the reference record of
interest.
Pick from the compared data,
initially only the delta is shown.
The system recommends an
action (e.g. “Overwrite”) which
can be overruled (e.g. with
“Add”).
4. Business Partner is enriched with
additional data.
2
3
4
Public
48
Use Stored References for a Comparison with the Business Partner
1. While using Look up and Create
or Look up and Enrich not only
the data is transferred to the
Business Partner, but also from
where the data was taken from.
This source information is stored
in the Identification Numbers References section.
2. With selecting the reference
record of interest and clicking
on Enrich the business partner is
compared to latest data of exact
that reference.
3. Business Partner is enriched
with additional data.
Public
3
1
2
49
Add Custom Fields
Custom Fields Enabled for Data Quality Management
High-Level Scope and Features
▪ Custom fields in all MDG Data Quality Management
processes and apps are enabled after being published
in the Custom Fields app.
▪ The domains Product and Business Partner (including
Customer and Supplier) are supported.
Rule
Definition
&
Implementation
Derivation
Validation
Custom
Fields
Benefits and Business Value
▪ Easy management for custom fields with MDG Data
Quality Management
▪ Reduced TCO for extensibility scenarios
Public
Evaluation
& Error
Analysis
Rule
Mining
Remediatio
n
51
Custom Fields Enabled for Data Quality Management
Custom fields are enabled in the following processes and apps of MDG Data
Quality Management:
 Validation Rule Definition: Custom fields can be used in the definition
and implementation of validation rules.
 Rule Mining: Rules can be discovered for custom fields in the rule mining
applications.
 Checks in Consolidation and Mass Processing: Validation rules using
custom fields can be executed in consolidation and mass processing apps.
 Data Quality Evaluation: Custom fields can be evaluated with validation
rules.
 Data Quality Analysis and Remediation: Custom fields can be filtered
and displayed in quality evaluation result apps and can be corrected in
mass processing apps.
 Derivation Scenario Definition: Custom fields can be used as condition
and result fields in derivation rules.
 Data Derivation in Mass Processing and Consolidation: Custom fields
can be derived in consolidation and mass processing apps.
This feature is enabled for the Business Partner and Product domains.
Public
52
Further Information
Further Information on MDG, Data Quality Management
Documentation on SAP Help Portal - Working with MDG, Data Quality Management
Guides and Videos: Community, How-to
Community Blogs, tagged with mdg-dqm
SAP Master Data Governance, Product management
Public
54
In-App help with SAP Companion (aka Web Assistant)
• Press F1 or click on the
help button
• Offers detailed
explanations and links to
help.com
Public
55
Learn more about SAP Master Data Governance
► SAP Road Maps
(direct link: SAP MDG in Roadmap Explorer)
► SAP Community
► SAP User Groups
► SAP Customer Influence
(Projects: MDG 2018, MDG 2019, MDG 2020, MDG 2021/22, MDG 2023, Continuous
Influence)
► SAP Master Data Governance on SAP.com
► SAP Master Data Governance Community
(recent strategy updates: mdgupdates)
► SAP Master Data Governance Application Help
► Education for SAP Master Data Governance /
SAP MDG Consultant Certification
Thank you.
Contact information:
SAP Master Data Governance, Product Management
© 2023 SAP SE or an SAP affiliate company. All rights reserved. See Legal Notice on www.sap.com/legal-notice for use terms, disclaimers, disclosures, or restrictions related to this material.