Common business software tools such as SharePoint, MS Office applications, and database applications are difficult to customize for use as a data governance solution. There are solutions available that are designed to support a data governance framework and that organize metadata associated with data assets. The better data governance solutions will also support document management and data governance role management. These solutions provide data users with a centralized place to access governed metadata (data about data). Because these solutions are independent of any data system or reporting tool, the solution can offer a comprehensive view of data assets in a single user interface that is easy to use. Data users can discover and evaluate data assets through self-service use of a data governance solution. The Data Cookbook, by IData, is an enterprise data governance solution that supports all aspects of metadata management, data governance role management, and document management, all within a solution that is easy to set up and use.
When selecting a data governance solution, consider the main criteria listed below. Each section concludes with a list of questions to ask a data governance solution vendor.
- Knowledge base, repository, and data processing catalog (metadata)
- Data quality management
- Handling of data requests
- Data governance roles, collaboration, and permissions
- Integration, import, and export capabilities
- Simplicity of deployment
- Management and oversight
- Ease of use
Let's take a closer look at each of these areas.
Knowledge Base, Repository, and Data Processing Catalog (Metadata)
The data governance solution needs to have a centralized knowledge base that includes curated metadata, a document repository, and a data processing catalog. A knowledge base provides a central location for data users to discover and evaluate data-related assets. It also provides a place to consolidate policy and training materials.
For curated metadata, look for features that support several types of metadata and that describe most data-related assets. Each type of metadata should have a set of default fields that pertain to that type, as well as the ability to create custom fields. Look for features in a solution that link the metadata to data assets.
A data processing catalog (sometimes called a report catalog) contains information on the movement and transformation of data among systems. This information can be narrative or highly technical. Common items found in a data processing catalog are specifications about reports and integration between data systems.
A document repository could contain training materials, data policies, data sharing agreements, security policies and other items.
The following list summarizes the data-related assets to look for in a data knowledge base:
- Business Glossary - functional and technical definitions
- Data Processing Catalog - reports, dashboards, ETLs, surveys, APIs, etc. as well as collections which are a group of associated reports
- Data System Catalog - system details, technical data models, tagging, access requirement, and ownership
- Reference Data (valid value lists) - management, mapping, and monitoring
- Data Quality - rules, assessments, monitoring, and issue resolution
- Policy Attributes - data access, security, sharing, privacy and retention
- Data Ownership - functional area, domain, and steward classification
- Data Lineage
- Data Requests (and resolution)
- Documents - data policies, procedures and business processes
- Change Management - versioning, status, dependency, and impact
- External Data Standards and Reporting
Ask the vendor:
- Which types of metadata can be documented and stored in the solution's knowledge base?
- What types of data-related assets can be described using the metadata types provided? How easy is it to link metadata to data assets?
- How much flexibility is there in labeling and naming conventions?
- Can metadata serve multiple audiences or data users?
- Are documents edited within the data governance solution or created elsewhere and uploaded?
- Are documents of different types stored separately?
Data Quality Management
Managing data quality from within a data governance solution is an efficient way to monitor data quality efforts across all data systems. The solution should include a catalog of rules, and the ability to assess rules against one or more data systems.
The solution should have the ability to manage reference data, including local valid value tables). When valid values (reference data) change in one data system, there can be a ripple effect in data systems further down the line. A comprehensive data governance solution manages valid value or reference data changes. Look for the ability of alerting data owners that a change has occurred and that can quickly show the impact of value changes.
Ask the vendor:
- Can data quality rules be added to the solution and assessments run from these rules?
- Can the same data quality rule be associated with multiple data systems or data collections?
- Is the metadata associated with a data quality rule both descriptive (an explanation of what is being measured) and technical (pinpointing exactly which columns and tables are measured)?
- Are alerts automatically generated when valid value tables or reference data change?
Handling of Data Requests
If data users use the solution as the main discovery platform for data-related assets, it makes sense that the solution includes a data request process. Ideally, data users would first discover a data-related asset through the solution and then make a request concerning it within the same solution. Or, if the user is unable to locate a data-related asset that meets their needs, a request to create or provide that asset can be submitted. Templates in the solution should accommodate the variety of data request types. There should be a workflow that includes the notification to the person handling the request.
Ask the vendor:
- Is a data request capability included?
- Is it easy to find the starting point to submit a data request?
- Is it easy for the requester and the request manager to track the status of a data request?
- Does the data request process within the data governance solution integrate with ticketing systems already in use at the organization?
- Is there a workflow where data stewards are notified about the request?
Data Governance Roles, Collaboration, and Permissions
Data users who have been assigned to data governance roles such as data steward or data system manager will have unique tasks within the data governance solution. Permissions should be assigned to users based on the data governance role(s) that the user holds. Role label names should be customizable to accommodate the nomenclature the organization uses. For example, an organization might use the term data manager instead of data steward or data owner.
Permission levels should be the common permission levels of View, Edit, and Approve. Additional permissions that support collaboration include the ability to assign permissions on an ad hoc basis. Ad hoc assignment allows a user to make a one-time action without the burden of long-term responsibility for similar actions. There should be reporting features that generate a list of users according to role or permission, and reports on the volume of content a user has authored or approved. To support collaboration, a flexible workflow that matches the preferred business process is important. Users, acting within their workflow role, can perform specific tasks in a proscribed order.
Ask the vendor:
- Can roles be labeled and defined according to an organization's structure?
- Can approval workflows be customized?
- Are multiple types of workflows available?
- Are standard permissions available such as Edit, View, and Approve?
Integration, Import, and Export Capabilities
Since a significant amount of the metadata that is targeted for inclusion in the data governance solution is found within existing data systems, it is important that the import and export capabilities are easy to use and integrate well with the popular data systems in use. Metadata can be extracted from data systems and imported into the data governance solution, thus avoiding tedious manual entry.
For import capabilities of the solution, look for templates and clear error messages in the import process. Understand if subsequent imports overwrite previous content and whether there is version control. In particular, look at the templates used to import technical metadata from data systems or repositories. Import templates for the data governance solution should accommodate the export templates from the popular data systems used within an industry.
Evaluate the export capabilities of the solution. Content from the data governance solution may need to be republished in private intranets or even in paper format. Auditors may require specialized reports as proof that personal identifiable information (PII) or legally restricted information is being managed correctly.
The solution needs to integrate with data systems used by the organization, such as ticketing systems. And the analytic, business intelligence (BI) and reporting tools used by the organization should be integrated with the data governance solution.
Ask the vendor:
- Is the export file format of the organization's primary data systems compatible with the import formats of the data governance solution?
- What are the import capabilities?
- How does the import feature overwrite existing metadata?
- What are the export capabilities?
- How is the integration between data systems managed in the solution?
Simplicity of Deployment
Data governance initiatives are often led by staff outside of an IT department, with little experience implementing an application, such as a data governance solution. Gauge the amount of training and installation effort needed to install the data governance solution. Evaluate the effort necessary to enter data-related assets, customize workflows, define codes and add settings.
Ask the vendor:
- How long does it take to install and configure the solution?
- What kind of technical skills are required for setup?
- Is a server needed? Can it be physical or virtual?
- What branding capabilities are available? Can a logo and color scheme be changed?
- How are user accounts provisioned and managed?
- What implementation services are available to assist the organization in the solution's use?
- What kind of training materials are provided such as videos?
Management and Oversight
Just like other solutions and major initiatives, data governance and the use of a data governance solution needs to monitored. Review the metrics that the data governance solution provides to measure content volume, user engagement, request status and data quality assessments.
Ask the vendor:
- What metrics are tracked in the solution?
- What reports or dashboards are available for the manager?
- Can managers be included in workflows?
Ease of Use
The user interface of the solution should be friendly to business users and technical experts. Labels should be descriptive and jargon free. Evaluate the solution from the point of view of a non-technical data person who might only interact with the solution a few times a month.
Ask the vendor:
- Is there a centralized search in the solution?
- Can discovery be siloed to a particular section of content?
- Is content immediately indexed or is there a lag time of 24 hours?
- Is it easy for a casual user to navigate the solution and complete tasks?
- Is there enough detail to satisfy technical experts?
- Is the solution intuitive, is it easy to find information and is it easy to make requests?
Data governance and data intelligence is critical for all organizations because it instills trust in data, improves decision making and helps people. For successful data governance, a data governance solution like the Data Cookbook provides the framework, processes and content that is required. This blog post mentions the features and capabilities necessary in a data governance solution. Additional information on this and other topics can be found here.