Skip to content

Data Quality Governance

Purpose

This workflow covers the Clarity Transform screens used to measure and correct the quality of CMDB data. Clarity scores completeness on two dashboards, Application Data Quality and Device Data Quality, using the fields marked Mandatory or Suggested on Settings → Field Management → Field Validations. A third dashboard, CMDB Data Correctness, reports duplicates, stale records, orphaned CIs and dangling references across the CMDB. Records are corrected on Discover → CMDB.

Stages

1. Choose the fields that are scored

Go to Settings → Field Management → Field Validations.

  • Each row sets one field of one CI class to Mandatory or Suggested, using the columns Ci Class, Field Name and Validation.
  • The Application and Device data quality scores are driven by these rows.

To add a row:

  1. Select the add icon on the grid toolbar. The Create new entry form opens.
  2. Select the Ci Class: Application, Cluster, Database, Device or Relationship.
  3. Select the Field Name. The list shows the chosen class's columns by their database names.
  4. Select the Validation: Mandatory or Suggested.
  5. Select Save.

See Field Validations.

2. Review application completeness

Go to Discover → Data Quality → Application Data Quality (header "Application Data Quality Dashboard").

  • Read the cards TOTAL APPLICATIONS, QUALITY SCORE, COMPLETE and MISSING DATA.
  • The Quality Distribution chart groups applications into the tiers Excellent (90-100%), Good (70-89%), Fair (50-69%) and Poor (0-49%).
  • Most Common Missing Fields lists up to 10 fields, Mandatory and Suggested together, each as "N apps (P%)". Field Completion Rates shows one bar per field set on Field Validations, Mandatory fields first. If no fields are set for Application, it reads "No field validation rules configured."
  • The Application Quality Details grid lists each application with QualityScore and QualityTier. MissingMandatoryFields and MissingSuggestedFields hold the names of the missing fields, not counts.

See Application Data Quality.

3. Review device completeness

Go to Discover → Data Quality → Device Data Quality (header "Device Data Quality Dashboard").

  • Read the cards TOTAL DEVICES, QUALITY SCORE, COMPLETE and MISSING DATA.
  • The Quality Distribution chart uses the same tiers as Application Data Quality (its lowest tier is labelled Poor (<50%)). A tier with no devices is left out of the chart. Field Completion Rates shows completion by field, highest first.
  • The Device Quality Details grid lists each device with QualityScore, QualityTier, MissingMandatoryFields and MissingSuggestedFields.

See Device Data Quality.

4. Check correctness

Go to Discover → Data Quality → CMDB Data Correctness.

  • Set Stale threshold (days) (default 30), then select Refresh. An empty value or 0 is treated as 30, and the value is not kept when you leave the page.
  • Read the cards OVERALL CORRECTNESS, CI TYPES, CRITICAL DUPLICATES ("Provider ID dupes") and NAME DUPLICATES.
  • The CI Type Breakdown table gives, per CI type, Total, Provider ID Dupes, Name Dupes, Stale, Orphaned and Dangling Refs.
  • The duplicate counts in CI Type Breakdown are links: the table's help text reads "Click a duplicate count to see the affected records." In the dialog, select a record's name to open it on the CMDB, with the name entered in the grid's Search: box for that visit. When you have corrected the record, return to CMDB Data Correctness and select Refresh to see the new counts.
  • The Dangling Refs counts are links too. They list the names that relationships point at but which have no record, most-referenced first. Import or add the missing records, or remove the stale relationships on the Relationship grid.
  • This screen reports issues only. It has no merge, archive or fix action.
  • OVERALL CORRECTNESS can show 0.0% even when CRITICAL DUPLICATES is 0 and only part of each CI type is stale, so read CI Type Breakdown as well as the card.

See CMDB Data Correctness.

5. Correct the records

Go to Discover → CMDB, then select the card for the CI type to correct (for example Application or Device).

  • The Device grid lists only In scope devices, and the Application and Database grids hide Out of scope records, until you select Display all scopes.... Select it if a record you want to correct is not listed.

  • Select Validations ("Toggle Highlight Mandatory and Suggested Fields") to highlight the fields set on Field Validations.

  • To correct one record, select its row by selecting any cell except Name, then select the edit icon on the grid toolbar. (Selecting the Name opens the record's edit form at once.) The Edit entry form opens with the record's fields, grouped in section cards. Change the fields, then select Save.
  • To correct several records in the grid itself, use Enable spreadsheet editing.

The CMDB grids start without a search or column filter each time you open them. If the grid shows fewer rows than you expect, clear the Search: box and the filter row, and check the scope filter.

6. Check the scores again

The two data quality dashboards load their figures when the page opens. They have no refresh button: open the dashboard again after correcting records. On CMDB Data Correctness, select Refresh.

See CMDB and CMDB Data Correctness.

Troubleshooting

Symptom Cause What to do
A device opened from a duplicates dialog is not listed on the CMDB. The Device grid lists only In scope devices until you select Display all scopes.... Select Display all scopes....
A duplicates dialog lists fewer records than its count. The dialogs keep their Search: text between opens, shared across CI types. Clear the dialog's Search: box.
OVERALL CORRECTNESS shows 0.0% with no Provider ID duplicates. The percentage is worked out on the server, and the screen does not show the formula. Read the CI Type Breakdown table for the counts per CI type.
A score has not changed after you corrected records. The dashboards load their figures only when the page opens. Open the dashboard again. On CMDB Data Correctness, select Refresh.
Field Completion Rates reads "No field validation rules configured." No Application fields are set on Field Validations. Add Mandatory or Suggested rows on Settings → Field Management → Field Validations.
The red banner "Failed to load correctness metrics" appears. The metrics could not be loaded. Check the Stale threshold (days) value, then select Refresh. If it persists, reload the page.
The stale count does not reflect the threshold you set. The threshold applies only when you select Refresh, and goes back to 30 when you leave the page. Enter the threshold and select Refresh on each visit.
A dangling reference name cannot be selected. The names in the Dangling References dialog are not links. Import or add the missing record, or remove the relationship on the Relationship grid.

Questions people ask

Does correcting a record update the score straight away? Not on screen. Open the dashboard again, or select Refresh on CMDB Data Correctness.

Which fields are scored? The Mandatory and Suggested rows for that CI class on Field Validations.

Why can Dangling Refs go over 100%? Dangling references are counted against the number of records of that type. Where relationships point at more missing names than there are records, the percentage goes over 100%.

Can I merge or delete duplicates from CMDB Data Correctness? No. The screen only reports. Correct or remove the records on Discover → CMDB.

Why is a tier missing from the Device Data Quality chart? A tier with no devices is left out.

Does the CMDB keep the name I opened from a duplicates dialog? No. The search applies to that visit only.