Skip to content

CSV Import & ETL

Purpose

This workflow loads data from a CSV file into the Clarity Transform CMDB. In Clarity, a CSV import is a saved import job. Each job has a Ci Class (Application, Cluster, Database, Device or Relationship) and is run through a five-step wizard: Upload CSV, Reconciliation Rules, Column Mapping, Verify Data and Import Data, in that order. A saved job is run again from its own wizard; there are no separate import templates.

For the screens themselves, see CSV Import.

Before you start

  • Have the data as a CSV file. Step 1 of the wizard reads "Upload a CSV file to start the ERTL process." (the wording, including "ERTL", is the product's own).
  • Use one file per CI class. A job has a single Ci Class, so devices and applications, for example, go through separate jobs.
  • Do not confuse this with Import CSV on a grid toolbar. That loads rows into the grid's own table ("Rows with an existing ID will be updated; rows without an ID will be inserted as new records."), not through the wizard.

Stage 1: open or create the import job

  1. Go to Discover → Integrations, then select the CSV Import card.
  2. The grid lists the saved import jobs with the columns Name, Ci Class, Source, Last Run, Last Result and Last Run By.
  3. To create a job, select the add icon on the grid toolbar. The Create new entry form opens with three fields:
    • Name
    • Ci Class: a list with Application (the default), Cluster, Database, Device and Relationship.
    • Source
  4. Select Save.
  5. Select a job's Name to open its wizard.

The create form has no schedule field, although step 5 of the wizard says the import runs daily "If selected during job creation".

The grid remembers its Search: text, column filters, sort order and page size between visits. If the grid shows fewer rows than you expect, clear the Search: box and the filter row.

Stage 2: run the wizard

The wizard shows all five steps on one page, each with its own button. The steps are, in order:

  1. Step 1 - Upload CSV: select Upload CSV. The Import data from CSV page opens with a file input. Choose the file, then select Upload CSV (the button stays disabled until a file is chosen).
  2. Step 2 - Reconciliation Rules: select Reconciliation Rules. The step reads "Define reconciliation rules to specify which fields are managed by each import job." The Data Reconciliation Rules page shows the job's CI class, a Search for a column... box and one tile per column, with Back and Save.

  3. Step 3 - Column Mapping: select Column Mapping. The step reads "Map source data fields with their corresponding elements in the target data structure." The Map CSV Columns page has one row per Database Column, with a CSV Column list and an If Empty list:

    • Preserve: "Keep existing DB value when CSV is empty"
    • Overwrite: "CSV always wins, even if empty"

    Select Save, or Back to leave the page.

  4. Step 4 - Verify Data: select Verify Data. The step reads "Import a sample of the source data and verify the mappings that you have created are satisfactory."

  5. Step 5 - Import Data: select Import Data. The step reads "Run the full data import on demand. If selected during job creation, the import will run on a daily schedule."

If a wizard step returns you to the job list, read the error message shown on the job list: it appears as a red toast when the list opens and gives the reason the step could not continue.

When the import has run, the job's row on the job list shows Last Run, Last Result (values seen in this build: Success and Failed) and Last Run By. Select Reload table data if the row has not changed.

To repeat an import later, open the same job from the grid and select Import Data again.

Stage 3: check the imported records

  1. Go to Discover → CMDB, then select the card for the job's CI type (for example Device, Application or Relationship) and review the records in the grid.
  2. To check field completeness, go to Discover → Data Quality → Device Data Quality or Discover → Data Quality → Application Data Quality and review the Field Completion Rates chart.

See CMDB, Device Data Quality and Application Data Quality.

Troubleshooting

Symptom Cause What to do
A wizard step sends you back to the job list with a red message The step could not continue. The message gives the reason. Read the message before you leave or reload the page, fix the cause (for example the file or the mapping), then open the job's Name and repeat the step.
Upload CSV stays greyed out on the Import data from CSV page No file has been chosen. Choose the CSV file first.
The job's Last Result is Failed The last import did not complete. Open the job's wizard and work through Verify Data before Import Data again.
Imported values have blanked existing data The column's If Empty is set to Overwrite ("CSV always wins, even if empty"). Set If Empty to Preserve ("Keep existing DB value when CSV is empty") on Column Mapping for columns your file does not always fill, and import again.
Records went into the wrong CI class The job's Ci Class does not match the file. Create a job with the right Ci Class for that file.
The toolbar's Import CSV reports "Import complete: ... inserted, ... updated, ... skipped." but the CMDB is unchanged Import CSV on the toolbar loaded rows into the list of import jobs, not the CMDB. Use a job's wizard instead. If unwanted job rows were added, select them and use Remove selected.
The import job list shows fewer jobs than expected The grid remembers its Search: text and column filters between visits. Clear the Search: box and the filter row.

Questions people ask

Can a CSV import run on a schedule? Step 5 says "If selected during job creation, the import will run on a daily schedule.", but the create form has no schedule field. Plan to run imports on demand with Import Data.

Do I have to create a new job for every file? No. Open the same job and work through the wizard again with the new file.

What do the If Empty choices mean? Preserve keeps the value already in Clarity when the CSV cell is empty. Overwrite writes the CSV value even when it is empty.

What are reconciliation rules for? The step reads "Define reconciliation rules to specify which fields are managed by each import job." They decide which fields this job owns.

How do I know the import worked? Check the job's Last Result on the job list, then open the matching CI type card on Discover → CMDB.