Skip to main content

Ingestion mapping editor

Outcome

Per-feed mappings are authored, dry-run previewed, published, and rolled back without a developer in the loop.

Prerequisites

  • ingestion.create to author and publish.
  • ingestion.read to dry-run.

Mental model

ConceptDetail
A mapping is per source feedEach row in ingestion.mapping_definition is one immutable version. The current one is is_current=TRUE.
Publishing creates a new rowServer takes a per-feed pg_advisory_xact_lock, computes version = MAX(version) + 1, flips the prior current row to is_current=FALSE, inserts the new row in a single transaction. Concurrency-safe.
Drafts are client-side onlyNo DRAFT state on the table — closing the browser tab abandons the draft. Collaborate by copying YAML out of band.
Rollback = republish prior YAMLNo is_current=TRUE toggle. Open the prior version → "Edit as draft" → Publish. The new version's YAML matches the old; it lands as vN+1.

Author a new mapping

  1. Sign in as an operator with ingestion.create.

  2. Open /admin/ingestion/mappings and pick the target feed in the combobox. The page navigates to /admin/ingestion/mappings/:feedId.

  3. Click New draft — lands on the editor with a starter YAML (charge_event target, single external_id field).

  4. Visual builder tab: add fields, pick the transform type, set parameters (e.g. inputFormat=MM/DD/YYYY for DATE). Validation rules edit the same way.

  5. YAML tab: free-form Monaco edit. The schema validator (zod-mirrored from MappingService.validateMappingYaml) flags missing target.type, empty fields, unknown transform kinds, etc. inline.

  6. Diff tab: side-by-side Monaco diff against the currently published version. Empty when there is no prior version.

  7. Dry run: paste a sample row (CSV header+row, or JSON object) into the right-hand panel. Each field's mapped value (and any transform-time error) renders in the preview table. LOOKUP and CUSTOM transforms show a "server-side only" notice — they require a real run.

  8. Click Publish. Pick Effective from (defaults to today) and optional Effective to. Confirm. The toast announces the new vN. The version list automatically picks up the new row and marks it Current.

Roll back to a prior version

  1. Open /admin/ingestion/mappings/:feedId and click the prior version in the list.

  2. Click Edit as draft in the YAML pane header.

  3. Confirm the YAML matches the desired prior content (no edits needed).

  4. Click Publish. The new row gets vN+1 but the YAML matches vN-1. The previously broken vN is Superseded.

Effective-date windows

  • effective_from is required.
  • effective_to is optional. Set it when sunsetting a vendor or switching to a new schema on a known date.
  • getEffectiveMapping(feedId, asOfDate) resolves the row that is current AND whose effective window contains asOfDate. Multiple current rows with overlapping windows is not a defined state — the publish path always flips the prior current row, so this only matters when seeding via SQL.

Validation

CheckExpected
Schema validator (YAML tab)Green before Publish
Dry-run previewSample row maps as expected
mapping_definition after publishNew row, is_current=true; prior row is_current=false

Troubleshooting

SymptomCauseFix
Publish 400 with mapping_yaml missing target.typeClient + server schema have driftedRe-run the schema validator; if still failing, file a follow-up to re-sync apps/rcm-app/.../yaml-schema.ts with mapping-service.ts.
Publish 409 UNIQUE violation on (source_feed_id, version)Concurrent publishes serialized via pg_advisory_xact_lock; this should not happenInspect mapping_definition and re-run migration if version numbers are non-contiguous.
Editor opens with empty YAML for a feed that has versionsRoute response missing rawYaml. Confirm feed isn't filtered by effective_from > asOfDateIf getEffectiveMapping returns null, editor falls back to default scaffold.

Cross-references

Next

5.2 — Ingestion feed admin