Fixed-width parser config builder

Outcome

A FIXED_WIDTH feed's parser config (ingestion.source_feed.parser_config) is authored visually, validated against a sample, and saved without a migration.

Prerequisites

• ingestion.create (write) and ingestion.read (dry-run preview).
• A small (≤ 50 lines) representative sample on hand.
• Feed type set to FIXED_WIDTH (or be ready to flip from CSV).

Pre-flight

1. Confirm the feed is FIXED_WIDTH. If you're inheriting an SFTP roster drop seeded as CSV, flip the column first:

   UPDATE ingestion.source_feed
      SET feed_type = 'FIXED_WIDTH'
    WHERE source_feed_id = '<id>';

   The Parser Config tab is hidden until the feed type matches.

2. Pull a small representative sample. EBCDIC samples work for the dry-run only if you transcode them upstream — the parser rejects EBCDIC at run time pending a pilot.

Build the schema (Visual pane)

1. Navigate to Configuration → Ingestion → Feeds → <feed> → Parser Config.

2. FileMeta block: set encoding + framing.

   Framing	When
   newline	Newline-delimited rosters (most common).
   record-length	Mainframe / AS-400 contiguous-byte exports — enter bytes-per-record.

3. Records: add HEADER + DETAIL + TRAILER (or just one DETAIL for flat layouts). For multi-record schemas every record needs a Discriminator start + Discriminator value — the literal byte at the offset that identifies the record type (typically H / D / T at offset 0).

4. Fields: declare each field's name, start (0-based), length, type. The grid validates inline:

   Failure	Why
   Duplicate names within a record	Rejected
   Overlapping ranges	Highlighted red
   date types without dateFormat	Rejected
   impliedDecimals on non-number	Rejected

5. Expectations (collapsed by default at the bottom of each record):

   • detailCountField on the TRAILER — a numeric field that should equal the count of DETAIL rows.
   • hashField + hashFieldSource to verify a sum-of-DETAILs total.

   These run as soft checks: mismatches surface as validationErrors in the dry-run summary instead of failing the parse.

Preview against a sample

1. Switch to the Preview pane.

2. Paste up to ~50 lines from the sample file into Sample input.

3. Click Run preview. The server runs parseFixedWidthFile with the unsaved schema and returns the first 50 parsed rows + the summary. The result table highlights every field column declared on each record.

4. Inspect the summary banner:

   Counter	What
   Parsed rows	Match the row count you pasted (modulo trailing blank lines).
   HEADER / DETAIL / TRAILER	Reflect role assignments — a 0 for DETAIL usually means the discriminator at offset 0 doesn't match D for any DETAIL row, so the parser left them under recordType=UNKNOWN.
   validationErrors	DETAIL_COUNT_MISMATCH / HASH_TOTAL_MISMATCH from your expectations. Soft — the rows still parse — but a mismatch in production fails the SLA check.

Save and activate

1. Save parser config writes the JSON onto the JSONB column. The server zod-validates again on PATCH; an invalid schema reaches the column only if you bypass the UI.

2. No separate activation flag — the next inbound batch processed by the process-batch worker picks up the schema via IngestionService.getParserConfig.

3. To roll back, open the YAML pane, copy the previous JSON from git history (or the prior version of parser_config from PITR), paste, save.

Common pitfalls

Symptom	Cause	Fix
DETAIL_COUNT_MISMATCH expected 99 actual 12	TRAILER's detailCountField points at the wrong byte range, OR file truncated at source	Re-run preview; if 12 looks right, the sample is incomplete.
shorter than required for field X	A field's start + length exceeds row width	For newline-framed files this usually means a vendor padded only the body and trimmed trailing whitespace. Shrink the field or accept shorter rows after re-checking the vendor spec.
VALIDATION_ERROR on save	Visual pane permits invalid intermediate states	Switch to YAML pane for line-level Monaco markers; fix; save. The wire response includes the full zod issue list under details.issues.
records is empty after save	JSONB serializes the whole document on PATCH; an empty visual pane writes {records:[]} not null	Send parserConfig: null instead — explicitly clears the column.

Validation

Check	Expected
Schema-valid badge	Green before Save
Dry-run preview	Reasonable HEADER/DETAIL/TRAILER counters
parser_config after save	JSONB matches the schema
Next inbound batch	Parses with the new schema

Cross-references

• Mapping editor for the transform side that runs after parsing.
• Feed admin for poll config + retry.

Next

6.1 — Per-diem & institutional billing