Ingestion feed admin

Outcome

A new feed is onboarded — push API key minted or SFTP poll configured — and inbound artifacts are visible for triage without engineering help.

Prerequisites

• ingestion.create for mint / rotate / config changes.
• ingestion.read to view artifacts.

Mental model

Concept	Detail
Each feed is one row in ingestion.source_feed	transport_type decides the UI: SFTP shows poll config, PUSH_API shows API keys, UPLOAD exposes neither.
Plaintext API keys exist only at mint/rotate	Server bcrypts the key, stores key_prefix + key_last_four on ingestion.api_credential, never returns plaintext again. Modal copy is the only reveal.
SFTP credentials are write-only	connection_config lives on ingestion.source_system as pgp_sym_encrypt-protected bytes. Saving without filling the credential sub-form leaves the existing encrypted value intact.
Cap: 5 active keys per feed	Service enforces via partial index WHERE revoked_at IS NULL. UI banner mirrors the threshold.

Mint a push API key

1. Open /admin/ingestion/feeds/<feedId>?tab=keys.

2. Click Mint key, optionally name it (e.g. "EMR vendor prod").

3. Copy the plaintext value from the reveal modal — it cannot be shown again. Store in your secret manager and configure the client to send Authorization: Bearer <plaintext>.

4. Verify usage by sending a test request and reloading the row's Usage drawer. The 1h window updates within a minute.

Rotate a push API key

1. Open the row → Rotate → confirm.

2. Copy the new plaintext from the reveal modal.

3. Switch your client over to the new key — the old one starts returning 401s on the next request (revoked in the same transaction).

Configure SFTP for a feed

1. Open /admin/ingestion/feeds/<feedId>?tab=sftp.

2. Fill the cron (shorthand every 15m, hourly, or a 5-field expression), inbound/archive/error paths, and an optional filename pattern.

3. Pick credential mode: Set password, Set private key, or Keep current credentials (when updating schedule only). Provide host, port, username, and the credential.

4. Click Save SFTP configuration. The status row re-probes via /feeds/:id/health and flips to Healthy once reachable.

5. Click Trigger poll to enqueue an immediate run. The toast carries the job id; the resulting artifacts appear under the Artifacts tab once processed.

Investigate a failed batch

1. Open the Artifacts tab. Filter by FAILED or PARTIAL.

2. Click the row to open /admin/ingestion/feeds/<feedId>/artifacts/<artifactId>. The detail page shows parsed preview (first 200 rows via /artifacts/:id/sample) and the most recent batch summary with total / processed / success / error counts.

3. Click Browse records to filter by status (ERROR by default) and read each row's error_detail JSON.

4. Click Retry failed records to flip every FAILED record back to PENDING. The processor picks them up on the next tick.

Decision tree — retry vs. raise an incident

Symptom	Likely cause	Action
Single record error_detail is a known transform failure (date format, missing required field)	Source data quality	Edit source, re-poll. Dedupe (row_hash) skips already-processed records.
All records in a batch failed with the same error	Mapping drift	Open the mapping editor, update the mapping, Retry the batch.
Health badge Unhealthy for > 15 min	SFTP host down or credentials rotated upstream	Re-test connection. If host is up, re-save credentials.
Mint returns 409 CREDENTIAL_CAP_REACHED	Cap of 5 active keys hit	Revoke an unused key before issuing a new one.
Trigger poll returns 503 JOBS_DISABLED	Consumer not running in this environment	Start it, retry.

Validation

Check	Expected
Mint → request with key	200
Rotate → old key	401
SFTP health	Healthy after save
Failed batch → retry	Records flip PENDING → SUCCESS on next tick

Cross-references

• Mapping editor for transform-level fixes.
• Fixed-width parser config for FIXED_WIDTH feeds.
• Handle SFTP failure for EDI-side SFTP issues (different infrastructure path).

Next

5.3 — Fixed-width parser config builder