ADR-0070: Ingestion is one wire contract shared by pull and push producers

Pull collectors poll on a schedule, push adapters emit on the source's cadence; both speak one ODD Specification contract, and the platform processes every payload identically.

Status

Accepted. Reconstructed from the codebase on 2026-05-31; the decision is live in the source today.

Context

Metadata originates in many kinds of system — warehouses, orchestrators, BI tools, ML platforms — each with its own access model. Some can be read on a schedule by an external process; others are best reported by the system itself as it runs, for example a pipeline emitting lineage per run. A platform could define a bespoke integration per source, but that couples the platform to every source's API and makes every new source a platform change. ODD's constraint is the opposite: any producer should be able to feed any compliant platform without the two being coupled.

Decision

Metadata reaches the platform through one wire contract — the ODD Specification — that two producer strategies share. Pull producers (collectors) poll their source on a schedule and map what they read into the contract; push producers (adapters / push-clients — in-process plugins, standalone gateways, SDK/CLI callers) emit on the source's own cadence. The two strategies differ only in who initiates, and when; both produce the same Specification payload and send it to the same Ingestion API.

The platform implements that contract as a generated server interface built from the shared Specification — it depends on the published ingestion-contract-server artifact rather than defining its own ingestion schema — and then processes every payload identically: the same idempotent, ODDRN-keyed upsert (ADR-0073), regardless of how the metadata was obtained. The pull/push distinction exists at the producer; once a payload crosses the Ingestion API, it is gone. Everything downstream — storage, lineage, search, alerting — is written once and is source-agnostic.

Consequences

• Producers and the platform are decoupled by the Specification: any producer that speaks it can feed any compliant platform, and a new source is a new adapter, not a platform change. The platform never learns a source's native API.

• Pull and push coexist by design — a pull collector can index a warehouse's catalog while a push plugin reports per-run lineage on the same entities; they converge because both attach the same ODDRN identity (ADR-0073).

• Because the split is only at the producer, a contributor adding ingestion behaviour works against the contract, not against producer types. Branching platform logic on "was this pulled or pushed" would violate the decision — that distinction is not visible past the Ingestion API by design.

• The cost is indirection: the platform cannot ingest a source the Specification cannot yet express, and supporting a genuinely new metadata shape means evolving the shared contract (and regenerating its artifact), not just editing the platform.

• The strategies carry different operational trade-offs — pull trades freshness against polling load and cadence; push trades infrastructure simplicity against producer availability (a producer that never emits leaves a gap the platform cannot poll for). Architecture.md's "Pull vs Push" section is the guidance on when to choose which.

Evidence

• odd-platform-api/.../controller/IngestionController.java:31 — the controller implements IngestionApi, the generated ingestion contract interface; :38-45 — postDataEntityList(Mono<DataEntityList>) is the single ingestion entry, which validates the payload and delegates to ingestionService.ingest(...) with no branch on producer type.

• gradle/libs.versions.toml:6,65 — the platform depends on org.opendatadiscovery:ingestion-contract-server (the server side of the ODD Specification), rather than defining its own ingestion schema: the wire contract is shared, not platform-specific.

• odd-platform-api/.../service/ingestion/IngestionServiceImpl.java:86,93-98 — the payload is processed uniformly: incoming items are keyed by ODDRN and partitioned into create vs update — identical regardless of which producer sent it (ADR-0073).

See also

• Architecture — the structural mental model: the data-flow stages, the deployment topology (Collector / Push-client), and the "Pull vs Push — when to choose which" guidance. This ADR is the decision behind that structure; the page is the operator's view of it.

• Main Concepts — The architecture chain — the producer-side vocabulary (adapter, plugin, collector, push adapter, push-client).

• ADR-0073 — ODDRN is the universal identity for every entity — the identity producers attach so that pull and push converge on the same entity.

• ADR-0001 — Contract-first HTTP layer — the same contract-first posture: the ingestion API is a generated contract, not hand-written.

• ADR-0072 — The platform is a contract-first, reactive, two-language stack — the reactive stack that serves this ingestion contract.