> For the complete documentation index, see [llms.txt](https://docs.opendatadiscovery.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.opendatadiscovery.org/developer-guides/architecture-decision-log/adr-0070-pull-push-ingestion-contract.md). # ADR-0070: Ingestion is one wire contract shared by pull and push producers ## Status **Accepted.** Reconstructed from the codebase on 2026-05-31; the decision is live in the source today. ## Context Metadata originates in many kinds of system — warehouses, orchestrators, BI tools, ML platforms — each with its own access model. Some can be read on a schedule by an external process; others are best reported by the system itself as it runs, for example a pipeline emitting lineage per run. A platform could define a bespoke integration per source, but that couples the platform to every source's API and makes every new source a platform change. ODD's constraint is the opposite: any producer should be able to feed any compliant platform without the two being coupled. ## Decision **Metadata reaches the platform through one wire contract — the ODD Specification — that two producer strategies share.** Pull producers (**collectors**) poll their source on a schedule and map what they read into the contract; push producers (**adapters / push-clients** — in-process plugins, standalone gateways, SDK/CLI callers) emit on the source's own cadence. The two strategies differ only in *who initiates, and when*; both produce the same Specification payload and send it to the same Ingestion API. The platform implements that contract as a **generated server interface built from the shared Specification** — it depends on the published `ingestion-contract-server` artifact rather than defining its own ingestion schema — and then processes every payload **identically**: the same idempotent, ODDRN-keyed upsert (ADR-0073), regardless of how the metadata was obtained. The pull/push distinction exists at the producer; once a payload crosses the Ingestion API, it is gone. Everything downstream — storage, lineage, search, alerting — is written once and is source-agnostic. ## Consequences * Producers and the platform are **decoupled by the Specification**: any producer that speaks it can feed any compliant platform, and a new source is a new adapter, not a platform change. The platform never learns a source's native API. * **Pull and push coexist by design** — a pull collector can index a warehouse's catalog while a push plugin reports per-run lineage on the same entities; they converge because both attach the same ODDRN identity (ADR-0073). * Because the split is **only at the producer**, a contributor adding ingestion behaviour works against the contract, not against producer types. Branching platform logic on "was this pulled or pushed" would violate the decision — that distinction is not visible past the Ingestion API by design. * The cost is indirection: the platform cannot ingest a source the Specification cannot yet express, and supporting a genuinely new metadata shape means evolving the shared contract (and regenerating its artifact), not just editing the platform. * The strategies carry different operational trade-offs — pull trades freshness against polling load and cadence; push trades infrastructure simplicity against producer availability (a producer that never emits leaves a gap the platform cannot poll for). Architecture.md's "Pull vs Push" section is the guidance on when to choose which. ## Evidence * `odd-platform-api/.../controller/IngestionController.java:31` — the controller `implements IngestionApi`, the generated ingestion contract interface; `:38-45` — `postDataEntityList(Mono)` is the single ingestion entry, which validates the payload and delegates to `ingestionService.ingest(...)` with **no branch on producer type**. * `gradle/libs.versions.toml:6,65` — the platform depends on `org.opendatadiscovery:ingestion-contract-server` (the server side of the ODD Specification), rather than defining its own ingestion schema: the wire contract is **shared, not platform-specific**. * `odd-platform-api/.../service/ingestion/IngestionServiceImpl.java:86,93-98` — the payload is processed uniformly: incoming items are keyed by ODDRN and partitioned into create vs update — identical regardless of which producer sent it (ADR-0073). ## See also * [Architecture](/introduction/architecture.md) — the structural mental model: the data-flow stages, the deployment topology (Collector / Push-client), and the "Pull vs Push — when to choose which" guidance. This ADR is the decision behind that structure; the page is the operator's view of it. * [Main Concepts — The architecture chain](/introduction/main-concepts.md#the-architecture-chain) — the producer-side vocabulary (adapter, plugin, collector, push adapter, push-client). * [ADR-0073 — ODDRN is the universal identity for every entity](/developer-guides/architecture-decision-log/adr-0073-oddrn-universal-identity.md) — the identity producers attach so that pull and push converge on the same entity. * [ADR-0001 — Contract-first HTTP layer](/developer-guides/architecture-decision-log/adr-0001-openapi-generated-controller-interfaces.md) — the same contract-first posture: the ingestion API is a generated contract, not hand-written. * [ADR-0072 — The platform is a contract-first, reactive, two-language stack](/developer-guides/architecture-decision-log/adr-0072-contract-first-reactive-stack.md) — the reactive stack that serves this ingestion contract. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.opendatadiscovery.org/developer-guides/architecture-decision-log/adr-0070-pull-push-ingestion-contract.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.