ADR-0058: Deletion is soft — DELETED is a status, not a row removal

Deleting a data entity sets its status to DELETED, not removing the row; list and search hide it, the detail view still shows it, and housekeeping physically purges it after a TTL.

Status

Accepted. Reconstructed from the codebase on 2026-05-31; the decision is live in the source today.

Context

A data entity can leave the catalog for several reasons — an owner deprecates it, a collector stops reporting it, an operator removes it. Hard-deleting the row the moment that happens would throw away the ability to show that it was deleted, to restore it, or to diagnose why it went; and it would force a synchronous cascade across the entity's entire dependent graph (lineage, ownership, metadata, alerts, metrics, task runs, dataset structure, …) onto the request that triggered it. The platform needed a deletion model that keeps deleted entities inspectable for a while and moves the expensive cascade off the request path.

Decision

Deletion is a status, not a row removal. DataEntityStatusDto is a closed five-member enum — UNASSIGNED, DRAFT, STABLE, DEPRECATED, DELETED — and "deleting" an entity sets its status to DELETED; the row stays. Status is a settable resource property (PUT /api/dataentities/{id}/statuses, gated by DATA_ENTITY_STATUS_UPDATE). Two statuses carry a structural isSwitchable flag (DRAFT, DEPRECATED); a scheduled job uses it to auto-advance eligible entities to DELETED.

The read surfaces treat a soft-deleted entity asymmetrically, on purpose: list, search, and lineage filter DELETED out (addSoftDeleteFilter), so the working catalog stays clean — but the per-entity detail read includes it (includeDeleted(true)), so the UI can still render, diagnose, or restore a deleted entity.

The physical purge is deferred to housekeeping. A housekeeping job selects entities that have been in DELETED longer than a configured TTL (measured from the status-update timestamp) and physically removes them together with their entire dependent graph in one transaction. Until that window elapses, a soft-deleted entity is recoverable.

Consequences

• A deleted entity remains inspectable and restorable during the TTL window — the detail view still resolves it — while disappearing from everyday list/search/lineage results.

• Deletion is cheap at request time (a single status write); the costly cascade across roughly two dozen dependent tables runs later, inside the housekeeping job's transaction, not on the user's request.

• There is a grace period, then permanence: once the TTL elapses and housekeeping runs, the entity and all its dependents are gone for good. Operators tune the window through the housekeeping TTL settings.

• The purge window is measured from the status-update timestamp, so the model leans on that timestamp being set whenever status changes; the housekeeping subsystem (ADR-0045) owns the purge schedule and can be opted out of (ADR-0046).

• The housekeeping purge is the one place physical deletion cascades (by entity id and ODDRN across the dependent graph); everywhere else in the platform, "deleted" means status = DELETED, and code that reads entities must opt in with includeDeleted to see them.

Evidence

• odd-platform-api/.../dto/DataEntityStatusDto.java:11-16 — the closed five-member enum (id, isSwitchable): UNASSIGNED(1,false), DRAFT(2,true), STABLE(3,false), DEPRECATED(4,true), DELETED(5,false); DELETED is a status value, not a row removal.

• odd-platform-api/.../service/job/DataEntityStatusSwitchJob.java:21-30 — a 10-minute scheduled (ShedLock) job that fetches getPojosForStatusSwitch() and changeStatusForDataEntities(pojos, DELETED) — the auto-advance of switchable statuses to DELETED.

• odd-platform-api/.../repository/reactive/ReactiveDataEntityRepositoryImpl.java:119 — addSoftDeleteFilter(...) excludes DELETED rows from list/datasource/namespace reads (:160,168); :186,208,220 — the detail read sets .includeDeleted(true), the deliberate asymmetry that surfaces soft-deleted entities to the detail view.

• odd-platform-api/.../housekeeping/job/DataEntityHousekeepingJob.java:73-82 — selects DATA_ENTITY where STATUS == DELETED and STATUS_UPDATED_AT <= now − dataEntityDeleteDays, then :84-129 physically deletes the entity and its dependent rows across ~25 tables, ending in deleteFrom(DATA_ENTITY) (:124-126).

See also

• ADR-0045 — Housekeeping is a separate subsystem from partition management — the subsystem that runs the deferred physical purge.

• ADR-0046 — Housekeeping ships enabled by default (opt-out) — so the purge runs unless an operator turns it off.

• ADR-0073 — ODDRN is the universal identity for every entity — the purge cascades across the dependent graph keyed on the entity's ODDRN.