Notifications

How alerts move in and out of ODD Platform — outbound through Slack incoming webhook, generic webhook, or SMTP email; inbound from Prometheus AlertManager as Distribution Anomaly alerts.

Notifications is the subsystem that moves alerts across the platform's boundary — out, through Slack, generic webhook, or SMTP email; or in, from a Prometheus AlertManager pushing distribution-anomaly events through the inbound webhook. Alerts the platform raises internally are described under Alerting; this page is about the channels that connect those alerts to the rest of an operator's stack.

The subsystem is disabled out of the box — the outbound dispatcher needs notifications.enabled=true plus a configured PostgreSQL logical-replication slot, and the inbound AlertManager webhook is gated by network reachability (it has no application-level toggle). Each channel is independently configurable; deployments use whatever subset matches their on-call workflow.

For setup steps and the full key list, see Configure ODD Platform → Enable Alert Notifications and Configure ODD Platform → Prometheus AlertManager Integration. This page is the user-facing description of what each channel does, what it carries, and what to know before enabling it.

What an outbound notification carries

Every alert dispatched out of the platform carries:

1. The name of the entity the alert was raised on.

2. The data source and namespace of that entity.

3. The owners attached to that entity (see Owners).

4. Affected downstream entities — the lineage neighbours within notifications.message.downstream-entities-depth levels (default 1).

Clickable links inside notification messages resolve back to the platform UI using the operator-set odd.platform-base-url — both the Slack and email senders consume that key. The generic webhook receiver does not consume odd.platform-base-url; it gets the full alert payload directly and is expected to construct any URLs it needs from that payload.

Outbound channels

The platform supports three outbound channels. They can be enabled together or individually; an alert dispatched to multiple channels is delivered to every channel that is enabled.

Slack incoming webhook

The platform POSTs a formatted alert message to a Slack incoming webhook URL. This is outgoing-only — there is no thread state, no reply ingestion, no per-channel routing logic; the platform writes one message per alert dispatch.

Configured by notifications.receivers.slack.url. The same Slack workspace can also host the bidirectional Data Collaboration Slack app — the two integrations use different Slack mechanisms (incoming webhook vs OAuth + Events API) and are configured independently.

This is the alert webhook, not the Discussions Slack app. The alerting Slack integration is a one-way notifications.receivers.slack.url POST — no replies, no thread state. The full Slack app used by Data Collaboration is a separate integration via OAuth (datacollaboration.slack-oauth-token) and the Slack Events API. Each is configured separately; enabling one does not enable the other. See Main Concepts → Terms & Aliases for the side-by-side comparison.

Email (SMTP)

The platform sends a formatted alert email through an operator-supplied SMTP relay. Configured by the notifications.receivers.email.* family of keys (host, port, protocol, sender, password, recipient list, optional STARTTLS). The reference walkthrough using Gmail's SMTP is on the operator-side configuration reference at Configure ODD Platform → Example: Gmail SMTP.

The SMTP integration carries several JavaMail-default-driven limitations that operators should know before relying on it for production on-call:

SMTP timeouts are unset — an unreachable SMTP server will hang notification delivery. The JavaMail defaults for connection / read / write timeouts are infinite, and ODD Platform does not override them. An unreachable or stalling SMTP relay will block the notification thread until the TCP stack tears the connection down. Use a relay you can monitor for availability separately from ODD.

Silent partial delivery if one recipient fails. The email sender iterates through notifications.receivers.email.notification.emails recipient by recipient; if recipient N fails (bad address, mailbox full, server-side rejection), the loop stops — recipients N+1, N+2, … never receive the alert. There is no retry and no partial-failure metric. Keep the recipient list short and use distribution lists on the SMTP side for fan-out.

The full set of email-side caveats (only STARTTLS supported, self-signed cert workaround, non-ASCII charset issue) lives on Configure ODD Platform → Enable Alert Notifications → Known limitations — operators authoring the SMTP configuration should walk through that section before enabling the channel.

Generic webhook

The platform POSTs the full alert payload (JSON) to an operator-supplied URL. Configured by notifications.receivers.webhook.url. Use this when you want to fan alerts into a tool the platform doesn't natively integrate with — your own incident-management system, an HTTP-driven on-call tool, a custom consumer that fans further to other channels.

Unlike the Slack and email senders, the generic webhook does not consume odd.platform-base-url — the receiver is expected to extract any URLs it needs from the alert payload itself.

Inbound channel — Prometheus AlertManager webhook

In addition to the alerts the platform raises from its own ingestion / evaluation pipeline, ODD exposes an inbound webhook that accepts Prometheus AlertManager notifications. Each accepted alert becomes a Distribution Anomaly alert on the referenced data entity, indistinguishable from internally-raised alerts from there on (it shows on the Alerts section, on the entity's page, and in the activity feed).

The endpoint is POST /ingestion/alert/alertmanager; the platform reads alerts[].labels, alerts[].generatorURL, and alerts[].startsAt from the AlertManager webhook body and ignores other top-level fields. The full payload shape, the AlertManager route/receivers example, the rule-side label requirement, and the authentication caveat live on Configure ODD Platform → Prometheus AlertManager Integration.

The entity_oddrn label is required. The platform reads alerts[].labels["entity_oddrn"] to attribute each inbound alert to a data entity. Alerts pushed without this label end up orphaned — stored, but not surfaced on any entity page. Configure your AlertManager route or your alerting rules to always include the target entity's ODDRN as a label. Configuring entity_oddrn on every alert this webhook receives is the operator's responsibility; the platform does not synthesise it.

The AlertManager webhook is unauthenticated. ODD Platform whitelists the entire /ingestion/** namespace at the Spring Security layer, and the ingestion auth filter (auth.ingestion.filter.enabled) only guards POST /ingestion/entities — it does not cover this AlertManager endpoint. Anyone with network reach to the platform can POST arbitrary alerts on any entity ODDRN they can guess. Protect the endpoint at the perimeter: a private network, a NetworkPolicy in Kubernetes that admits only the AlertManager pod, an authenticating reverse proxy, or mTLS termination at the load balancer. See Configure ODD Platform → AlertManager Integration → Authentication for the recommended controls.

Setting up notifications

The full step-by-step — the PostgreSQL replication prerequisites (max_wal_senders, wal_level, max_replication_slots, the REPLICATION role grant, AWS RDS specifics), the notifications.* configuration keys, the YAML and environment-variable formats, the Gmail SMTP example, and the AlertManager receiver / rule example — lives on Configure ODD Platform → Enable Alert Notifications and Configure ODD Platform → Prometheus AlertManager Integration. Operators wiring up notifications work from those two sections — this page is the orientation surface, not the configuration reference.

Email notification example

Disabling notifications

Set notifications.enabled=false to stop the platform from dispatching outbound notifications. The PostgreSQL replication slot and publication created by the platform persist after the toggle flips — clean them up explicitly to avoid the database holding WAL indefinitely. The cleanup SQL plus the SQL-injection-safe steps live on Configure ODD Platform → Enable Alert Notifications → Cleaning up.

Disabling outbound notifications does not disable the AlertManager inbound webhook — the inbound endpoint is gated only by network reachability today. To stop accepting inbound AlertManager events, remove the AlertManager-side receiver pointing at the platform or block the perimeter route to /ingestion/alert/alertmanager.

Where to next

• For the alert types the platform raises internally (failed jobs, failed DQ tests, schema drift, distribution anomalies) and the alert lifecycle (OPEN, RESOLVED, RESOLVED_AUTOMATICALLY) → Alerting.

• For the operator-side configuration keys, the PostgreSQL replication prerequisites, and the SMTP / AlertManager caveats → Configure ODD Platform → Enable Alert Notifications and Configure ODD Platform → Prometheus AlertManager Integration.

• For the activity-feed events that record alert state transitions (OPEN_ALERT_RECEIVED, RESOLVED_ALERT_RECEIVED, ALERT_STATUS_UPDATED, ALERT_HALT_CONFIG_UPDATED) → Activity Feed.