GenAI assistant

GenAI assistant — proxy natural-language questions to an external AI service. API-only today; configuration, contract, and operator caveats.

The GenAI assistant is an opt-in feature that lets the ODD Platform proxy natural-language questions to an external AI service that the operator deploys and operates separately. The platform itself does not embed an LLM; it forwards the question, waits for a response, and returns it through its API.

The feature is disabled by default (genai.enabled: false) and is API-only today — the generated TypeScript client (GenaiApi.ts, GenAIRequest.ts, GenAIResponse.ts) ships with the platform UI, but no in-app affordance currently calls it. Operators who turn this on either drive it from their own UI / scripts against the platform's /api/genai/ask endpoint, or wait for a future UI surface.

What gets proxied

Each call from the operator → platform → external AI service has this shape:

[client] → POST /api/genai/ask  (JSON: {"body": "<question text>"})
              → platform forwards to: POST {genai.url}/query_data
                                       (JSON: {"question": "<question text>"})
              ← external service returns raw JSON-encoded string
              ← platform un-quotes + Java-unescapes, returns JSON: {"body": "<answer>"}
[client] ← 200 OK (JSON: {"body": "<answer>"})

The platform does not modify the question text it forwards (just rewraps it from body → question). Any catalog context, retrieval-augmentation, or prompt construction must happen inside the external AI service — the platform is a thin proxy.

Configuration

The three genai.* keys (enabled, url, request_timeout), their defaults, the silent-misconfiguration warning when enabled=true is set without url and request_timeout, the working YAML / environment-variable example, and the platform-restart requirement when any of the three change all live on the operator-facing configuration reference at Configure ODD Platform → GenAI Configuration. That page is the canonical home for the platform's configuration keys; this page links rather than embedding so the two surfaces never drift.

External AI service contract

When genai.enabled=true, the platform's genAiWebClient is configured at startup with baseUrl = genai.url and responseTimeout = Duration.ofMinutes(genai.request_timeout). Each call from /api/genai/ask forwards to the external service as:

• Method: POST

• Path: {genai.url}/query_data (the /query_data suffix is fixed in GenAIServiceImpl.java:22)

• Headers: none added by the platform (default WebClient headers; no auth, no API key)

• Request body (application/json):

  { "question": "<the question text from the GenAIRequest.body field>" }

• Response body (the platform expects application/json): a raw JSON-encoded string — that is, the response payload is a single JSON string value (with leading and trailing " quotes). The platform strips the outer quotes via CharMatcher.is('"').trimFrom(item) and then runs StringEscapeUtils.unescapeJava(...) to interpret embedded escape sequences (\", \n, etc.) before placing the result in the GenAIResponse.body field.

A minimal-but-conforming AI service implementation in Python (Flask), for reference:

from flask import Flask, request, jsonify
import json

app = Flask(__name__)

@app.post("/query_data")
def query_data():
    payload = request.get_json(force=True)
    question = payload["question"]
    # ... your AI / RAG / LLM call here ...
    answer = my_ai_pipeline(question)
    # Return a raw JSON-encoded string. The platform expects to read this with
    # bodyToMono(String.class) and trim the surrounding quotes; jsonify wraps
    # it correctly.
    return app.response_class(json.dumps(answer), mimetype="application/json")

The platform sends no authentication to the external service. If the service must reject anonymous traffic, deploy it behind a network policy / mesh / ingress that authenticates / restricts callers — there is no genai.token or similar in GenAIProperties today. The external service is also fully responsible for prompt construction, catalog-aware retrieval, and rate limiting; the platform forwards questions verbatim and returns whatever the service returns.

Platform endpoint

The platform's GenAI surface is a single endpoint:

Method	Path	Operation ID	Purpose
POST	/api/genai/ask	genAiQuestion	Forward a question to the configured external AI service and return its answer.

Request schema (GenAIRequest):

{ "body": "<question text>" }

Response schema (GenAIResponse):

{ "body": "<answer text>" }

When genai.enabled=false, the endpoint responds with a BadUserRequestException carrying the message "Gen AI is disabled" (HTTP 400) — operators get a clear error rather than a silent no-op.

When the external service times out, the endpoint responds with a GenAIException carrying "Gen AI request take longer that {minutes} min" (HTTP 500) — the timeout duration in the message is the configured genai.request_timeout value, so operators see exactly what they configured.

For non-timeout errors from the external service (HTTP 5xx, connection refused, malformed response), the platform wraps the underlying cause in a GenAIException (HTTP 500) and surfaces it on the /api/genai/ask response. There is no retry loop; a single attempt per request.

UI status

API-only today. The platform UI ships generated clients (odd-platform-ui/src/generated-sources/apis/GenaiApi.ts plus the request/response models) but no view, button, or panel currently calls them — verified via grep across odd-platform-ui/src/ excluding generated-sources/. To use the feature, drive POST /api/genai/ask from your own client (curl, an internal app, a custom in-platform fork). When a UI affordance ships, this section will be updated.

Known limitations

• @ConfigurationProperties defaults bite when only genai.enabled is set. url defaults to null (no field initializer in GenAIProperties.java) and request_timeout defaults to 0 (= immediate timeout). The platform accepts the misconfiguration at startup and fails at first request. Set genai.url and genai.request_timeout explicitly when enabling — the operator-side guidance lives on Configure ODD Platform → GenAI Configuration. The upstream-side improvement (defaults declared in GenAIProperties or a startup validation) is queued as a separate platform issue.

• No authentication to the external service. If the external AI must reject anonymous callers, do it at the network layer (mesh, ingress, NetworkPolicy) — there is no per-request auth header or token added by the platform.

• No retries. A single POST /query_data per /api/genai/ask invocation. The external service must be reliable enough that a single attempt is acceptable, or operators must add retry / circuit-breaker logic upstream of the platform's caller.

• request_timeout is in minutes (not seconds, not milliseconds). The minimum effective value is 1 minute; setting 0 means immediate timeout (see above).

• Bean is built once at startup. WebClientConfiguration reads genai.url and genai.request_timeout to construct the genAiWebClient bean — changing those values requires a Platform restart for the new WebClient to pick them up.

• No UI today. Generated TypeScript clients exist but are not wired into any view; the feature is callable only via the API.

Where to next

• Configure ODD Platform → GenAI — the same three keys in the platform's full configuration reference, with environment-variable equivalents.

• Main Concepts → AI aspects — where GenAI sits among the other AI capabilities (data profiling, ML lineage).

• POST /api/genai/ask (operationId genAiQuestion, tag genai) — the request/response shape and verb live in opendatadiscovery/odd-platform-specification → openapi.yaml.