Architecture of a model-based system

The fixed components

An input layer that validates and cleans; a retrieval layer that brings context with permissions; a model-call layer with retries, timeout and a fallback vendor; an output-validation layer against a schema; and a logging and measurement layer.

Each exists in ordinary systems too. What changes is that the output is non-deterministic, so validation and measurement matter many times more.

Synchronous vs asynchronous

If the answer takes more than a few seconds, move to a background process with a task identifier and status updates. Holding a request open for a minute is a recipe for timeouts and a bad user experience.

Boundaries you must not cross

Model output does not go directly into execution — not a query, not a command, not an external call — without validation and an allowed-actions list.