eval_752
  • English

      • Provider Management Workflow

      Last updated: 2026-03-13 · Phase 2

      This page describes how provider management works in the current architecture: how to register a provider, store encrypted keys, configure models and aliases, and keep the resulting workflow auditable.

      Goals

      • Manage the provider lifecycle inside the application instead of through scattered environment variables.
      • Store API keys encrypted in PostgreSQL so rotation and disable flows stay inside the product.
      • Support aliases for comparable models across providers.
      • Keep room for future rate-limit and budget controls without lying about what is already shipped.

      Adding a Provider

      1. Open Providers at /providers.
      2. Fill the create form:
        • Name: operator-friendly label
        • Base URL: root endpoint, for example https://api.openai.com/v1
        • Type: provider family
        • Capabilities: system prompt, JSON mode, reasoning, image support, and so on
      3. Add one or more API keys in the encrypted key section.
      4. Save the provider.
      5. Run a smoke test with the exact model you intend to benchmark.

      Saved provider secrets are encrypted with ENCRYPTION_KEY and stored only as ciphertext.

      Managing Models and Aliases

      After saving a provider, use the model management surface to attach:

      • model_name: the exact provider-side identifier used during execution
      • display_label: the human-readable name shown in the UI
      • alias: the normalization key used to align similar models across providers

      Aliases are especially useful in Comparison, where the same model family might be reachable through different vendors or endpoints.

      Key Rotation and Disable Flow

      To rotate a key:

      1. Open the provider in edit mode.
      2. Add a replacement key.
      3. Save the provider.
      4. Disable or delete the old key once the new one has been validated.

      Keep ENCRYPTION_KEY safe during backup and restore. Provider rows live in PostgreSQL, but the decryption key does not.

      Rate Limits and Runtime Policy

      Provider metadata can already describe rate-related fields such as rpm, rps, or concurrency, but the main operator path today still centers on:

      • a provider-level smoke test
      • workspace-wide runtime defaults in Settings
      • real run execution through LiteLLM

      For the exact split between bootstrap env vars and UI-managed runtime controls, see Configuration.

      LiteLLM Integration Notes

      • FastAPI and Celery both call providers through the LiteLLM Python SDK.
      • eval_752 does not require a separate LiteLLM Proxy container for the main path.
      • Third-party gateways or self-hosted proxies can still be used by pointing Base URL at them.

      For the deeper integration plan, see LiteLLM SDK Blueprint.

      Common Questions

      Why not manage provider keys only through environment variables?

      Because operator workflows need creation, testing, rotation, and comparison inside the app. Pushing all secrets into environment variables makes those flows harder to audit and harder to hand off.

      How do I back up provider data?

      Provider metadata and encrypted keys are stored in PostgreSQL. Backups therefore need:

      • the database backup itself
      • the corresponding ENCRYPTION_KEY

      For infrastructure recovery, see Backup & Recovery.