Skip to content

llm

Repository: github.com/datarobot-community/af-component-llm

Adds LLM integration to your recipe via DataRobot's LLM Gateway or a directly deployed model. This is the building block for any recipe that needs to call a language model — it's also a prerequisite for the agent component.

The component provisions the Pulumi infrastructure to create an LLM Deployment in DataRobot and exposes configuration for switching between LLM Gateway, an external model, or an already-deployed model — all via environment variables.

Prerequisites

  • Python 3.11+
  • uv installed
  • dr installed
  • The base component already applied

Apply

dr component add https://github.com/datarobot-community/af-component-llm .

Or with copier directly:

uvx copier copy https://github.com/datarobot-community/af-component-llm .

Component dependencies

Component Required
base Yes

Configuration

After applying, run the environment setup wizard:

dr dotenv setup

When prompted for INFRA_ENABLE_LLM, choose your integration type:

Option When to use
gateway_direct.py LLM Gateway — simplest, most production-ready.
blueprint_with_llm_gateway.py LLM Gateway + Vector Database support.
blueprint_with_external_llm.py External model (OpenAI, Anthropic, etc.) via LLM Gateway.
deployed_llm.py An already-deployed DataRobot LLM Deployment.
registered_model.py An existing registered model with an LLM Blueprint.

Deploy

dr task deploy

Retrieve deployment information at any time:

dr task infra:info

Update

dr component update .datarobot/answers/llm-LLM_NAME.yml

Or with copier directly:

uvx copier update -a .datarobot/answers/llm-LLM_NAME.yml -A

What it adds

  • infra/infra/llm.py—Pulumi resources for the LLM deployment and playground.
  • infra/configurations/llm/—LLM configuration blueprints.
  • .datarobot/answers/llm-NAME.yml—recorded answers.
  • .datarobot/cli/llm.yml—CLI configuration for dr dotenv setup.

Troubleshooting

"Model not found in LLM Gateway catalog"

Check that LLM_NAME_DEFAULT_MODEL is set to a valid model ID (for example datarobot/azure/gpt-5-mini-2025-08-07) and that the model is active. Call verify_llm_gateway_model_availability() from the library module to list available models.

"Feature flags required but not enabled"

Some configuration strategies require DataRobot platform feature flags (MLOPS, TEXT_GENERATION, and others). Contact DataRobot support to have the required flags enabled on your account.

"Credential validation failed" for external providers

Verify that the environment variables for your chosen provider are set correctly. The required variable names differ per provider and are listed in the generated library module.

Copier update conflicts

If uvx copier update reports conflicts, review the diff carefully. Re-run the update with -A to skip all conflict prompts and accept the latest template version.