Knowledge Base Academy Community My Tickets

Home Knowledge Base Academy Community My Tickets

Fabric 8.5

Knowledge Base Custom LLM Providers

Custom LLM Providers

Besides the built-in provider entries (Anthropic, OpenAI, Google, Hugging Face, Ollama and others described in AI Features Settings), Studio AI can talk to any endpoint that implements the OpenAI or Anthropic API. This is what a custom provider is: an entry that names the endpoint URL and the model, instead of relying on the vendor's default endpoint.

Custom providers are used to connect Studio AI to:

An LLM gateway or proxy operated by your organization, which enforces routing, quota or logging policies.
A self-hosted model running in your own premises or cloud.
Fabric itself, so that developers consume LLMs through a Fabric AI LLM interface and need no API key of their own. See Fabric as a Provider.

Custom providers are defined in the Studio Settings UI, under AI Features, in the Open AI Custom Models or Anthropic Custom sub-sections. Each is a JSON array of model entries.

Custom OpenAI Providers

Setting key: ai-features.openAiCustom.customOpenAiModels.

"ai-features.openAiCustom.customOpenAiModels": [
  {
    "id": "my-openai-compatible-endpoint",
    "model": "gpt-4.1",
    "url": "https://llm-gateway.example.com/v1",
    "apiKey": "",
    "enableStreaming": true,
    "useResponseApi": false,
    "supportsStructuredOutput": false,
    "developerMessageSettings": "system"
  }
]

Field	Description
`id`	Identifier of the entry. This is the name the model is offered under in the chat model selector and in model aliases.
`model`	The model name sent to the endpoint.
`url`	Base URL of the OpenAI-compatible endpoint.
`apiKey`	API key sent to the endpoint. Leave empty when the endpoint does not require one.
`enableStreaming`	Whether responses are streamed back to the chat as they are generated.
`useResponseApi`	Whether to call the OpenAI Responses API instead of chat completions. Set to `false` for endpoints that implement chat completions only.
`supportsStructuredOutput`	Whether the endpoint supports structured (schema-constrained) output.
`developerMessageSettings`	How the system prompt is delivered - for example `system` to send it as a system message.
`headers`	Optional additional HTTP headers. See Custom Headers.

Custom Anthropic Providers

Setting key: ai-features.anthropicCustom.customAnthropicModels. The entry names the model, the endpoint URL and an identifier, in the same way as a custom OpenAI entry.

"ai-features.anthropicCustom.customAnthropicModels": [
  {
    "id": "gateway-test-claude",
    "model": "claude-sonnet-5",
    "url": "http://127.0.0.1:3999",
    "apiKey": true,
    "headers": {
      "Wm-Llmgw-User-Type": "",
      "Wm-Llmgw-User-Name": ""
    }
  }
]

Custom Headers

Starting from V8.5.1, both custom OpenAI and custom Anthropic entries accept a headers object. Every key/value pair in it is added to each request sent to that endpoint. This is typically required by an LLM gateway that identifies the calling user or team through its own headers.

A header value can be given in three forms:

Form	Description
A literal string	The value is sent as written.
`${env:<envName>}`	The value is read from the named environment variable of the Fabric Dev server.
`${file:<pathToFile>}`	The value is read from the contents of the named file.

"headers": {
  "X-Team": "data-platform",
  "X-Gateway-Token": "${env:LLM_GATEWAY_TOKEN}",
  "X-Client-Cert": "${file:/etc/k2view/llm-gateway/client.token}"
}

The ${env:...} and ${file:...} forms keep secrets out of the Studio configuration, so that the configuration can be shared or version-controlled while the secret stays on the server.

Fabric as a Provider

Starting from V8.5.1, Fabric exposes an OpenAI-compatible endpoint of its own, backed by the project's AI LLM interface. Pointing Studio AI at it means that:

Developers enter no API key at all - the provider credentials are defined once, on the Fabric interface.
The organization controls which models are reachable, and which roles may use them.
The model can be swapped by editing the Fabric interface, without changing any Studio setting.

Because the Fabric endpoint is OpenAI-compatible, it is configured as a custom OpenAI provider: set url to the Fabric /api/v1 base URL and leave apiKey empty.

"ai-features.openAiCustom.customOpenAiModels": [
  {
    "id": "fabric-openai-compatible-llm-endpoint",
    "model": "default",
    "url": "http://localhost:3213/api/v1",
    "apiKey": "",
    "enableStreaming": true,
    "useResponseApi": false,
    "supportsStructuredOutput": false,
    "developerMessageSettings": "system"
  }
]

Here model selects which LLM Fabric uses: default for the project's default LLM engine, or the name or tag of a specific AI LLM interface. The Fabric role of the calling user must be granted the LLM_INVOKE permission.

For the endpoint reference, the authorization details and the interface settings themselves, see Fabric as an LLM Provider.

Using a Custom Provider

Once an entry is saved, its id appears wherever a model can be chosen:

In the chat toolbar's model dropdown, for a single session. See Using the AI Chat.
In the Agents tab of AI Configuration, as an agent's default model.
In a Model Alias priority list, which is the recommended way to move all agents onto a provider at once. See AI Configuration: Managing Agents and Settings.