Configure Starburst AI#

To use the Starburst AI connector and the supported models and functions, configure external large language models (LLMs) or embeddings with a supported provider.

Prerequisites#

To use and configure external AI models, consider the following:

You need a valid AI workflows license to configure external AI models.
The Models tab requires SEP’s built-in access control authorization.
You must have the AI models user interface entities feature privilege to access the Models tab.
Only a user with the sysadmin role can create, view, edit, or delete trusted connections, which store the endpoint and credentials for an AI model.
To connect a new model, you must have the CREATE permission and access to All AI models. You select an existing trusted connection when you connect the model. You do not enter connection credentials directly.
All credentials are configured on the trusted connection and stored using a secrets manager. Keys or secrets cannot be entered as plaintext and must be provided by secret reference.
For AWS Bedrock connections:
- The AWS account must have Bedrock service enabled and access to the desired models.
- Users must have the "Action": ["bedrock:InvokeModel"] AWS IAM permission to use models.
- To manage models and perform operations as an AWS administrator, you must have the "Action": ["bedrock:*"] permission.
- Some models require the use of an inference profile. AWS should indicate this requirement in the model card when requesting access. For language models, set the Model ID to the inference profile ID. Starburst recommends using the inference profile ID over the ARN. For embedding models, set the Model ID to the functional model ID, and specify the inference profile ID in the Advanced configuration options section. This is required because each model may have custom encoding requirements.
For Google Gemini Enterprise Agent Platform connections:
- You need a Google Cloud service account granted the Vertex AI User (roles/aiplatform.user) role. For the full list of roles, read the Vertex AI access control documentation.
- You need a service account key in JSON format, stored using a secrets manager and provided by secret reference.
- Starburst reads the Google Cloud project ID from the service account key.
- Embedding models are not supported.

Configuration#

To use the AI functionality, Starburst Enterprise now auto-creates a built-in catalog named starburst referencing the sep_ai connector if a valid license is present.

Configuration properties that were previously in starburst.properties must now be placed in config.properties, on both the coordinator and worker nodes:

ai.client.cache.refresh.enabled=true
ai.client.cache.refresh.interval=1s
ai.client.cache.ttl=1h
ai.client.models.storage=EXTERNAL

You can use an embedding or language model. The AI functions are available with the ai schema name. The functions use the starburst.ai catalog and schema prefix.

In the ai schema, there are two tables that list the available embedding and language models, embedding_models and language_models, respectively.

Catalog configuration properties#

The following table describes general catalog configuration properties for the sep_ai connector:

General catalog configuration properties#
Property name	Description	Default
`ai.client.cache.refresh.enabled`	Enables periodic refreshing of the model configuration information cache.	`false`
`ai.client.cache.refresh.interval`	Define the interval between automatic cache refreshes. The possible value can be `1s` to `10s`.	`1s`
`ai.client.cache.ttl`	The duration model configuration information is cached. The possible value can be `10m` to `3h`.	`1h`

Model Integration#

AI functions invoke an external LLM. Access to the LLM API must be configured in the Models tab. Performance, output, and cost of all AI function invocations are dependent on the LLM provider and the model used. Choose a model that aligns with your specific use case and performance requirements. Starburst supports AWS Bedrock, OpenAI compatible APIs, and Google Gemini Enterprise Agent Platform.

AWS Bedrock#

AWS Bedrock offers access to a suite of foundation models hosted on AWS. Integrating AWS Bedrock models requires configuring AWS credentials and specifying the models you wish to use for different AI functions. To use AWS Bedrock models, configure the necessary model connection details.

Supported models#

The following models are tested and supported by Starburst:

AWS Bedrock supported models#
Model name	Description
`amazon.titan-embed-text-v2:0`	The AWS Bedrock Titan Text Embedding v2 embedded model.
`cohere.embed-multilingual-v3`	The AWS Bedrock Cohere embed multilingual model.
`anthropic.claude-3-5-haiku-20241022-v1:0`	The AWS Bedrock provided Claude3.5 LLM.
`anthropic.claude-sonnet-4-5-20250929-v1:0`	The AWS Bedrock provided Claude Sonnet 4.5 LLM.
`meta.llama3-2-3b-instruct-v1:0`	The AWS Bedrock provided Llama 3.2 3B LLM.

Note

Starburst only supports compatible AWS Bedrock models that support the Converse API. This includes most AWS Bedrock models, but might not include custom models and newer OS models.

Note

For AIDA, Starburst recommends using Claude Sonnet 4.5 or newer. Models that are older or less capable than the current Claude Sonnet models are not recommended, because they might not consistently meet the reasoning and tool-use requirements of AIDA.

For more information about compatible AWS Bedrock’s providers and models, read the documentation.

OpenAI#

OpenAI offers access to a number of language and embedding models. To use OpenAI models, configure the necessary connection details.

Supported models#

The following models are tested and supported by Starburst:

OpenAI supported models#
Model name	Description
`text-embedding-3-small`	The OpenAI text embedding 3 model.
`text-embedding-3-large`	The OpenAI text embedding 3 model.
`gpt-4o-mini`	The OpenAI GPT-4o mini. A compact version of the standard GPT-4o LLM.

For more information about compatible OpenAI’s models and APIs, read the documentation.

Google Gemini Enterprise Agent Platform#

Google Gemini Enterprise Agent Platform (formerly Vertex AI) offers access to a suite of foundation models hosted on Google Cloud. To use Gemini models, configure a trusted connection with a Google Cloud service account key and region.

Supported models#

The following models are tested and supported by Starburst:

Google Gemini Enterprise Agent Platform supported models#
Model name	Description
`gemini-2.5-flash`	The Gemini 2.5 Flash language model.

Note

Google Gemini Enterprise Agent Platform supports language models only. Embedding models are not supported.

For more information about AI models available on Google Gemini Enterprise Agent Platform, read the Google Cloud documentation.

Compatible APIs#

Starburst Enterprise supports a number of AI models from Amazon Bedrock and OpenAI. Starburst lets you integrate with compatible APIs from these supported providers, whether these external models are deployed on-premises or in the cloud. This lets you connect to external LLMs for inferencing tasks and embedding models to generate vector embeddings for AI search.

Azure OpenAI#

Azure OpenAI lets you connect to OpenAI models hosted in the Microsoft Azure cloud through Azure AI Foundry (formerly Azure Studio). You can use Azure OpenAI for both language and embedding tasks by connecting through the OpenAI-compatible API.

To configure Azure OpenAI with Starburst, select OpenAI as the provider when you configure a trusted connection. Then, log in to your Azure OpenAI portal, copy the deployment URL and API key from Azure OpenAI foundry portal, and enter them in the connection information fields.

Gemini API#

Gemini provides large language and embedding models accessible through an OpenAI-compatible API. To use Gemini with Starburst, select OpenAI as the provider when you configure a trusted connection, get the API key from your cloud console in Google, and use the following endpoint:

`https://generativelanguage.googleapis.com/v1beta/openai/`

Model configuration#

To access the Models tab in the Starburst Enterprise web UI, built-in access control authorization is required and you must have the AI models UI feature privilege. After configuring the necessary privileges and catalog, navigate to AI > Models and click Connect external model.

The following sections provide more detail for connecting to an AI model.

Caution

Starburst recommends that you restrict AI model creation and management to the sysadmin role.

Before you connect a model, a sysadmin must configure a trusted connection for the provider you want to use. When you connect a model, you select an existing trusted connection instead of entering connection credentials.

The following sections provide more detail for connecting to an AI model.

Select a model integration and type#

Follow these steps to select one of the supported providers and connect to a model using a compatible API.

From the Model integration drop-down menu, select Amazon Bedrock, OpenAI & OpenAI API Compatible, or Google Gemini Enterprise Agent Platform.
From the Model type drop-down menu, select LLM or Embedding. Google Gemini Enterprise Agent Platform supports LLM only.
In the Model ID text field, enter the fully qualified name of the model from the provider. For example, if using the AWS Bedrock provided Llama 3.2 LLM, specify meta.llama3-2-3b-instruct-v1:0. The Amazon Model ID property value can be a foundation model name or an inference profile. When the model integration is Amazon Bedrock and model type is Embedding, the Model ID field becomes a drop-down menu.

For more information about AWS Bedrock’s providers and models, read the documentation.

Model alias and description#

The Model alias is the identifier used to reference the model in an AI function writing SQL and or showing the catalog and its nested schemas and tables in client applications. The model alias has the following requirements:

Max length of 64 characters.
Only lowercase letters, numbers, and special characters are allowed.
No spaces.

If you are configuring a model with Amazon Bedrock, read the prerequisites section.

The Description is a short optional paragraph that provides further details about the AI model.

Select a trusted connection#

In the Connection information section, select a Trusted connection. The drop-down menu groups connections by provider type. The model uses the endpoint and credentials from the selected connection. You cannot view or change them.

Only a sysadmin can manage trusted connections. See Trusted connections.

Custom request headers#

In the Custom request headers section, you can add custom HTTP headers to include with this model’s requests to the AI provider. These headers apply only to this model, and are separate from the additional headers configured on the trusted connection.

Click Add header.
Header name: Specify the header name or key.
Values: Specify one or more comma-separated values for the header.

Advanced configuration options#

In addition to the basic connection details for an external AI model, you can specify additional configuration options to fine-tune and control model behavior. The advanced configuration options available change depending on the Model type.

When the model type is set to LLM, the following options are available:
- Max Tokens: The maximum number of tokens expected to be returned by the model in its output. This generally includes words, punctuation, spaces, and special formatting.
- Temperature: Given a value from 0.1 and higher, this controls the perceived randomness or creativity of the model’s output. A lower value makes the output more deterministic.
- Top P: Controls the diversity of the output. An alternative to temperature sampling. The model selects tokens from the smallest group whose cumulative probability mass exceeds the specified threshold. For example, if the specified value is 0.1, only the combined tokens that make up the top ten percent of the probability mass are considered for the next word generation.
When the model type is set to Embedding, the following options are available:
- Dimensions: The number of numerical components in each embedding vector. This determines the size and complexity of the vector space.
- Inference profile: A profile of preset configuration settings. Read the AWS documentation. to learn more about inference profiles in AWS Bedrock.

General system prompts#

Use a system prompt to customize and improve the output of your language model by providing instructions or messages that define its behavior, role, tone, or context before any interaction takes place.

To add add a general system prompt, follow these steps:

Click Add general system prompts.
Click Add general system prompt.
In the text field, define a specific action or behavior for the LLM.
If you want to add additional system prompts, click Add general system prompt.
Click Save.

For more information about system prompts, see the general system prompts.

Prompt overrides#

Prompt overrides let you override general system prompts or use a different model to further refine the output from your LLM.

To add a prompt override, follow these steps:

Click Add prompt override.
Select the function you would like to add the override to, and specify the parameters for either User or System. You can add overrides to multiple functions at the same time.
Click Save.

For more information about prompt overrides, see the prompt overrides section.

Read more about Starburst AI functions to see a list of the supported functions and use cases.

Access control#

Starburst AI lets you connect to a number of supported models, invoke functions, and embeddings. Starburst Enterprise enables administrators to configure access and privileges to these models and functions for specific roles.

For more information about roles and privileges in Starburst Enterprise, see Built-in access control overview.