Cluster requirements#
Before you add a cluster to Starburst control plane, make sure that it meets the requirements in this topic. The requirements depend on the mode that you use. For the modes, see Deployment modes.
Requirements for every cluster#
License and version. The cluster has an Elite SEP license and runs SEP version 480-e or later. Otherwise, control plane marks the cluster as
UNSUPPORTED. Use the same SEP version for control plane and for its clusters. Control plane does not support a mix of versions.Name. The
node.environmentproperty of the coordinator matches the cluster name that you use in control plane. Otherwise, control plane marks the cluster asUNSUPPORTED. Each cluster needs its ownnode.environmentvalue, because control plane requires the name of every cluster to be unique.Cluster group. The cluster belongs to exactly one cluster group.
Requirements in managed mode#
In managed mode, the cluster gets its catalogs from control plane, and control
plane authenticates its users. Control plane routes only to clusters that run in
managed mode. A cluster in standalone mode becomes UNSUPPORTED. Make sure of
the following:
You set
deployment.modeanddeployment.manager.uriin the configuration of the cluster. See Connect a cluster to control plane.You register the cluster in control plane with Use discovery turned on. Control plane does not accept a fixed URL for a cluster in managed mode. See Discovery in managed mode.
The cluster can reach control plane over the network. The coordinator gets the catalog definitions from control plane for each transaction, and queries fail when it cannot.
The cluster has no catalogs of its own. In managed mode, all catalogs live in control plane.
To use the users and groups that control plane provisions, the coordinator has the file
etc/group-provider.propertieswithgroup-provider.name=portal. See Connect a cluster to control plane.
Requirements in standalone mode#
In standalone mode, each cluster keeps its own catalogs and access control. Make sure of the following:
The authentication configuration of the cluster matches control plane. This includes the authentication type, the user mapping rules, and the authentication signing secret. The secret is
web-ui.shared-secretfor password or insecure authentication, andhttp-server.authentication.oauth2.state-keyfor OAuth2. See Configure cluster authentication and Shared secrets.You have the
internal-communication.shared-secretvalue of the cluster. You enter it in the Shared secret field when you add the cluster.All clusters in the same cluster group have the same catalog, access control, and authentication configurations.
Example configuration#
The following files show the settings of a cluster in standalone mode. They apply to the coordinators and to the workers of the cluster.
node.properties (coordinators and workers):
# node.properties
# The environment is unique for each cluster.
# It must equal the cluster name in control plane.
node.environment=cluster-a
config.properties (coordinators and workers):
# config.properties
# Do not set deployment.mode. A cluster without it runs in standalone mode.
# Coordinator high availability: announce to control plane.
# Use the same value on coordinators and workers.
discovery.uri=https://control-plane.internal.example.com
# Enter the same value in the Shared secret field
# when you add the cluster to control plane.
internal-communication.shared-secret=${ENV:INTERNAL_SHARED_SECRET}
# Authentication must match control plane exactly.
http-server.authentication.type=oauth2
web-ui.authentication.type=oauth2
http-server.authentication.oauth2.issuer=https://idp.example.com/
# Use the same value as on control plane.
http-server.authentication.oauth2.state-key=${ENV:OAUTH2_STATE_KEY}
# Catalogs and access control are identical on every cluster.
# static is the default.
catalog.management=static
Set discovery.uri only if the cluster uses coordinator high availability, and
register the cluster in control plane with discovery. See
Configure coordinator high availability. In a Helm installation
with internalTls: true, the init container overrides a discovery.uri that
you set in config.properties. See the limitations in How coordinator high
availability works. For the secrets, see
Shared secrets.
If control plane runs in managed mode, it marks a cluster that does not run in
managed mode as UNSUPPORTED. So this configuration applies only when control
plane runs in standalone mode. See Monitor cluster status.
Requirements for the clusters in a cluster group#
The clusters in a cluster group must be interchangeable, so that a query returns the same result on any of them. Make sure that all clusters in a group have:
the same execution modes, such as FTE or Warp Speed;
the same database objects, such as tables and materialized views. In standalone mode, the clusters also need the same data products and the same built-in access control, because each cluster keeps its own;
the same SEP version.
The clusters in a group can have different sizes and different numbers of worker nodes. Set the query limit of each cluster to the number of queries that it can run at once. See Set query limits.
Cluster groups#
Every cluster belongs to exactly one cluster group, and exactly one cluster group is the default. If you create a cluster and do not name a group, control plane puts the cluster in the default group. If you update a cluster and do not name a group, it stays in its group.
In managed mode, you need the CREATE_CLUSTER privilege to create a cluster. To
choose a cluster group for a cluster, you also need one of the privileges
MANAGE_SECURITY, CREATE_CLUSTER, or MANAGE_QUERY_ROUTING_RULES. With one
of them, you can choose any group. Owning a group does not let you use it for a
cluster, and the default group is not an exception. If you own a cluster, you
can update it without any of these privileges, as long as it stays in its group.
If you create or update clusters through the API or with automation, the role
that the automation uses needs one of those privileges to name a group. Name the
group by its ID. The older routingGroup field, which carries the group name,
still works but is deprecated.