Connect a cluster to control plane#

Connect a SEP cluster to a Starburst control plane that runs in managed mode, which means that control-plane.enabled is true. The cluster then gets its catalogs from control plane, and Starburst access control manages the access to them. A cluster in standalone mode works as before.

Before you begin#

Make sure of the following:

  • Control plane runs in managed mode (control-plane.enabled=true). See Choose the mode.

  • The cluster meets the requirements for managed mode. See Requirements in managed mode.

  • You know the address of control plane, and the cluster can reach it over the network.

  • You know the name of the cluster. The name must match the node.environment property of the coordinator.

Connect the cluster#

  1. In the configuration of the coordinator and of each worker of the cluster, set deployment.mode to MANAGED, and set deployment.manager.uri to the address of control plane. Set both properties as literal values. They cannot use secret references.

  2. Set the shared secret of the cluster, internal-communication.shared-secret. You use the same value when you register the cluster.

  3. Configure the cluster to trust the certificate of control plane, because managed clusters connect to control plane over HTTPS. See Trust internal certificates.

  4. To let the cluster use the users and groups that control plane provisions, create the file etc/group-provider.properties on the coordinator, with group-provider.name=portal. Managed mode makes this group provider available, but it does not select it. See Use the synchronized groups on a managed cluster.

  5. Restart the coordinator and the workers.

  6. Register the cluster in control plane, with discovery turned on. See Add a cluster.

Managed mode turns on Starburst access control, and it sets the properties http-server.authentication.type and web-ui.authentication.type for you. It also sets discovery.uri if you do not. Do not set catalog.management. The cluster does not start if you set it.

Configure a Helm deployment#

The Helm chart sets its own discovery.uri, and managed mode needs discovery.uri to equal deployment.manager.uri. Otherwise, the coordinator does not start, and it logs the error Property 'discovery.uri' must be equal to deployment.manager.uri when deployment.mode=MANAGED. Set the three properties in the config.properties file of the coordinator and of the workers:

environment: cluster-a

internalTls: false

coordinator:
  etcFiles:
    properties:
      config.properties: |
        # Keep the other properties of config.properties here.
        deployment.mode=MANAGED
        deployment.manager.uri=https://control-plane.example.com
        discovery.uri=https://control-plane.example.com

worker:
  etcFiles:
    properties:
      config.properties: |
        # Keep the other properties of config.properties here.
        deployment.mode=MANAGED
        deployment.manager.uri=https://control-plane.example.com
        discovery.uri=https://control-plane.example.com

Do not set internalTls: true. With that setting, the SEP init container overrides the discovery.uri that you set.

Check the connection#

On the Clusters tab, check the status of the cluster. When control plane can reach it, and it runs in managed mode, the status is HEALTHY. If the status is UNSUPPORTED, read the reason, and see When a cluster is unsupported.

Example configuration#

To connect a SEP cluster to control plane in managed mode, set the following properties in the configuration of the cluster.

node.properties:

# node.properties
# The environment must match the cluster name in control plane.
node.environment=cluster-a

config.properties:

# config.properties
# Set both properties as literal values. They cannot use ${ENV:...}
# or other secret references.
deployment.mode=MANAGED
deployment.manager.uri=https://control-plane.example.com

# Managed mode sets discovery.uri to deployment.manager.uri if you do not set
# it. With Helm, you must set it. See the Helm example.

# Managed mode sets these properties for you. Do not set them yourself.
# The cluster does not start if a value differs:
#   http-server.authentication.type                      = managed
#   web-ui.authentication.type                           = managed
#   http-server.process-forwarded                        = ACCEPT
#   http-server.authentication.allow-insecure-over-http  = false

# Do not set catalog.management. Managed clusters get their catalogs
# from control plane. The cluster does not start if it is set.

# Do not turn these on. The cluster does not start if you do:
#   starburst.jobs.enabled
#   starburst.data-product-sharing.enabled
#   starburst.data-product.enabled
#   starburst.agent.enabled

# Set this only if the cluster reaches control plane at a different
# address than its public URL, for example through a Kubernetes service:
# http-server.authentication.managed.issuer=<gateway.clusters.authentication.issuer>

etc/group-provider.properties:

Create this file only if the cluster uses the users and groups that control plane provisions with LDAP synchronization or SCIM:

# etc/group-provider.properties
# deployment.mode=MANAGED makes this group provider available.
# It does not select it. This file selects it.
group-provider.name=portal

Control plane marks a cluster that does not run in managed mode as UNSUPPORTED. See Monitor cluster status.