Operating SEP in a Kubernetes cluster #

Congratulations! Your cluster is up and running. Now it is time to test it out by connecting your clients like the CLI or a BI application using the JDBC driver.

With that usage you gain insights on the correct configuration, data sources and sizing of the cluster for the desired usage and performance. This in turn leads you to further work such as updating configuration, scaling your cluster, or even upgrading to a newer release.

This document covers these and other aspects associated to running your SEP cluster, or even clusters.

Running SEP #

Once you have successfully installed SEP, you can confirm that the coordinator pod is running with tools such as kubectl or Octant.

You can also check the log in the server pod. A successful start of the coordinator, and also each worker, shows with a log entry similar to the following snippet:

  INFO	main	io.trino.server.Server	======== SERVER STARTED ========

With these checks performed, you can access the Starburst Enterprise web UI on the coordinator. Determine the FQDN and port of the coordinator as configured in the expose section of the Helm chart.

A default deployment exposes the UI via HTTP and does not require a password. Production deployments should use HTTPS and have authentication configured.

The main page of the web UI displays the number of connected workers. Confirm that the number is identical to the number specified by the worker.replicas value of the worker section of the Helm chart. The default number of workers is two.

Connecting tools #

With the knowledge that you can access the Starburst Enterprise web UI and therefore the coordinator, and that some workers are running, you are ready to connect client tools for running queries.

Use the protocol, FQDN, and port with the --server option for starting the CLI to access SEP:

$ trino --server https://trino.example.com:9999

The same URL can be used to configure the JDBC driver or ODBC driver connection and connection from any other client.

Updates #

From time to time you need to make incremental changes to your configuration or move to a new release. To do so:

• Create or update the correct YAML files with the new or changed configuration properties.
• Run the helm upgrade command with the updated version of the chart.
• Verify that the desired changes are applied successfully. We suggest making changes to one area of the application at a time.
• Repeat the process with the next desired change.

Typically Helm applies the relevant changes in a granular fashion. For example, if you change the worker configuration, no coordinator changes are performed and the coordinator continues to run while workers are updated.

If you need to ensure that pods are recreated completely, you can use the --recreate-pods option for your helm command. This essentially restarts all workers as well as the coordinator and therefore restarts the cluster. This includes a downtime for users.

Scaling #

You can scale your SEP cluster manually by adding more workers. This is done by updating the worker replicas and updating your deployment. Sufficient resources available in the cluster is a requirement.

Alternatively, you can enable autoscaling in your worker configuration and updating the deployment. Sufficient resources available in the cluster is a requirement.

You can also increase the memory resources allocated to the workers and updating your deployment, if sufficiently sized pods are available and you recreate pods as part of the update.