Configure the backend database#

What the database holds#

Starburst control plane stores the following in the backend database:

  • The clusters that you add, with their credentials, which control plane encrypts.

  • Your routing rules and tagging rules, and your cluster groups.

  • Query history. The persistence.retention.max-age property sets how long control plane keeps it.

  • The statement queue. See Statement queue table.

  • In managed mode, the catalogs, the files that the catalogs use, and the access control data.

Control plane encrypts the cluster credentials and the catalog files with a key that it derives from gateway.clusters.encryption.secret. Use the same value on every control plane instance and across restarts. If you change it, control plane cannot read the data that it encrypted before.

Catalog files#

In managed mode, you can upload the files that your catalogs use, such as a truststore, a keytab, or a name-mapping file. To use an uploaded file, reference it by name with ${catalog-file:<name>} when you create the catalog. Control plane stores each file in the database, in the tables catalog_files and catalog_file_revisions. A file can have up to 256 kB. To change the limit, set catalogs.files.max-size, which has a maximum of 512 kB. Control plane keeps the replaced contents of a file for catalogs.files.revision-retention, 72 hours by default, so that clusters that run an older catalog version can still load it.

Backend database properties#

Use the following properties to configure the backend database:

Property

Description

Default

persistence.jdbc.url

JDBC connection URL for the control plane backend database. The URL must include the protocol, hostname, port, and database name. The supported formats are:

  • PostgreSQL: jdbc:postgresql://hostname:5432/database_name

  • MySQL: jdbc:mysql://hostname:3306/database_name

  • Oracle with a service name: jdbc:oracle:thin:@hostname:1521/service_name

  • Oracle with a SID: jdbc:oracle:thin:@hostname:1521:sid

None (required)

persistence.jdbc.user

Username for authenticating to the database. This user must have permissions to create and modify tables for schema migrations.

None (required)

persistence.jdbc.password

Password for the database user. For production deployments, use secrets substitution such as ${ENV:VAR_NAME}.

None (required)

persistence.retention.max-age

Duration to retain query history records. Control plane automatically deletes older records. Use h for hours or d for days.

60d

persistence.jdbc.connection-pool.max-size

Maximum number of database connections in the connection pool. Increase this value based on your workload and database capacity.

10

Control plane also supports Kerberos and AWS IAM authentication for the backend database.

Kerberos authentication#

To use Kerberos authentication with an Oracle database, set the following properties in your config.properties file:

persistence.jdbc.url=jdbc:oracle:thin:@oracle.example.com:1521/starburstenterprise
persistence.jdbc.authentication-type=KERBEROS
persistence.jdbc.kerberos.client.principal=example@STARBURSTDATA.COM
persistence.jdbc.kerberos.client.keytab=<path-to-keytab-file>
persistence.jdbc.kerberos.config=/etc/kerberos/krb5.conf

AWS IAM authentication#

To use AWS IAM token-based authentication with a MySQL database, set the following properties in your config.properties file:

persistence.jdbc.url=jdbc:mysql://mysql.example.com:3306/starburstenterprise?sessionVariables=sql_mode=ANSI
persistence.jdbc.authentication-type=AWS_IAM
persistence.jdbc.connection-user=db_user
persistence.jdbc.aws.region-name=us-east-2
persistence.jdbc.aws.iam-role=${ENV:RDS_ROLE_ARN}
persistence.jdbc.aws.external-id=<external-id>
persistence.jdbc.aws.token-expiration-timeout=1s

Statement queue table#

By default, control plane stores the statements in the smart load balancing statement queue in the query_queue table of the backend database. The table holds the text of each statement and the headers that the client sent. Control plane keeps a finished statement for three days by default, which is the statement-queue.retention.period property. Control plane creates the table at startup, for MySQL, Oracle, and PostgreSQL. The database user needs permission to create and change tables.

For the properties that control the smart load balancing statement queue, such as its maximum size and the maximum wait time of a statement, see Turn smart load balancing on or off.