Secrets#

Starburst Enterprise platform (SEP) supports two secrets management methods for storing and retrieving sensitive data:

External secrets management#

With external secrets management, SEP manages configuration details in static properties files. This configuration needs to include values such as usernames, passwords, and other strings, that are often required to be kept secret. Only a few selected administrators or the provisioning system has access to the actual value.

The secrets support in SEP allows you to use environment variables as values for any configuration property. All properties files used by SEP, including config.properties and catalog properties files, are supported. When loading the properties, SEP replaces the reference to the environment variable with the value of the environment variable.

Environment variables are the most widely supported means of setting and retrieving values. Environment variables can be set in the scope of the task being performed, preventing external access. Most provisioning and configuration management systems include support for setting environment variables. This includes systems such as Ansible, often used for virtual machines, and Kubernetes for container usage. You can also manually set an environment variable on the command line.

export DB_PASSWORD=my-super-secret-pwd

To use this variable in the properties file, you reference it with the syntax ${ENV:VARIABLE}. For example, if you want to use the password in a catalog properties file like etc/catalog/db.properties, add the following line:

connection-password=${ENV:DB_PASSWORD}

With this setup in place, the secret is managed by the provisioning system or by the administrators handling the machines. No secret is stored in the SEP configuration files on the filesystem or wherever they are managed.

Internal secrets management#

With internal secrets management, SEP dynamically retrieves secrets from secure configuration provider services at runtime. Because secrets are not stored in static properties files, internal secrets management provides a more secure way to retrieve sensitive data. Internal secrets management supports more complex scenarios such as automatic secret rotation, access controls, and audit logging.

SEP allows you to use the following configuration providers:

Use a secrets.toml file to enable and configure your secrets provider. By default, this file is located in the /etc directory. Specify a different location using the secretsConfig system property.

The following example secrets.toml file enables the Vault configuration provider:

secrets-plugins-dir="/usr/lib/starburst-cache-service/secrets-plugins"

[vault]
secrets-provider.name = "vault"
address = "http://vault:8200"
token = "your-token"

Set the secrets-plugins-dir property to the file path for the directory containing secrets provider plugins. If you are using a standalone cache service deployment, see the cache service documentation for more information.

To configure multiple instances of the same secrets provider, use unique section names:

[vault_prod]
secrets-provider.name = "vault"
address = "http://vault-prod:8200"
token = "prod-token"

[vault_dev]
secrets-provider.name = "vault"
address = "http://vault-dev:8200"
token = "dev-token"

Reference a secrets provider instance in your configuration files using the section name:

connection-user = ${vault_prod:secret/db:user}
connection-password = ${vault_dev:secret/db:password}

Vault#

To enable Vault by Hashicorp as your configuration provider, set the following properties in your secrets.toml file:

[vault]
secrets-provider.name = "vault"
address = "http://vault:8200"
token = "your-token"

Reference values from Vault using the syntax ${VAULT:path:key}, where path is a secret path and key is a secret key. For example:

connector.name=postgresql
connection-url=jdbc:postgresql://postgresql:15432/test
connection-user=${vault_one:secret/postgres:user}
connection-password=${vault_two:secret/postgres:password}

The following configuration properties are available:

Configuration properties for the Vault configuration provider#

Property

Description

address

The address of the Vault server instance. This is a required property.

token

The Vault authentication token. This is a required property.

namespace

The global namespace of the Vault server instance.

engine-version

The KV Secrets Engine version of the Vault server instance.

ssl.client-key-pem-path

The path of a file containing an RSA private key, in unencrypted PEM format with UTF-8 encoding.

ssl.client-pem-path

The path of a file containing an X.509 certificate, in unencrypted PEM format with UTF-8 encoding.

ssl.keystore-path

A JKS keystore file, containing a client certificate registered with Vault’s TLS Certificate auth backend.

ssl.keystore-password

The password to the keystore.

ssl.truststore-path

A JKS truststore file, containing the Vault server’s X509 certificate.

ssl.verify

A Boolean indicating whether to verify the SSL certificate used by Vault with HTTPS connections.

open-timeout-seconds

The timeout value, in seconds, for establishing an HTTP(S) connection to the Vault server instance.

read-timeout-seconds

The timeout value, in seconds, for downloading data after establishing an HTTP(S) connection.

prefix-path-depth

The depth of the prefix file path.

max-retries

The number of retry attempts for API operations after a failure.

retry-interval

The wait time, in seconds, between retry attempts for API operations after a failure.

AWS Secret Manager#

To enable AWS Secrets Manager as your configuration provider, set the following properties in your secrets.toml file:

[asm]
secrets-provider.name = "asm"
region = "us-east-2"

Reference values from AWS Secrets Manager using the syntax ${ASM:path} or ${ASM:path:key}. For example:

connector.name=postgresql
connection-url=jdbc:postgresql://postgresql:15432/test
connection-user=test
connection-password=${asm:test:password}

If your secret contains a JSON object, reference a property by using a JSON pointer expression.

The following configuration properties are available:

Configuration properties for the ASM configuration provider#

Property

Description

aws-access-key

The AWS access key used for authenticating with AWS Secrets Manager.

aws-secret-key

The AWS secret key used for authenticating with AWS Secrets Manager.

endpoint

The endpoint URL for your AWS Secrets Manager service.

region

The AWS region for your AWS Secrets Manager service.

iam-role

The ARN of the IAM role used when connecting to AWS Secrets Manager.

role-session-name

The role session name used when connecting to AWS Secrets Manager.

external-id

External ID for the IAM role trust policy used when connecting to AWS Secrets Manager.

sts.endpoint

The endpoint URL for the AWS Security Token Service (STS) used for authenticating with AWS Secrets Manager.

sts.region

The AWS region for the AWS STS service.

max-connections

The maximum number of concurrent connections to AWS Secrets Manager.

http-proxy

The URL of the HTTP proxy server used for connecting to AWS Secrets Manager.

http-proxy.secure

Boolean indicating whether the HTTP proxy uses TLS.

Azure Key Vault#

To enable Azure Key Vault as your configuration provider, set the following properties in your secrets.toml file:

[azure-key-vault]
secrets-provider.name = "azure-key-vault"
secrets.azure.key-vault-name = "secrets"

Google Cloud Secret Manager#

To enable Google Cloud Secret Manager as your configuration provider, set the following properties in your secrets.toml file:

[gcp-secret-manager]
secrets-provider.name = "gcp-secret-manager"
gcp.credentials-key = "your-credentials-key"

Keystore#

To enable a Keystore as your configuration provider, set the following properties in your secrets.toml file:

[keystore]
secrets-provider.name = "keystore"
keystore-file-path = "/docker/presto-product-tests/conf/presto/etc/credential.jckes"
keystore-type = "pkcs12"
keystore-password = "password"

File#

To use one or more files as your configuration provider, set the following properties in your secrets.toml file:

secrets-provider.name="file"
allowed-file-variable-dir="/docker/presto-product-tests/conf/presto/etc"

You must specify a directory using the allowed-file-variable-dir property. This ensures that you only allow access to a specific directory on the host.

Note

Your files must be accessible on all nodes.

Reference values from a file using the syntax ${FILE:path}. path must begin with the directory you specify in allowed-file-variable-dir. For example:

connector.name=postgresql
connection-url=jdbc:postgres://pg-instance-hostname:5432
connection-user=${FILE:/<path>/pg_user}
connection-password=${FILE:/<path>/pg_password}