Manage secrets#

Starburst Enterprise platform (SEP) manages configuration details in static properties files. This configuration needs to include values such as usernames, passwords, and other strings, that are often required to be kept secret. Only a few selected administrators or the provisioning system has access to the actual value.

The secrets support in SEP allows you to use environment variables as values for any configuration property. All properties files used by SEP, including config.properties and catalog properties files, are supported. When loading the properties, SEP replaces the reference to the environment variable with the value of the environment variable.

Environment variables are the most widely supported means of setting and retrieving values. Environment variables can be set in the scope of the task being performed, preventing external access. Most provisioning and configuration management systems include support for setting environment variables. This includes systems such as Ansible, often used for virtual machines, and Kubernetes for container usage. You can also manually set an environment variable on the command line.

export DB_PASSWORD=my-super-secret-pwd

To use this variable in the properties file, you reference it with the syntax ${ENV:VARIABLE}. For example, if you want to use the password in a catalog properties file like etc/catalog/db.properties, add the following line:

connection-password=${ENV:DB_PASSWORD}

With this setup in place, the secret is managed by the provisioning system or by the administrators handling the machines. No secret is stored in the SEP configuration files on the filesystem or wherever they are managed.

Configuration providers#

SEP allows you to use the following configuration providers:

You can also use one or more files as your configuration provider.

Note

Configuration providers only resolve references in catalog properties.

To enable additional configuration providers, use the configuration-provider.config-files configuration property in the additionalProperties section of config.properties. Insert a comma-separated list of the configuration provider property files. For example:

configuration-provider.config-files=etc/asm.properties

The following example shows how to create a configuration provider property file:

coordinator:
  etcFiles:
    properties:
      asm.properties: |
        configuration-provider.name=asm
      log.properties: |

Vault configuration provider#

To enable Vault by Hashicorp as your configuration provider, set the following configuration properties:

configuration-provider.name=vault
address=http://${ENV:VAULT_ADDRESS}:8200
token=${ENV:VAULT_TOKEN}

Reference values from Vault using the syntax ${VAULT:path:key}, where path is a secret path and key is a secret key.

The following configuration properties are available:

Configuration properties for the Vault configuration provider#

Property

Description

address

The address of the Vault server instance. This is a required property.

token

The Vault authentication token. This is a required property.

namespace

The global namespace of the Vault server instance.

engine-version

The KV Secrets Engine version of the Vault server instance.

ssl.client-key-pem-path

The path of a file containing an RSA private key, in unencrypted PEM format with UTF-8 encoding.

ssl.client-pem-path

The path of a file containing an X.509 certificate, in unencrypted PEM format with UTF-8 encoding.

ssl.keystore-path

A JKS keystore file, containing a client certificate registered with Vault’s TLS Certificate auth backend.

ssl.keystore-password

The password to the keystore.

ssl.truststore-path

A JKS truststore file, containing the Vault server’s X509 certificate.

ssl.verify

A Boolean indicating whether to verify the SSL certificate used by Vault with HTTPS connections.

open-timeout-seconds

The timeout value, in seconds, for establishing an HTTP(S) connection to the Vault server instance.

read-timeout-seconds

The timeout value, in seconds, for downloading data after establishing an HTTP(S) connection.

prefix-path-depth

The depth of the prefix file path.

max-retries

The number of retry attempts for API operations after a failure.

retry-interval

The wait time, in seconds, between retry attempts for API operations after a failure.

AWS Secret Manager configuration provider#

To enable AWS Secrets Manager as your configuration provider, set the following configuration property:

configuration-provider.name=asm

Reference values from AWS Secrets Manager using the syntax ${ASM:secret-id} or ${ASM:secret-id:key}.

If your secret contains a JSON object, reference a property by using a JSON pointer expression.

The following configuration properties are available:

Configuration properties for the ASM configuration provider#

Property

Description

aws-access-key

The AWS access key used for authenticating with AWS Secrets Manager.

aws-secret-key

The AWS secret key used for authenticating with AWS Secrets Manager.

endpoint

The endpoint URL for your AWS Secrets Manager service.

region

The AWS region for your AWS Secrets Manager service.

iam-role

The ARN of the IAM role used when connecting to AWS Secrets Manager.

role-session-name

The role session name used when connecting to AWS Secrets Manager.

external-id

External ID for the IAM role trust policy used when connecting to AWS Secrets Manager.

sts.endpoint

The endpoint URL for the AWS Security Token Service (STS) used for authenticating with AWS Secrets Manager.

sts.region

The AWS region for the AWS STS service.

max-connections

The maximum number of concurrent connections to AWS Secrets Manager.

http-proxy

The URL of the HTTP proxy server used for connecting to AWS Secrets Manager.

http-proxy.secure

Boolean indicating whether the HTTP proxy uses TLS.

File configuration provider#

To use one or more files as your configuration provider, set the following configuration properties:

configuration-provider.name=file
allowed-file-variable-dir=/<path>

You must specify a directory using the allowed-file-variable-dir property. This ensures that you only allow access to a specific directory on the host.

Note

Your files must be accessible on all nodes.

Reference values from a file using the syntax ${FILE:path}. path must begin with the directory you specify in allowed-file-variable-dir. For example:

connector.name=postgresql
connection-url=jdbc:postgres://pg-instance-hostname:5432
connection-user=${FILE:/<path>/pg_user}
connection-password=${FILE:/<path>/pg_password}