Secrets#

Starburst Enterprise platform (SEP) manages configuration details in static properties files. This configuration needs to include values such as usernames, passwords, and other strings, that are often required to be kept secret. Only a few selected administrators or the provisioning system have access to the actual value.

Secrets management in SEP allows you to reference values from security managers, files, or environment variables in any configuration property. All configurable SEP components, such as config.properties and catalog properties files, are supported. When loading the properties, SEP replaces the reference with the value from a specific secrets provider.

export DB_PASSWORD=my-super-secret-pwd

To use this variable in the properties file, you reference it with the syntax ${ENV:VARIABLE}. For example, if you want to use the password in a catalog properties file like etc/catalog/db.properties, add the following line:

connection-password=${ENV:DB_PASSWORD}

With this setup in place, the secret is managed by the provisioning system or by the administrators handling the machines. No secret is stored in the SEP configuration files on the filesystem or wherever they are managed.

SEP allows you to use the following configuration providers:

Vault
AWS Secret Manager
Azure Key Vault
Google Cloud Secret Manager
CyberArk Conjur
Keystore
File
Environment variables

Use a secrets.toml file to configure your secrets provider. By default, this file is located in the /etc directory. Specify a different location using the secretsConfig system property.

If the secretsConfig system property is not set, the environment varibles provider is enabled.

The secrets.toml file establishes the connection to a secrets provider. The file itself does not store your secrets. Instead, the file contains the configuration that SEP uses to retrieve secrets directly from the provider at runtime.

The following example secrets.toml file enables the Vault configuration provider:

secrets-plugins-dir="/usr/lib/starburst-cache-service/secrets-plugins"

[vault]
secrets-provider.name = "vault"
address = "http://vault:8200"
token = "your-token"

Set the secrets-plugins-dir property to the file path for the directory containing secrets provider plugins.

If you are using a standalone cache service deployment, you must copy the starburst-secrets-provider.zip file to your secrets-plugins-dir directory. For more information, see the cache service documentation.

To configure multiple instances of the same secrets provider, use unique section names:

[vault_prod]
secrets-provider.name = "vault"
address = "http://vault-prod:8200"
token = "prod-token"

[vault_dev]
secrets-provider.name = "vault"
address = "http://vault-dev:8200"
token = "dev-token"

Reference a secrets provider instance in your configuration files using the section name:

connection-user = ${vault_prod:secret/db:user}
connection-password = ${vault_dev:secret/db:password}

Vault#

To enable Vault by Hashicorp as your configuration provider, set the following properties in your secrets.toml file:

[vault]
secrets-provider.name = "vault"
address = "http://vault:8200"
token = "your-token"

Reference values from Vault using the syntax ${vault:path:key}, where:

vault is the case-insensitive section name in your secrets.toml file.
path is the secret path.
key is the secret key.

connector.name=postgresql
connection-url=jdbc:postgresql://postgresql:15432/test
connection-user=${vault:secret/postgres:user}
connection-password=${vault:secret/postgres:password}

The following configuration properties are available:

Configuration properties for the Vault configuration provider#
Property	Description
`address`	The address of the Vault server instance. This is a required property.
`token`	The Vault authentication token. This is a required property.
`namespace`	The global namespace of the Vault server instance.
`engine-version`	The KV Secrets Engine version of the Vault server instance.
`ssl.client-key-pem-path`	The path of a file containing an RSA private key, in unencrypted PEM format with UTF-8 encoding.
`ssl.client-pem-path`	The path of a file containing an X.509 certificate, in unencrypted PEM format with UTF-8 encoding.
`ssl.keystore-path`	A JKS keystore file, containing a client certificate registered with Vault’s TLS Certificate auth backend.
`ssl.keystore-password`	The password to the keystore.
`ssl.truststore-path`	A JKS truststore file, containing the Vault server’s X509 certificate.
`ssl.verify`	A Boolean indicating whether to verify the SSL certificate used by Vault with HTTPS connections.
`open-timeout-seconds`	The timeout value, in seconds, for establishing an HTTP(S) connection to the Vault server instance.
`read-timeout-seconds`	The timeout value, in seconds, for downloading data after establishing an HTTP(S) connection.
`prefix-path-depth`	The depth of the prefix file path.
`max-retries`	The number of retry attempts for API operations after a failure.
`retry-interval`	The wait time, in seconds, between retry attempts for API operations after a failure.

AWS Secret Manager#

To enable AWS Secrets Manager as your configuration provider, set the following properties in your secrets.toml file:

[asm]
secrets-provider.name = "asm"
region = "us-east-2"

Reference values from AWS Secrets Manager using the syntax ${asm:path} or ${asm:path:key}. For example:

connector.name=postgresql
connection-url=jdbc:postgresql://postgresql:15432/test
connection-user=test
connection-password=${asm:test:password}

If your secret contains a JSON object, reference a property by using a JSON pointer expression.

The following configuration properties are available:

Configuration properties for the ASM configuration provider#
Property	Description
`aws-access-key`	The AWS access key used for authenticating with AWS Secrets Manager.
`aws-secret-key`	The AWS secret key used for authenticating with AWS Secrets Manager.
`endpoint`	The endpoint URL for your AWS Secrets Manager service.
`region`	The AWS region for your AWS Secrets Manager service.
`iam-role`	The ARN of the IAM role used when connecting to AWS Secrets Manager.
`role-session-name`	The role session name used when connecting to AWS Secrets Manager.
`external-id`	External ID for the IAM role trust policy used when connecting to AWS Secrets Manager.
`sts.endpoint`	The endpoint URL for the AWS Security Token Service (STS) used for authenticating with AWS Secrets Manager.
`sts.region`	The AWS region for the AWS STS service.
`max-connections`	The maximum number of concurrent connections to AWS Secrets Manager.
`http-proxy`	The URL of the HTTP proxy server used for connecting to AWS Secrets Manager.
`http-proxy.secure`	Boolean indicating whether the HTTP proxy uses TLS.

Azure Key Vault#

To enable Azure Key Vault as your configuration provider, set the following properties in your secrets.toml file:

[azure-key-vault]
secrets-provider.name = "azure-key-vault"
secrets.azure.key-vault-name = "secrets"

Google Cloud Secret Manager#

To enable Google Cloud Secret Manager as your configuration provider, set the following properties in your secrets.toml file:

[gcp-secret-manager]
secrets-provider.name = "gcp-secret-manager"
gcp.credentials-key = "your-credentials-key"

CyberArk Conjur#

To enable CyberArk Conjur as your configuration provider, set the following properties in your secrets.toml file:

[conjur]
secrets-provider.name="conjur"
conjur.appliance-endpoint="https://conjur-proxy-nginx"
conjur.account="sep"
conjur.authentication-login="user@test"
conjur.api-key="your-credentials-key"

Configuration properties for the CyberArk Conjur configuration provider#
Property	Description
`conjur.appliance-endpoint`	The endpoint URL for your CyberArk Conjur server instance. It can be an HTTP or HTTPS endpoint.
`conjur.account`	The CyberArk account that contains your secrets.
`conjur.authentication-login`	The username to authenticate to CyberArk.
`conjur.api-key`	The API key or password to authenticate to CyberArk.
`conjur.tls.enabled`	Set to `true` to enable TLS.
`conjur.ssl.keystore.path`	The file path to your SSL keystore.
`conjur.ssl.keystore.password`	The password for your SSL keystore.
`conjur.ssl.truststore.path`	The file path to your SSL truststore.
`conjur.ssl.truststore.password`	The password for your SSL truststore.
`conjur.token.authentication.enabled`	Set to `true` to enable token-based authentication.
`conjur.token-file-path`	The file path for your token, if you enable token-based authentication.
`conjur.token.refresh-period`	The duration between token refreshes. This property is optional. Defaults to `1m`.

Keystore#

To enable a Keystore as your configuration provider, set the following properties in your secrets.toml file:

[keystore]
secrets-provider.name = "keystore"
keystore-file-path = "/docker/presto-product-tests/conf/presto/etc/credential.jckes"
keystore-type = "pkcs12"
keystore-password = "password"

File#

To use one or more files as your configuration provider, set the following properties in your secrets.toml file:

secrets-provider.name="file"
allowed-file-variable-dir="/docker/presto-product-tests/conf/presto/etc"

You must specify a directory using the allowed-file-variable-dir property. This ensures that you only allow access to a specific directory on the host.

Note

Your files must be accessible on all nodes.

Reference values from a file using the syntax ${FILE:path}. path must begin with the directory you specify in allowed-file-variable-dir. For example:

connector.name=postgresql
connection-url=jdbc:postgres://pg-instance-hostname:5432
connection-user=${FILE:/<path>/pg_user}
connection-password=${FILE:/<path>/pg_password}

Environment variables#

Environment variables are a widely supported means of setting and retrieving values within services. SEP supports the use of environment variables to manage secrets without needing any external services.

To use environment variables as your configuration provider, set the following properties in your secrets.toml file:

secrets-provider.name="env"

Note

Your environment variables must be accessible on all nodes.

Reference values of environment variables using the syntax ${ENV:name}. For example:

connector.name=postgresql
connection-url=jdbc:postgres://${ENV:PGHOST}:${ENV:PGPORT}
connection-user=${ENV:PGUSER}
connection-password=${ENV:PGPASSWORD}

Note

This provider is enabled by default, if the secretsConfig system property is not set (no secrets.toml file is used). If other providers are configured, it must also be enabled, if any configuration properties include references to environment variables.