Starburst Stargate#

Starburst Stargate is a connector that lets you link a local catalog on your Starburst Enterprise cluster to a catalog on a remote Starburst Enterprise cluster with the same version.

Starburst Stargate can access a single catalog on the remote cluster for each catalog. Thus, if you have multiple remote catalogs, or you want to connect to multiple remote clusters, you must configure multiple catalogs using Starburst Stargate.

Starburst Stargate uses a JDBC connection to the remote cluster. This means you can connect to any data source and catalog supported by the remote cluster, but features of the connection are restricted to those supported by JDBC.

In addition, note that the connector only allows read access.

Requirements#

  • Starburst Enterprise clusters using the same version.

  • Network connectivity between the coordinators of the clusters using the configured HTTP or HTTPS protocol.

  • A valid Starburst Enterprise Elite license.

Configuration#

There is no special preparation required on the remote cluster. Make sure both local and remote clusters run the exact same SEP version.

On the local cluster’s nodes, create a catalog properties file in etc/catalog named, for example, remote_postgres.properties.

Configure the catalog to use Starburst Stargate by setting the connector.name property to stargate.

For the connection-url property, specify a JDBC string as follows:

  • The JDBC protocol is trino.

  • Specify the URL and port of the remote cluster’s coordinator.

  • Follow the URL with */catalog-name* to specify the catalog on the remote cluster to access.

For the connection-user and connection-password properties, specify a credentials valid on the remote SEP cluster. This connector supports usernames and passwords specified as plain text.

Alternatively you can configure password credential pass-through to use the credentials of the user on the local SEP cluster to connect to the remote cluster.

The remote catalog is available for access using the local catalog name, such as remote_postgres.

If the name of the catalog file on the local cluster is identical to the name of the catalog file in the remote cluster, you can use the same catalog.schema.table access in your queries. As a result, any query on that catalog you run in the remote cluster also works in the local cluster without modifications.

If you’re using an authenticated connection to the remote cluster, SSL must be configured. To enable SSL you need to set the ssl.enabled property to true. If you are not using globally trusted certificates you have to configure ssl.truststore.path, ssl.truststore.password, and ssl.truststore.type to set the truststore to use for the JDBC connection. Consult the TLS documentation for more information.

Example 1: A catalog remotehive.properties to access a remote Hive catalog:

connector.name=stargate
connection-url=jdbc:trino://remote-cluster.example.net:8080/hive
connection-user=some_sep_user
connection-password=some_password

You can access the remote catalog with the remotehive catalog in the local cluster.

Example 2: A catalog remotedb to access a remote PostgreSQL catalog called postgresdb accessed, with plain text password:

connector.name=stargate
connection-url=jdbc:trino://remote-cluster.example.net:8080/postgresdb
connection-user=some_sep_user
connection-password=some_password

You can access the remote catalog with the remotedb catalog in the local cluster.

Example 3: A catalog website to access a remote PostgreSQL catalog, which is called website there as well. The remote cluster requires a TLS connection:

connector.name=stargate
connection-url=jdbc:trino://remote-cluster.example.net:8443/website
connection-user=remotepost
ssl.enabled=true

Type mapping#

Starburst Stargate to Trino read type mapping#

The following read type mapping applies when data is read from existing tables on the remote data source.

Remote data source connector to Starburst Stargate local SEP type#

Remote data source type

Local SEP type

Notes

BOOLEAN

BOOLEAN

TINYINT

TINYINT

SMALLINT

SMALLINT

INTEGER

INTEGER

BIGINT

BIGINT

REAL

REAL

DOUBLE

DOUBLE

DECIMAL

DECIMAL

all precisions

CHAR

CHAR

all precisions

VARCHAR

VARCHAR

all precisions

VARBINARY

VARBINARY

DATE

DATE

TIME

TIME

all precisions

TIME WITH TIME ZONE

TIME WITH TIME ZONE

all precisions

TIMESTAMP

TIMESTAMP

all precisions

TIMESTAMP WITH TIME ZONE

TIMESTAMP WITH TIME ZONE

all precisions

JSON

JSON

No other type is supported.

General configuration properties#

The following properties can be used to configure how data types from the connected data source are mapped to Trino data types and how the metadata is cached in Trino.

Property name

Description

Default value

unsupported-type-handling

Configure how unsupported column data types are handled:

  • IGNORE, column is not accessible.

  • CONVERT_TO_VARCHAR, column is converted to unbounded VARCHAR.

The respective catalog session property is unsupported_type_handling.

IGNORE

jdbc-types-mapped-to-varchar

Allow forced mapping of comma separated lists of data types to convert to unbounded VARCHAR

case-insensitive-name-matching

Support case insensitive database and collection names

False

case-insensitive-name-matching.cache-ttl

1 minute

metadata.cache-ttl

Duration for which metadata, including table and column statistics, is cached

0 (disabled caching)

metadata.cache-missing

Cache the fact that metadata, including table and column statistics, is not available

False

SQL support#

The connector supports globally available and read operation statements to access data and metadata in JDBC.

Performance#

The connector includes a number of performance features, detailed in the following sections.

Table statistics#

Starburst Stargate supports table and column statistics if the remote catalog’s connector supports it, such as the Hive connector. In this case, this feature improves query processing performance based on the actual data in the data source.

For remote connectors that support this feature:

  • The statistics are collected by the remote data source and are retrieved by Starburst Stargate.

  • To collect statistics for a table, use the command required by the connector there. For example, for Hive catalogs you can use the following statement on the remote data source:

    ANALYZE table_schema.table_name;

  • Retrieving statistics for Hive views defined in the remote data source is supported. Statistics for other views, such as on PostgreSQL catalogs, are not supported.

Pushdown#

The connector supports pushdown for a number of operations:

Aggregate pushdown for the following functions:

Dynamic filtering#

Dynamic filtering is enabled by default. It causes the connector to wait for dynamic filtering to complete before starting a query.

You can disable dynamic filtering by setting the property dynamic-filtering.enabled in your catalog properties file to false.

Security#

The connector includes a number of security-related features, detailed in the following sections.

User impersonation#

The connector supports user impersonation.

You can enable it in the catalog file with the following configuration:

stargate.impersonation.enabled=true

Kerberos authentication#

The connector supports Kerberos-based authentication with the following configuration:

stargate.authentication.type=KERBEROS
kerberos.client.principal=example@example.com
kerberos.client.keytab=/etc/kerberos/example.keytab
kerberos.config=/etc/krb5.conf
kerberos.remote.service-name=server-service-name
kerberos.service-principal-pattern=${SERVICE}@${HOST}
kerberos.service-use-canonical-hostname=true

With this configuration the user example@example.com, defined in the principal property, is used to connect to the database, and the related Kerberos service ticket is located in the example.keytab file. kerberos.service-principal-pattern is optional and defaults to ${SERVICE}@${HOST}. kerberos.service-use-canonical-hostname is also optional, and defaults to true.

Password credential pass-through#

The connector supports password credential pass-through. It uses the user credentials on the local cluster to connect to the remote cluster.

To enable it, edit the catalog properties file to include the authentication type, and remove the connection-user and connection-password properties:

stargate.authentication.type=PASSWORD_PASS_THROUGH

For more information about configurations and limitations, see Password credential pass-through.

Starburst Cached Views#

The connector supports table scan redirection to improve performance and reduce load on the remote catalog.