Release 443-e LTS (31 May 2024)#

Starburst Enterprise platform (SEP) 443-e LTS is the follow up release to the 438-e STS release and the 435-e LTS release.

This release is a promotion of the original 443-e STS release in May 2024 into a long term support (LTS) release.

It contains all improvements from Starburst Enterprise releases since the 435-e LTS release:

The 443-e release includes all improvements from the following Trino releases:

This release is a long term support (LTS) release.

Highlights since 435-e#

Breaking changes#

  • As of SEP 438-e, Starburst Warp Speed uses a new file system caching mechanism. To use Starburst Warp Speed, you must add the --enable-preview flag to your jvm.config configuration. Additionally, the following catalog configuration properties have been removed:

    • warp-speed.workerdb.db.path

    • warp-speed.file-system-reserve-percentage

    • warp-speed.call-home.enable

    You must remove these configuration properties from your cluster configuration or the cluster fails to start.

  • The Starburst Elasticsearch connector no longer supports Elasticsearch version 6.x or OpenSearch 1.x. Update Elasticsearch to version 7.x or 8.x to continue using the connector. To connect to OpenSearch, use the OpenSearch connector.

  • SEP now requires JDK 21 to run. See the Java runtime environment requirements for more information.

  • The hive.cache.enabled configuration property has been deprecated in favor of fs.cache.enabled. Remove and replace the deprecated property from all Hive catalog configurations.

  • The legacy value for hive.security has been removed, the new default value is allow-all. See Authorization for more information.

  • The following Hive authorization configuration properties have been removed. These properties must be removed from all configurations or the cluster does not start:

    • hive.allow-drop-table

    • hive.allow-rename-table

    • hive.allow-add-column

    • hive.allow-drop-column

    • hive.allow-rename-column

    • hive.allow-comment-table

    • hive.allow-comment-column

  • Removed the service-database.connection-pool.enabled configuration property from the cache service. You must remove this configuration property or the cluster fails to start.

  • The cache service now supports the same authentication methods as the SEP backend service database. As part of this change, the following cache service configuration properties have changed and must be updated in your configuration:

    • service-database.user to insights.jdbc.user.

    • service-database.password to insights.jdbc.password.

    • service-database.jdbc-url to insights.jdbc.url.

    • service-database.connection-pool.max-size to insights.jdbc.connection-pool.max-size.

    • service-database.connection-pool.idle-timeout to insights.jdbc.connection-pool.idle-timeout.

    Read more about the requirements for the cache service storage in the documentation.

  • If you are using MySQL as the externally-managed database for the cache service, you must append the parameter sessionVariables=sql_mode=ANSI to the connection string you use in the insights.jdbc.url property or the cluster fails to start.

  • This release removes the snowflake_distributed connector. You must remove or migrate existing Snowflake catalogs that use the distributed connector to the parallel connector or the cluster fails to start.

  • If you are using Oracle as the externally-managed database for the Backend service and upgrading to this version from SEP version 435-e or earlier, you must take additional steps prior to upgrading SEP. Contact Starburst Support for assistance.

443-e initial changes#

General#

  • Added public preview support for MaxCompute connector.

  • Enabled PyStarburst dataframe API by default.

  • Added support for Vault by Hashicorp and AWS Secrets Manager as external configuration providers.

  • Added support for updating and creating specific views or materialized views with data products in the SEP REST API.

  • Added support for automatic internal transport layer security (TLS) for managed statistics.

  • Added support for using a stored procedure to manually refresh tables in a table scan redirection.

  • Added keyboard shortcuts to the query name links in Saved queries and the assign button in Roles and Privileges in the Starburst Enterprise web UI to improve accessibility.

  • Added a toggle switch in the What can they do? dialog and in the Switch role dialog in the Starburst Enterprise web UI to improve accessibility.

  • Improved subquery cache hits by removing redundant predicates on data columns from cache key.

  • Changed the experimental.thread-per-driver-scheduler-enabled property to be disabled by default.

  • Increased character limit from 40 to 255 in the SEP REST API.

  • Fixed a bug that caused the creation of materialized views to fail when using MySQL as the cache service backend database if materialized_view_definitions is longer than 64K characters.

  • Fixed issue where a dynamic row filtering fallback mechanism could cause invalid results.

Db2 connector#

  • Added support for variable-precision timestamps to the nanosecond.

Delta Lake connector#

  • Improved speed at which tables and views are listed.

DynamoDB connector#

  • Added limited support for partial predicate pushdown.

  • Fixed unbounded VARCHAR handling.

Hive connector#

  • Added support for comments on partitioned columns in the File and Thrift Hive metastores.

  • Improved speed at which tables and views are listed.

  • Fixed bug that caused DESCRIBE materialized_view to fail.

Iceberg connector#

  • Improved speed at which tables and views are listed.

443-e.1 changes (31 May 2024)#

  • Fixed failure when translating Hive views that contain EXISTS clauses.

  • Fixed under-accounting of memory usage when writing strings to Parquet files.

  • Fixed potential failure when reading ORC files larger than 2GB.

  • Fixed startup failure when fault-tolerant execution is enabled with Google Cloud Storage exchange.

  • Fixed potential loss of a query completion event when multiple queries fail at the same time.

  • Fixed potential failure when queries contain filtered aggregations.

  • Fixed under-accounting of memory usage when writing strings to Parquet files.

  • Fixed complex predicate handling with table scan redirection.

  • Fixed last openRecordGroup not processed in FlatArrayBuilder.

  • Fixed potential query hang when there is an error processing data.

  • Fixed incorrect results for distinct count aggregations over a constant value.

443-e.2 was skipped

443-e.3 changes (14 Jun 2024)#

  • Fixed potential correctness issue on receivers refresh that could cause query hanging.

  • Backported IMDSv2 service metadata access.

443-e.4 changes (28 Jun 2024)#

  • Fixed incorrect results when specifying a value for the cassandra.partition-size-for-batch-select configuration property.

  • Fixed failure when reading Parquet files without field-id on structured types.

  • Fixed failure when writing to tables with Iceberg VARBINARY values.

  • Fixed rare query failure for array types when the data dictionary is encoded.

  • Fixed failure when partition column name contains uppercase in UNLOAD.

443-e.5 was skipped

443-e.6 changes (11 Jul 2024)#

  • Added flag for cleaning the storage when the system is loaded when using Starburst Warp Speed.

  • Added encoding to error code in OAuth2 callback handler.

  • Fixed reading empty files from S3 and GCS.

  • Fixed issue syncing partition metadata which could cause data deletion.

  • Fixed a bug preventing use of Starburst security in the Delta Lake connector.