Release 413-e LTS (30 May 2023)#

Starburst Enterprise platform (SEP) 413-e LTS is the follow up release to the 413-e STS release and the 407-e LTS release.

This release is a promotion of the original 413-e STS release in April 2023 into a long term support (LTS) release.

The 413-e release includes all improvements from the following Trino project releases:

It contains all improvements from the Starburst Enterprise releases since 407-e LTS:

Highlights since 407-e#

Breaking changes since 407-e#

  • SEP now disallows setting access-control.name=starburst as an access control system when built-in access control is not enabled on the coordinator. If this property is configured in an access control properties file but starburst.access-control.enabled is not set to true in config.properties on the coordinator node, the cluster fails to start.

  • Upgrades to this release involve an automated migration in the backend service database. If the cluster fails to start with an error message Migration of schema "public" to version "413.2 - Category" failed, contact Starburst Support for assistance.

  • SEP configuration validation now verifies that there is no combination of insecure and secure authentication types between the web-ui.authentication.type and http-server.authentication.type configuration properties. If this validation fails, the cluster fails to start and a descriptive error message appears. It is strongly recommended to fix this misconfiguration by setting both configuration properties to secure authentication types. You may also disable configuration validation by setting starburst.config-validation.enabled=false in your config.properties file.

  • The http-server.authentication.oauth2.groups-field property was renamed to deprecated.http-server.authentication.oauth2.groups-field. You must update the property name, or the cluster fails to start.

  • The accelerated Parquet reader was removed, and replaced with an improved implementation in Trino. The catalog configuration properties iceberg.accelerated-parquet-reader.enabled, hive.accelerated-parquet-reader.enabled, and delta.accelerated-parquet-reader.enabled, and the related catalog session properties are deprecated and must be removed from catalog configurations or the cluster does not start. The new implemenetation is enabled by default, see Parquet format configuration properties for details applicable to all object storage connectors.

  • The following connector names have been removed and must be changed to allow successful cluster starts:

    • hive-hadoop2 to hive

    • memsql to singlestore

  • The following connector names have been changed. The old connector names continue to work but it is strongly recommended to change catalog configurations to the new names:

    • delta-lake to delta_lake

    • generic-jdbc to generic_jdbc

    • sap-hana to sap_hana

    • snowflake-jdbc to snowflake_jdbc

    • snowflake-distributed to snowflake_distributed

    • teradata-direct to teradata_direct

    • warp-speed to warp_speed

  • This release introduced a Trino bug that causes some INSERT INTO ... SELECT statements to fail with an “invalid offset” exception when exchange data compression is enabled as part of a Fault-tolerant execution configuration. For a temporary workaround, set the exchange.compression-enabled configuration property to false.

  • The fault_tolerant_execution_target_task_input_size and fault_tolerant_execution_target_task_split_count configuration properties for Fault-tolerant execution are deprecated and must be removed from the config.properties file or the cluster does not start.

413-e initial changes#

The following changes from the 413-e STS are all part of the first public release.

General#

Security#

IBM Db2 connector#

  • Added support for fault-tolerant execution of read operations with a TASK retry policy.

MongoDB connector#

MySQL connector#

413-e.1 changes (30 May 2023)#

  • Fixed Starburst Enterprise web UI Overview page error when internal TLS is enabled.

  • Fixed incorrect handling of BOOLEAN data type in Starburst Warp Speed.

  • Fixed a potential internal communication secret leak. See the security advisory for more information.

  • Remediated CVE-2023-1370.

  • Fixed read permissions for columns required by table functions. See the full security advisory for more information.

413-e.2 changes (12 Jun 2023)#

  • Multiple backend improvements to Starburst Warp Speed.

413-e.3 changes (14 Jul 2023)#

  • Fixed resource leak in Starburst Warp Speed resiliency.

  • Fixed Iceberg statistics loading after snapshot expiration.

  • Fixed accessing local db when warming an empty group in Starburst Warp Speed.

413-e.4 changes (26 Jul 2023)#

  • Fixed Starburst Warp Speed loading issue in GKE and AKS deployments.

  • Fixed issues when querying Hive legacy views in Starburst Warp Speed.

413-e.5 changes (9 Aug 2023)#

  • Fixed issue where querying Redshift catalogs returned incorrect results when encountering unsupported types.

  • Fixed Iceberg query failure when reading ORC files with nullable time columns.

  • Fixed reading concatenated GZIP streams.

  • Fixed TEXTFILE and RCTEXT handling of an escaped escape character.

  • Fixed missing data when reading large text and sequence files with a single header row.

413-e.6 changes (29 Aug 2023)#

  • Fixed parallel read of tables with non-clustered index in SQL Server.

  • Fixed write conflict detection for UPDATE/DELETE/MERGE operations. In rare situations, this issue may have resulted in duplicate rows when multiple of these operations were run at the same time, or at the same time as an optimize procedure.

  • Fixed caching in LDAP group provider.

  • Improved performance in BIAC for queries with large numbers of columns.

413-e.7 changes (19 Sep 2023)#

  • Fixed issue with Query Editor not running queries to completion which may have prevented results from being visible.

  • Fixed potential unnecessary autoscaling triggers when Warp Speed is enabled.

  • Fixed Kudu merge error between two CTAS tables.

413-e.8 was skipped.

413-e.9 changes (2 Oct 2023)#

  • Fixed license issue in DynamoDB connector.

  • Fixed run and download when browser policies disallow downloading larger result sets.

  • Fixed potential for incorrect results when a table has deletion vectors in the Delta Lake connector.

  • Fixed issue where the catalog name was included twice when creating an access policy for a catalog session property (BIAC).

  • Added access control check to redirected tables when getting comments.

  • Fixed an edge case that might result in a correctness issue when some data is indexed and cached by Warp Speed.

  • Fixed JavaScript policy evaluation in Ranger and Privacera.

  • Fixed performance when planning queries involving multiple window functions.

413-e.10 changes (18 Oct 2023)#

  • Fixed performance issue when reading with the native CSV reader on Hive.

  • Improved performance for filtering catalogs, schemas, tables, and columns in BIAC.

  • Fixed MongoDB mixed case schemas with custom roles.

  • Remediated the following CVEs:

    • CVE-2018-20839

    • CVE-2023-36478

    • CVE-2023-36479

413-e.11 changes (13 Nov 2023)#

  • Fixed potential Starburst Warp Speed crash that can happen when more than a single Starburst Warp Speed catalog is used.

  • Fixed incorrect column statistics for Parquet file format in manifest files.

  • Cast char fields, if necessary, to varchar type in Hive view translations.

  • Fixed incorrect results for queries involving ORDER BY and window functions with ordered frames.

  • Masked additional sensitive values in log files.

  • Fixed incorrect results in MongoDB when a query contains several != or NOT IN predicates.

  • Fixed JavaScript policy evaluation in Ranger and Privacera.

  • Improved support for concurrent updates of table statistics in Glue.

  • Support RENAME SCHEMA and RENAME TABLE when snowflake.database-prefix-for-schema.enabled=true.

  • Remediated CVE-2023-39410.

  • Fixed incorrect results for queries involving an aggregation in a correlated subquery.

413-e.12 changes (27 Nov 2023)#

  • Fixed possible JVM crash when reading short decimal columns in parquet files created by Impala (Hive, Hudi, Delta, Iceberg).

  • Remediated CVE-2023-41900.

  • Granting execution rights on a non-qualified function no longer makes all catalogs visible.

  • Fixed inability to create Data Domains and Data Products when Global Ranger Access Control is enabled.

413-e.13 changes (21 Dec 2023)#

Warning

This release upgrades Ranger to 2.4, and to avoid a breaking change, adhere to the following steps. Before upgrading to this version of SEP, record your current version of Ranger (for k8s deployments, found in your starburst-ranger values.yaml file as admin.image.tag and usersync.image.tag). While upgrading SEP, before deploying the update to your cluster nodes, you must revert the Ranger 2.4 version tag back to the previous version.

  • Improved query planning time on Hive tables without statistics generated.

  • Upgraded JDK to 17.0.8 to fix worker JVM crashes.

  • Fixed long query planning times for queries with many local exchanges.

  • Fixed query failure when reading parquet column index for timestamped columns in Hive, Delta, Iceberg, and Hudi tables.

  • Fixed a potential Starburst Warp Speed crash when creating Lucene text indexes.

  • Fixed reading JSON columns with more than 128 keys.

  • Fixed deletes in Delta tables with partitions with special characters.

  • Fixed coordinator memory leak.

413-e.14 changes (18 Jan 2024)#

  • Fixed a potential issue with SEP inadvertently changing users’ passwords in Ranger when used with Ranger Admin 2.4.0.

  • Fixed incorrect results on parquet files containing page indexes when the query has filters on multiple columns in Hive, Delta, and Hudi tables.

413-e.15 changes (14 Feb 2024)#

  • Fixed query failure when reading array columns.

  • Fixed a bug where query history displayed queries of another user.

413-e.16 changes (11 Mar 2024)#

  • Updated Kubernetes external secret operator.

  • Fixed UI authentication for large authentication tokens.

  • Fixed schema, table, and function visibility in BIAC filtering.

  • Fixed a bug where column statistics created in SEP would not be visible in Hive when using CDP 7.

413-e.17 changes (28 Mar 2024)#

  • Fixed an issue which caused the sync_partition_metadata operation to fail when partition paths had case changes.

  • Restored support for SymlinkTextInputFormat for text formats.

  • Fixed reading Delta Lake files with encoded characters on Azure.

413-e.18 changes (17 Apr 2024)#

  • Fixed possible worker crashes when running aggregation queries due to out-of-memory error.

  • Fixed incorrect results when querying a table being modified concurrently.

413-e.19 changes (20 May 2024)#

  • Fixed correctness bug in constant literal distinct aggregation.

  • Fixed Prometheus whiteListObjectNames being overwritten when KEDA is enabled.