Release 407-e LTS (28 Feb 2023)#

Starburst Enterprise platform (SEP) 407-e is the follow up release to the 406-e STS release and the 402-e LTS release.

This release is a long term support (LTS) release.

The 407-e release includes all improvements from the following Trino project releases:

It contains all improvements from the Starburst Enterprise releases since 402-e LTS:

Highlights since 402-e#

  • Added Run and troubleshoot to the query run options in the query editor.

  • Added SEP configuration validation that checks cluster configuration files for severe misconfigurations that can cause unwanted behavior.

  • Added table properties and schema properties entity types to built-in access control privileges.

  • Added support for DELETE in the MongoDB connector.

  • Expanded Fault-tolerant execution to support write operations on the MongoDB and BigQuery connectors, as well as exchange spooling in HDFS.

Breaking changes since 402-e#

  • SEP configuration validation now checks for catalog security misconfigurations that may cause security issues. If built-in access control is in use but a catalog is configured with the hive.security or delta.security configuration properties set to a value other than starburst, the cluster fails to start. The same failure occurs if the iceberg.security catalog configuration property is set to a value other than system. It is strongly recommended to fix this misconfiguration by setting all hive.security and delta.security properties to starburst and iceberg.security properties to system, but you may also disable configuration validation by setting starburst.config-validation.enabled=false in your config.properties file.

  • Built-in access control audit log now uses an improved backend schema for log storage. Existing log entries stored in the old schema are not automatically migrated to the new storage, so those older entries do not appear in the audit logs upon upgrading to this version of SEP.

  • The new SEP release requires usage of Trino Python client version 0.317 or newer.

  • The deprecated event-logger listener has been removed, and any SEP cluster configuration that uses this listener now fails to start. To resolve this, remove the event-listener.name=event-logger configuration property from any event-listener.properties configuration files.

  • If you use the Teradata direct connector, you must update the table operator. WARNING: This update is not backward-compatible. If you have multiple environments, such as a developement cluster, any cluster on a release before SEP 406-e is no longer able to connect to Teradata using the direct connector.

407-e.1 initial changes#

The following changes from 407-e.1 are all part of the first public release.

General#

  • Updated the Starburst Enterprise web UI to hide the Saved queries page in navigation when the query logger is not configured.

  • Fixed issue that caused query editor worksheet contents to be lost when renaming the saved query tab.

  • Fixed issue that prevented authorized users from downloading a query archive with the Run and troubleshoot option in the query editor.

Data products#

  • Added UI buttons to scroll horizontally through dataset tabs.

Security#

  • Released Lake formation security mapping as a public preview feature.

  • Improved user interface for function privileges.

  • Added support to the built-in access control privileges UI for custom schema, table, or column entity names.

  • Updated the Starburst Ranger plugin to use Ranger 2.3.

  • Fixed SSL connection failure for LDAP-based user impersonation.

Delta Lake connector#

  • Fixed query failure when reading parquet files written by Impala.

Hive connector#

Hudi connector#

  • Fixed query failure when reading parquet files written by Impala.

Iceberg connector#

  • Fixed query failure when reading parquet files written by Impala.

Oracle connector#

PostgreSQL connector#

Teradata connector#

Teradata direct connector#

  • Added support for fault-tolerant execution of read operations with the QUERY retry policy.

407-e.2 changes (15 Mar 2023)#

  • Fixed issue that caused query editor worksheets from previous SEP versions to fail after upgrading.

  • Fixed query failures on reading Parquet files generated by Kafka Connect. Applies to the Hive, Hudi, Iceberg, and Delta Lake connectors.

  • Fixed ANALYZE when Hive partition has non-canonical value.

  • Fixed rendering data cells with multiple space characters.

  • Fixed partition projection storage location.

407-e.3 changes (3 Apr 2023)#

  • Fixed NPE when DEBUG logging is enabled.

  • Fixed hive.timestamp-precision in Hive views.

  • Fixed unnecessary identity impersonation for column masks and row filters.

  • Fixed query loading error in query overview.

  • Fixed a possible query failure with a Kerberized Hive connector when the query executes longer than the Kerberos ticket lifetime.

  • Updated Teradata table operator to OpenSSL 3.1.0.

  • Fixed error message for catalog errors when listing metadata.

  • Fixed mixed-case AWS roles not working when using AWS Lake Formation access control.

407-e.4 changes (13 Apr 2023)#

  • Removed SnakeYAML library references from dependency trees and Ranger plugin as a preventative measure for CVE-2022-1471.

407-e.5 changes (28 Apr 2023)#

  • Fixed incorrect results when the query passed to the MongoDB query table function contains helper functions like ISODate.

  • Fixed reading INT32 values without decimal logical annotation in parquet files as a DECIMAL type in Trino. Applies to the Hive, Hudi, Delta Lake, and Iceberg connectors.

  • Fixed a performance issue with access control on Ranger with a large number of user-based policies.

  • Fixed query failure when a Kafka topic contains messages with a NULL value, also known as a tombstone message.

  • Fixed a potential internal communication secret link. See the security advisory for more information.

  • Remediated the following CVEs:

    • CVE-2020-36518

    • CVE-2022-3509

    • CVE-2022-3510

    • CVE-2022-40152

    • CVE-2022-42003

    • CVE-2023-1370

407-e.6 changes (15 May 2023)#

  • Fixed incorrect handling of boolean data type in Starburst Warp Speed.

  • Fixed Starburst Enterprise web UI overview page error when internal TLS is enabled.

407-e.7 changes (12 Jun 2023)#

  • Fixed error on query details page for some queries executed on SEP versions prior to 395-e.

  • Fixed falure when invoking current_timestamp.

  • Changed access control systems to always enable built-in access control for data products when built-in access control is enabled for the cluster. access-control.name=starburst can no longer be set in an access-control.properties file when built-in access control is not enabled for the cluster.

407-e.8 changes (14 Jul 2023)#

  • Fixed Iceberg statistics loading after snapshot expiration.

407-e.9 changes (9 Aug 2023)#

  • Fixed issue where querying Redshift catalogs returned incorrect results when encountering unsupported types.

  • Fixed reading concatenated GZIP streams.

  • Fixed query results for joins on bucketed Hive tables.

407-e.10 changes (29 Aug 2023)#

  • Fixed parallel read of tables with non-clustered index in SQL Server.

  • Fixed parsing logic for long values out of text rc files.

  • Fixed write conflict detection for UPDATE/DELETE/MERGE operations. In rare situations, this issue may have resulted in duplicate rows when multiple of these operations were run at the same time, or at the same time as an optimize procedure.

  • Fixed caching in LDAP group provider.

  • Improved performance in BIAC for queries with large numbers of columns.

407-e.11 changes (19 Sep 2023)#

  • Fixed issue with Query Editor not running queries to completion which may have prevented results from being visible.

  • Fixed potential unnecessary autoscaling triggers when Warp Speed is enabled.

  • Fixed Kudu merge error between two CTAS tables.

407-e.12 was skipped.

407-e.13 changes (2 Oct 2023)#

  • Fixed potential for incorrect results when a table has deletion vectors in the Delta Lake connector.

  • Fixed issue where the catalog name was included twice when creating an access policy for a catalog session property (BIAC).

  • Added access control check to redirected tables when getting comments.

  • Fixed JavaScript policy evaluation in Ranger and Privacera.

  • Fixed performance when planning queries involving multiple window functions.

407-e.14 changes (18 Oct 2023)#

  • Fixed performance issue when reading with the native CSV reader on Hive.

  • Improved performance for filtering catalogs, schemas, tables, and columns in BIAC.

  • Fixed MongoDB mixed case schemas with custom roles.

  • Remediated the following CVEs:

    • CVE-2018-20839

    • CVE-2023-36478

    • CVE-2023-36479

407-e.15 changes (13 Nov 2023)#

  • Fixed incorrect column statistics for Parquet file format in manifest files.

  • Cast char fields, if necessary, to varchar type in Hive view translations.

  • Fixed incorrect results for queries involving ORDER BY and window functions with ordered frames.

  • Masked additional sensitive values in log files.

  • Fixed incorrect results in MongoDB when a query contains several != or NOT IN predicates.

  • Fixed JavaScript policy evaluation in Ranger and Privacera.

  • Improved support for concurrent updates of table statistics in Glue.

  • Support RENAME SCHEMA and RENAME TABLE when snowflake.database-prefix-for-schema.enabled=true.

  • Fixed incorrect results for queries involving an aggregation in a correlated subquery.

407-e.16 changes (27 Nov 2023)#

  • Fixed possible JVM crash when reading short decimal columns in parquet files created by Impala (Hive, Hudi, Delta, Iceberg).

  • Remediated CVE-2023-41900.

  • Granting execution rights on a non-qualified function no longer makes all catalogs visible.

407-e.17 changes (21 Dec 2023)#

Warning

This release upgrades Ranger to 2.4, and to avoid a breaking change, adhere to the following steps. Before upgrading to this version of SEP, record your current version of Ranger (for k8s deployments, found in your starburst-ranger values.yaml file as admin.image.tag and usersync.image.tag). While upgrading SEP, before deploying the update to your cluster nodes, you must revert the Ranger 2.4 version tag back to the previous version.

  • Upgraded JDK to 17.0.8 to fix worker JVM crashes.

  • Fixed query failure when reading parquet column index for timestamped columns in Hive, Delta, Iceberg, and Hudi tables.

  • Fixed a potential Starburst Warp Speed crash when creating Lucene text indexes.

  • Fixed deletes in Delta tables with partitions with special characters.

  • Fixed coordinator memory leak.

407-e.18 changes (18 Jan 2024)#

  • Fixed a potential issue with SEP inadvertently changing users’ passwords in Ranger when used with Ranger Admin 2.4.0.

  • Fixed incorrect results on parquet files containing page indexes when the query has filters on multiple columns in Hive, Delta, and Hudi tables.

407-e.19 changes (14 Feb 2024)#

  • Fixed query failure when reading array columns.

  • Fixed a bug where query history displayed queries of another user.