Release 312-e LTS#

This release is a long term support (LTS) release.

General changes#

  • Display actual statistics in EXPLAIN ANALYZE VERBOSE.

  • Improve cardinality estimation for aggregations.

  • Add Oracle compatibility functions

  • Add support for column statistics in Glue.

  • Add support for Glue API proxy.

  • Support prepared statements that are longer than 4K bytes.

  • Add Starburst Generic JDBC connector.

  • Add Oracle connector (requires license).

  • Add Starburst BigQuery Connector (requires license).

  • Add Starburst Teradata connectors (requires license).

  • Add MapR Hive support (requires license).

  • Support resource group selection based on client tags for enterprise ODBC and JDBC drivers.

  • Add compression and encryption support for Spill to Disk.

  • Enable Cost-Based Optimizer (CBO) by default.

  • Add CloudWatch Agent compatible logger.

  • Fix incorrect results when dividing certain decimal numbers.

Security changes#

  • Add security audit log collection.

  • Add ldap.bind-dn and ldap.bind-password LDAP properties to allow LDAP authentication access LDAP server using service account.

  • Add optional additional security configuration options for authorizing resource access on HTTP level. Find more information about resource rules in System access control.

  • Raise required Java version to 8u161. This version allows unlimited strength cryptography.

CLI changes#

  • Fix failure when selecting a value of type UUID.

JDBC changes#

  • Fix failure when selecting a value of type UUID.

Hive connector changes#

Warning

The Hive configuration properties hive.empty-bucketed-partitions.enabled and hive.multi-file-bucketing.enabled were removed in this release. Behaviour of Hive Connector with hive.empty-bucketed-partitions.enabled=true and hive.multi-file-bucketing.enabled=true is incorporated into default implementation and it cannot be disabled.

  • Add support for Azure Data Lake Storage gen 2 (ABFS).

  • Enable Hive views execution using a config toggle.

  • Support CSV table storage format.

  • Support Apache Ranger authorization (requires licence).

  • Support Apache Sentry authorization (requires licence).

  • Support Avro tables with avro.schema.url in kerberized HDFS.

  • Fix Avro schema loading when SERDEPROPERTIES set.

  • Support partitioned Avro table with very long column type definition.

  • Support tables located in HDFS encryption zones managed with Hadoop KMS.

  • Add support for user impersonation when communicating with Hive Metastore.

  • Add hive.s3.positioned-reads.enabled configuration property allowing disabling positioned reads optimization. Consider disabling the optimization when working with small Parquet files.

SQL Server connector changes#

  • Support user impersonation.

  • Support retrieving table and column statistics. The statistics can be inspected with SHOW STATS and are taken into account in Cost-based optimizations.

PostgreSQL connector changes#

  • Support user impersonation.

  • Support retrieving table and column statistics. The statistics can be inspected with SHOW STATS and are taken into account in Cost-based optimizations.

MySQL connector changes#

JDBC based connectors changes#

  • Add metadata cache that is configured with metadata.cache-ttl and metadata.cache-missing. By default caching is disabled (equivalent to metadata.cache-ttl=0s and metadata.cache-missing=false).

  • Collect JMX metrics about metadata information collection and connection usage.

MongoDB connector changes#

  • Fix queries involving joins or aggregations on ObjectId type.

  • Fix queries involving joins or aggregations on row types containing ObjectId.

Server RPM changes#

  • Change default location of the http-request.log to /var/log/presto. Previously, the log would be located in /var/lib/presto/data/var/log by default.

Update 2#

Security changes#

  • Add support for selective disabling of KRB authentication for Presto clients.

  • Improve authenticated user authorization.

RPM changes#

  • Limit the size of http-request.log to 100MB by default.

Hive connector changes#

  • Optimize checking object privileges for Sentry. Especially queries like SHOW TABLES on schemas with thousands of tables should be now much faster.

JDBC based connectors changes#

  • Fail query when accessing unsupported column. This is backward incompatible change, user may restore previous behaviour by using unsupported-type.handling-strategy=IGNORE connector configuration property or by setting unsupported_type_handling_strategy catalog session property to IGNORE.

  • Add option to convert unsupported column values to unbounded VARCHAR. This could be achieved by using unsupported-type.handling-strategy=CONVERT_TO_VARCHAR connector configuration property or by setting unsupported_type_handling_strategy catalog session property to CONVERT_TO_VARCHAR.

Oracle connector changes#

  • Introduce oracle.number.default-scale configuration property (and matching number_default_scale session property). This value will be used when mapping Oracle NUMBER data type (without precision and scale) to Presto DECIMAL.

  • Introduce oracle.number.rounding-mode configuration property (and matching number_rounding_mode session property). This value will be used for rounding fractional part of Oracle NUMBER data type (without precision and scale) value to Presto DECIMAL where needed scale exceeds configured (with oracle.number.default-scale) Presto DECIMAL scale.

  • Push down LIMIT clause to Oracle Database.

Teradata connector changes#

  • Push down LIMIT clause to Teradata Database.

BigQuery connector changes#

  • Push down LIMIT clause.

Update 3#

Teradata Direct connector changes#

  • Fix handling unsupported types.

Update 4#

Hive connector changes#

  • Update Apache Ranger dependency to 1.2.0

  • Support column masking for tag based policies in ranger Hive security.

Update 5#

General changes#

  • Respect X-Forwarded-For header when retrieving query submitting user’s source address. The behavior can be controlled with dispatcher.forwarded-header configuration property.

Security changes#

  • Allow schema owner to create, drop and rename schema when using file-based connector access control.

Teradata connector changes#

  • Fix bug causing incorrect results for LIMIT pushdown.

  • Support Kerberos authentication.

Teradata Direct connector changes#

  • Introduce system table system.table_operator_logs that can expose logs from Table Operator executions.

  • Propagate proper Teradata errors.

  • Support data transfer encryption.

  • Support Kerberos authentication.

  • Support NUMBER (without precision and scale) data format.

JDBC based connectors changes#

  • Restoring default behaviour for column with unsupported types. Such columns will be ignored. User may configure to fail such queries by using unsupported-type.handling-strategy=FAIL connector configuration property or by setting unsupported_type_handling_strategy catalog session property to FAIL.

Generic JDBC connector changes#

  • Support user impersonation.

Update 6#

Teradata connector changes#

  • Support views with optimized metadata queries.

Teradata Direct connector changes#

  • Support views with optimized metadata queries.

Update 7#

General changes#

  • Fix a rare failure when running EXPLAIN ANALYZE on a query containing window functions.

  • Fix failure when querying /v1/resourceGroupState endpoint for non-existing resource group.

  • Allow to display details of query that is in the queue or during semantic analysis. Query details are accessible from the web ui or system.runtime.queries.

  • Allow to cancel query that is in the queue or during semantic analysis. Query can be cancelled from the web ui or using builtin procedure system.runtime.kill_query.

  • Remove distributed_planning_time_ms column from system.runtime.queries.

  • Change the semantics of analysis_time_ms column from system.runtime.queries. Previously it was showing the time spent on query planning, now it shows time spent on semantic analysis.

  • Add planning_time_ms column to system.runtime.queries that shows time spent on query planning.

  • Improve performance of information schema tables.

  • Eliminate cross joins in some queries with coercions and complex expressions on join conditions.

Hive connector changes#

  • Fix reading collection delimiter set by Hive version earlier than 3.0.

  • Fix reading LZOP compressed text files.

  • Support INSERT INTO bucketed Hive table.

  • Allow to ignore partitions that do not have matching data directory with hive.ignore-absent-partitions=true connector configuration property.

Teradata Direct connector changes#

  • Add system.validate_table_operator procedure that can validate if connector is installed and configured properly. This can be executed with CALL teradata.system.validate_table_operator().

Kudu connector changes#

  • Update Kudu client library version to 1.10.0.

Update 8#

Hive connector changes#

  • Add support for custom S3 filesystems.

  • Add ranger.policy-cache-dir property that enables Presto startup without connectivity with Ranger.

JDBC based connectors changes#

  • Improve metadata caching by storing list of schemas and tables.

Teradata connectors changes#

  • Optimize metadata query to verify table existence.

  • Optimize metadata query to retrieve table columns.

Update 9#

General changes#

  • Improve configuration file parsing. Trailing whitespace in configuration values will is now ignored just as the leading whitespace used to be. Parsing of boolean values is now strict, anything other than true or false (case-insensitively) is now considered an error. Previously anything other than true was considered a false value.

  • Limit total size of http-request.log.*.gz files. This is controlled by http-server.log.max-history * http-server.log.max-size, which is 1500MB by default.

  • Allow overriding session time zone for clients via the sql.forced-session-time-zone configuration property.

  • Allow specifying column lists to analyze in ANALYZE.

Teradata connector changes#

  • Create table in Teradata with NO PRIMARY INDEX.

Oracle connector changes#

  • Fix CREATE TABLE AS SELECT when table is created in non user schema.

  • Support Kerberos authentication.

JDBC driver changes#

  • Fix listing of views in java.sql.DatabaseMetaData.getTables.

Hive connector changes#

  • Add ranger.cache-refresh-interval property that allows ranger groups data to be refreshed when ranger cache is enabled.

  • Fix parametric column types support for Hive views.

  • Fix projection pushdown during cross-join elimination when outer join is present.

MySQL connector changes#

  • Add mysql.jdbc.use-information-schema configuration option to control whether MySQL JDBC driver should use MySQL information_schema to answer metadata queries. This may be helpful when diagnosing problems.

Update 10#

General changes#

  • Fix rare failure on the client side due to the server sending incorrect additional row of data.

  • Add procedure drop_stats for removing statistics for a table or a specified set of partitions.

Hive connector changes#

  • Allow to to specify only hive.metastore-cache-ttl when enabling Hive Metastore caching (without setting hive.metastore-refresh-interval, which is disabled by default).

  • Optimize checking object privileges for Sentry. Especially queries like SHOW TABLES on schemas with thousands of tables should be now much faster.

  • Improve hive query execution when view was created with tables without schema specified.

Security changes#

  • Improve query results authorization security with one-time tokens.

  • Add user groups and user group providers.

  • Add group support to file-based access control rules.

  • Add group provider which uses LDAP group information.

Teradata connector changes#

  • Make data transfer resilient to network outages.

Update 11#

Hive connector changes#

  • Fix reading Parquet statistics for varchar data type. Previously queries would fail with Corrupted statistics for column ... error.

  • Allow using several Hive catalogs with different configuration (with different Kerberos authentication setup).

Update 12#

Hive connector changes#

  • Reduce Metastore load when inserting data or analyzing tables.

Update 13#

General changes#

  • Limit total size of server.log.*.gz files. This is controlled by log.max-history * log.max-size, which is 1500MB by default.

  • Reduce number of metadata queries during planning. This change disables stats collection for non-EXPLAIN queries. If you want to have access to such stats in query completed event listener, you need to re-enable stats by collection using collect-plan-statistics-for-all-queries property.

Hive connector changes#

  • Fix query failure when reading Parquet file with character data and no statistics.

  • Add support for Parquet files with compatible decimal precision.

Update 14#

Hive connector changes#

  • Fix listing user’s roles and applicable privileges when roles form a diamond inheritance structure.

  • Fix incorrect data returned when using S3 Select on uncompressed files. In our testing, S3 Select was apparently returning incorrect results when reading uncompressed files, so S3 Select is disabled for uncompressed files.

  • Fix incorrect data returned when using S3 Select on a table with skip.header.line.count or skip.footer.line.count property. S3 Select API does not support skipping footers or more than one line of a header. In our testing, S3 Select was apparently sometimes returning incorrect results when reading a compressed file with header skipping, so S3 Select is disabled when any of these table properties is set to non-zero value.

Security changes#

  • Fail LDAP authentication if username contains any special LDAP characters

Update 15#

Hive connector changes#

  • Fix INSERT or CREATE TABLE ... AS query failure when hive.collect-column-statistics-on-write is enabled and one of the inserted REAL or DOUBLE values is infinite or NaN.

  • Improve performance of SHOW SCHEMAS and SHOW TABLES queries with hive.security=sentry enabled.

Update 16#

Hive connector changes#

  • Add hive.fs.new-directory-permissions configuration property for setting the permissions of new directories created by Presto. Default value is 0777.

Versions 17-22 were not released to the public.

Update 23#

General changes#

  • Backport cluster-internal communication

    • Support only shared secret usage

    • Remove support for Kerberos for internal communication

    • Cluster accessed via HTTPS must have internal-communication. shared-secret configured