Starburst Iceberg connector

The Starburst Iceberg connector is an extended, tested and fully supported version of the Iceberg connector with identical configuration and usage.

Requirements

• Fulfill the Iceberg connector requirements.

• Additional features of the connector require a valid Starburst Enterprise license, unless otherwise noted.

• Hive Metastore 3.1.2 or later.

Extensions

The connector includes all the functionality described in the Iceberg connector as well as these features:

• Ceph storage support

• Dell object storage support

• MinIO storage support

SQL support

The connector supports all of the SQL statements listed in the Iceberg connector documentation.

The connector supports reads of Iceberg tables written by SEP in ORC and Parquet formats.

Performance

Dynamic row filtering

Dynamic filtering, and specifically also dynamic row filtering, is enabled by default. Row filtering improves the effectiveness of dynamic filtering for a connector by using dynamic filters to remove unnecessary rows during a table scan. It is especially powerful for selective filters on columns that are not used for partitioning, bucketing, or when the values do not appear in any clustered order naturally.

As a result the amount of data read from storage and transferred across the network is further reduced. You get access to higher query performance and a reduced cost.

You can use the following properties to configure dynamic row filtering:

Dynamic row filtering properties

Property name	Description
dynamic-row-filtering.enabled	Toggle dynamic row filtering. Defaults to true. Catalog session property name is dynamic_row_filtering_enabled.
dynamic-row-filtering.selectivity-threshold	Control the threshold for the fraction of the selected rows from the overall table above which dynamic row filters are not used. Defaults to 0.7. Catalog session property name is dynamic_row_filtering_selectivity_threshold.
dynamic-row-filtering.wait-timeout	Duration to wait for completion of dynamic row filtering. Defaults to 0. The default causes query processing to proceed without waiting for the dynamic row filter, it is collected asynchronously and used as soon as it becomes available. Catalog session property name is dynamic_row_filtering_wait_timeout.

Security

If you have enabled built-in access control for SEP, you must add the following configuration to all Iceberg catalogs:

iceberg.security=system

Limitations

• Presorting tables is not supported.