Icehouse LakeOps overview#
Note
Icehouse LakeOps is a public preview feature. Contact Starburst Support with questions or feedback.
The Icehouse LakeOps service of Starburst Control Plane performs automatic table maintenance tasks on registered catalogs. It runs on its own coordinator and workers, separate from your SEP clusters.
Icehouse LakeOps runs automatic table maintenance at regular intervals and executes the following optimization tasks when needed:
Maintenance task |
Description |
|---|---|
Snapshot expiration |
Reduces storage by deleting older data snapshots. This action expires snapshots older than the retention period that you set for the catalog. |
Compaction |
Improves performance by optimizing your data file size. |
Orphan file removal |
Reduces storage by deleting files that are no longer linked to active data. |
Dangling delete file cleanup |
Reduces storage by deleting delete files that no longer apply to any data file. |
Requirements#
To use Icehouse LakeOps, you need the following:
A role with the Manage Icehouse Ops account-level privilege.
Starburst Control Plane in managed mode, with
control-plane.enabled=true.A Kubernetes cluster with Helm on Amazon EKS, Red Hat OpenShift, or RKE2.
Iceberg catalogs that you create in the Control Plane and that use AWS Glue or an Iceberg REST catalog, such as the Starburst Metastore.
Table data in Amazon S3 or S3-compatible object storage.
Limitations#
Icehouse LakeOps doesn’t support the following:
Catalogs that use a Hive Metastore
Azure Data Lake Storage or Google Cloud Storage
S3-compatible object storage that requires custom request headers
Bare metal or virtual machine deployments
See also#
For more information about Icehouse LakeOps, see the following pages: