GPU acceleration#

Note

GPU acceleration is a limited general availability (limited GA) feature. It is supported for production use only on qualified hardware configurations.

GPU acceleration improves query performance in SEP by offloading eligible parts of query processing to Nvidia GPUs installed on worker nodes. SEP automatically decides how much of a given query to run on GPUs based on the query plan and input data, so no query changes are required. Performance gains vary by query, ranging from no improvement to an order of magnitude faster than running on CPU alone.

Requirements#

GPU acceleration requires the following:

A supported Nvidia GPU (Ampere, Ada Lovelace, Hopper, or Blackwell architecture) on each SEP worker. For example, the AWS EC2 g6 and g7e instance families provide compatible GPU hardware.
At least 24 GB of GPU memory (VRAM). 80 GB or more is recommended.
SEP uses only one GPU per worker. On physical hosts with multiple GPUs, run multiple workers to take advantage of all of them.
The Nvidia driver installed on each machine running a SEP worker. For Kubernetes deployments, install the Nvidia GPU operator so the driver is loaded in each pod.

Supported data sources#

You can use GPU acceleration with data from most SEP connectors. The best performance is achieved with the Iceberg and Hive connectors. The Delta Lake and Great Lakes connectors, and Starburst Warp Speed, are not yet fully supported.

Note

For best performance, configure file system cache on SSD-backed worker nodes for Iceberg and Hive catalogs.

Supported data types#

GPU acceleration supports the following data types:

Integers of all sizes: TINYINT, SMALLINT, INTEGER, BIGINT
Floating-point numbers: REAL, DOUBLE
DECIMAL, at all precisions
BOOLEAN
Strings: VARCHAR and CHAR
VARBINARY
DATE
TIMESTAMP, up to nanosecond precision

Supported query operations#

GPU acceleration supports the following query operations:

Table scans
Filters
Projections
Arithmetic and logical expressions
LIKE patterns
CASE statements with a single WHEN clause
GROUP BY clauses, and aggregate functions such as min, max, sum, avg, count, and any_value
Inner and left joins
A limited subset of built-in scalar functions, such as length, substring, date_trunc, regexp_replace, and extracting the day, hour, minute, or second from a date or timestamp value
TopN queries

Note

Use EXPLAIN ANALYZE to see which parts of a query plan ran on a GPU, and why any remaining parts were not eligible for GPU acceleration.

Configuration#

Set the following configuration properties to enable GPU acceleration.

In the coordinator’s config.properties file:

gpu-execution=true
experimental.force-single-node-query=true

In each worker’s config.properties file:

task.gpu-execution.enabled=true

GPU acceleration configuration properties#
Property name	Description
`gpu-execution`	Allows the planner to consider queries for GPU execution. Equivalent session property: `gpu_execution_enabled`.
`experimental.force-single-node-query`	Enables single node execution mode, which maximizes GPU benefit since most of the query plan can be delegated to the GPU without moving intermediate data in and out of GPU memory.
`task.gpu-execution.enabled`	Enables GPU execution for tasks running on that node.

Caution

Setting task.gpu-execution.enabled on the coordinator causes SEP to fail at startup. GPU execution is not supported on the coordinator.

Catalog configuration#

For Hive and Iceberg catalogs, tune the following properties to control split sizes for GPU-accelerated queries:

GPU acceleration catalog configuration properties#
Property name	Description
`hive.max-split-size`	Increases the split size for better performance. Equivalent session property: `max_split_size`.
`hive.parquet.max-split-size`	Increases the split size for Parquet files. Equivalent session property: `parquet_max_split_size`.
`iceberg.max-split-size`	Increases the split size for better performance. Equivalent session property: `max_split_size`.
`iceberg.experimental.composite-splits.enabled`	Improves performance when files are small. Equivalent session property: `experimental_composite_splits_enabled`.

Kubernetes configuration#

For Kubernetes deployments, request a GPU for each worker by adding nvidia.com/gpu: 1 to the worker’s resources.requests and resources.limits sections in your deployment YAML:

worker:
  resources:
    requests:
      memory: "55Gi"
      cpu: "7"
      nvidia.com/gpu: 1
    limits:
      memory: "55Gi"
      nvidia.com/gpu: 1

After you deploy the Helm chart, apply the following patch, replacing <namespace> with the namespace of your SEP deployment. The worker pod repeatedly restarts until this patch is applied:

kubectl patch deployment worker -n <namespace> --type=json -p='[{"op":"add","path":"/spec/template/spec/containers/0/resources/limits/nvidia.com~1gpu","value":"1"}]'

Note

This patch modifies the deployment directly and is not tracked by Helm. A subsequent helm upgrade can reset the deployment and remove the patch, requiring you to reapply it.

On AWS, the nvidia-device-plugin-daemonset attempts to run on all worker nodes by default, even those without a GPU. Restrict it to only the GPU-enabled node group used by SEP workers, replacing <your-node-group-name> with your managed node group name:

kubectl patch daemonset nvidia-device-plugin-daemonset -n kube-system --type merge -p '{"spec":{"template":{"spec":{"nodeSelector":{"eks.amazonaws.com/nodegroup":"<your-node-group-name>"}}}}}'

Monitoring#

GPU memory usage is available through JMX as GpuDeviceStats on each SEP worker. Retrieve the stats with a command such as:

curl -s http://<worker-host>:8080/v1/jmx/mbean/trino.operator.gpu:name=GpuDeviceStats

GPU acceleration JMX attributes#
Attribute	Layer	Description
`DeviceTotalBytes`	Physical	Total physical GPU memory capacity. Constant for the life of the worker process.
`DeviceUsedBytes`	Physical	Physical GPU memory currently in use, including CUDA context overhead not tracked by RMM.
`DeviceUsedDistribution`	Physical	Histogram of `DeviceUsedBytes`, sampled at the interval configured by `gpu.device-stats.sampling-interval` (default `5s`).
`RmmAllocatedBytes`	Logical	Bytes currently allocated by SEP’s GPU operators through RMM, not including CUDA context overhead.
`RmmAllocatedDistribution`	Logical	Histogram of `RmmAllocatedBytes`, sampled at the same interval.
`RmmPeakAllocatedBytes`	Logical	High-water mark of RMM-allocated memory since the worker started.

Limitations#

Queries against Starburst Warp Speed catalogs do not benefit from GPU acceleration, regardless of the underlying source connector. See Supported data sources for other connector restrictions.

The following operations are not eligible to run on GPUs, and run on CPUs instead:

Any non-SELECT portion of a query
Unsupported SQL scalar functions
Unsupported aggregation and window functions
Custom user-defined functions (UDFs)
RIGHT and CROSS joins
Table scans other than Iceberg and Hive tables using the Parquet file format
Data exchanges between workers
DISTINCT clauses in SELECT clauses
CASE statements with multiple WHEN clauses
Unsupported data types, including ROW, MAP, ARRAY, and JSON