GPU execution#

Starburst Torin is the GPU execution engine in SEP. It improves query performance by offloading eligible parts of query processing to NVIDIA GPUs installed on worker nodes. SEP automatically decides how much of a given query to run on GPUs based on the query plan and input data, so no query changes are required. Performance gains vary by query, ranging from no improvement to an order of magnitude faster than running on CPU alone.

Note

Starburst Torin is supported for production use only on qualified hardware configurations.

Requirements#

Torin requires the following:

A supported NVIDIA GPU (Ampere, Ada Lovelace, Hopper, or Blackwell architecture) on each SEP worker. For example, the AWS EC2 g6 and g7e instance families provide compatible GPU hardware.
At least 24 GB of GPU memory (VRAM). 80 GB or more is recommended.
SEP uses only one GPU per worker. On physical hosts with multiple GPUs, run multiple workers to take advantage of all of them.
CUDA 13, which SEP uses to run queries on the GPU.
The NVIDIA driver installed on each machine running a SEP worker. For Kubernetes deployments, install the NVIDIA GPU operator so the driver is loaded in each pod.

Supported data sources#

You can use Torin with data from most SEP connectors. It is fully supported with the Iceberg, Hive, Delta Lake, and Great Lakes connectors. Starburst Warp Speed is not yet fully supported.

Note

For best performance, configure file system cache on SSD-backed worker nodes for Iceberg, Hive, and Delta Lake catalogs.

Supported data types#

Torin supports the following data types:

Integers of all sizes: TINYINT, SMALLINT, INTEGER, BIGINT
Floating-point numbers: REAL, DOUBLE
DECIMAL, at all precisions
BOOLEAN
Strings: VARCHAR and CHAR
VARBINARY
DATE
TIMESTAMP, up to nanosecond precision

Supported query operations#

Torin supports the following query operations:

Table scans
Filters
Projections
Arithmetic and logical expressions
LIKE patterns
CASE statements
GROUP BY clauses, and aggregate functions such as min, max, sum, avg, count, and any_value
Inner, left, right, and cross joins
Window functions: row_number, rank, dense_rank, and avg on DECIMAL values
A limited subset of built-in functions and expressions, including:
- String functions: concat, length, lower, regexp_replace, replace, substring, trim, and upper
- Conditional and comparison functions: coalesce, nullif, and least
- Date and time functions: date_add, date_trunc, from_unixtime, last_day_of_month, to_unixtime, and extracting the day, hour, minute, or second from a date or timestamp value
TopN queries

Note

Use EXPLAIN ANALYZE to see which parts of a query plan ran on a GPU, and why any remaining parts were not eligible for Torin.

Configuration#

Set the following configuration properties to enable Torin.

In the coordinator’s config.properties file:

gpu-execution=true
experimental.force-single-node-query=true

In each worker’s config.properties file:

task.gpu-execution.enabled=true

Torin configuration properties#
Property name	Description
`gpu-execution`	Allows the planner to consider queries for GPU execution. Equivalent session property: `gpu_execution_enabled`.
`experimental.force-single-node-query`	Enables single node execution mode, which maximizes GPU benefit since most of the query plan can be delegated to the GPU without moving intermediate data in and out of GPU memory.
`task.gpu-execution.enabled`	Enables GPU execution for tasks running on that node.

Caution

Setting task.gpu-execution.enabled on the coordinator causes SEP to fail at startup. GPU execution is not supported on the coordinator.

Catalog configuration#

For Hive, Iceberg, and Delta Lake catalogs, tune the following properties to control split sizes for queries that use Torin:

Torin catalog configuration properties#
Property name	Description
`hive.max-split-size`	Increases the split size for better performance. Equivalent session property: `max_split_size`.
`hive.parquet.max-split-size`	Increases the split size for Parquet files. Equivalent session property: `parquet_max_split_size`.
`iceberg.max-split-size`	Increases the split size for better performance. Equivalent session property: `max_split_size`.
`iceberg.experimental.composite-splits.enabled`	Improves performance when files are small. Equivalent session property: `experimental_composite_splits_enabled`.
`delta.max-split-size`	Increases the split size for better performance. Equivalent session property: `max_split_size`.
`delta.experimental.composite-splits.enabled`	Improves performance when files are small. Equivalent session property: `experimental_composite_splits_enabled`.

Kubernetes configuration#

For Kubernetes deployments, request a GPU for each worker by adding nvidia.com/gpu: 1 to the worker’s resources.requests and resources.limits sections in your deployment YAML:

worker:
  resources:
    requests:
      memory: "55Gi"
      cpu: "7"
      nvidia.com/gpu: 1
    limits:
      memory: "55Gi"
      nvidia.com/gpu: 1

After you deploy the Helm chart, apply the following patch, replacing <namespace> with the namespace of your SEP deployment. The worker pod repeatedly restarts until this patch is applied:

kubectl patch deployment worker -n <namespace> --type=json -p='[{"op":"add","path":"/spec/template/spec/containers/0/resources/limits/nvidia.com~1gpu","value":"1"}]'

Note

This patch modifies the deployment directly and is not tracked by Helm. A subsequent helm upgrade can reset the deployment and remove the patch, requiring you to reapply it.

On AWS, the nvidia-device-plugin-daemonset attempts to run on all worker nodes by default, even those without a GPU. Restrict it to only the GPU-enabled node group used by SEP workers, replacing <your-node-group-name> with your managed node group name:

kubectl patch daemonset nvidia-device-plugin-daemonset -n kube-system --type merge -p '{"spec":{"template":{"spec":{"nodeSelector":{"eks.amazonaws.com/nodegroup":"<your-node-group-name>"}}}}}'

Monitoring#

GPU memory usage is available through JMX as GpuDeviceStats on each SEP worker.

Torin JMX attributes#
Attribute	Layer	Description
`DeviceTotalBytes`	Physical	Total physical GPU memory capacity. Constant for the life of the worker process.
`DeviceUsedBytes`	Physical	Physical GPU memory currently in use, including CUDA context overhead not tracked by RMM.
`DeviceUsedDistribution`	Physical	Histogram of `DeviceUsedBytes`, sampled at the interval configured by `gpu.device-stats.sampling-interval` (default `5s`).
`RmmAllocatedBytes`	Logical	Bytes currently allocated by SEP’s GPU operators through RMM, not including CUDA context overhead.
`RmmAllocatedDistribution`	Logical	Histogram of `RmmAllocatedBytes`, sampled at the same interval.
`RmmPeakAllocatedBytes`	Logical	High-water mark of RMM-allocated memory since the worker started.

Limitations#

Queries against Starburst Warp Speed catalogs don’t benefit from Torin, regardless of the underlying source connector.

The following operations are not eligible to run on GPUs, and run on CPUs instead:

Any non-SELECT portion of a query
Unsupported SQL scalar functions
Unsupported aggregation and window functions
avg window functions with a frame other than the default (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)
Custom user-defined functions (UDFs)
Table scans other than Iceberg, Hive, and Delta Lake tables using the Parquet file format
Data exchanges between workers
DISTINCT clauses in SELECT clauses
Unsupported data types, including ROW, MAP, ARRAY, and JSON