Starburst Documentation
Starburst Galaxy
  •  Get started
  •  Working with data
  •  Data engineering
  •  Developer tools
  •  Cluster administration
  •  Security and compliance
  •  Troubleshooting
  • Galaxy status
  •  Reference

    • starburst galaxy > cluster administration > Rate limiting

    Rate limiting #

    This page discusses:
    Rate limit errors
    What are the limits?
    What to do

    Rate limit errors #

    Like all cloud applications, Starburst Galaxy implements rate limiting to make sure all Galaxy customers have fair access to cloud computing resources.

    If your query client shows you the full returned text of error messages, and you receive an error that includes one of the following errors, then your query is affected by the rate limit for the target cluster on your account:

    • error 429
    • statusCode=429

    Rate limit error messages always include 429 but may also reference error 1015.

    A query that pauses or stops and starts can occur for many reasons other than rate limiting. Only receipt of these errors is a reliable indicator of hitting the rate limit

    What are the limits? #

    When Starburst Galaxy receives a query, it first checks to confirm that the customer account is active, and that the logged-in user’s role has the necessary access permissions. If so, the query is either executed immediately or queued if there is a backlog of queries awaiting execution on the target cluster.

    The rate limit for Starburst Galaxy is:

    • 150 query submissions per second

    This limit applies independently to each cluster in your account. This limit is subject to change. If you consistently receive rate limiting responses, we encourage you to reach out to Starburst Support to discuss options.

    What to do #

    If your queries are hitting a rate limit, try these mitigation practices:

    • Wait for a few minutes, then resubmit the same query. The rate limit is assessed as an average over successive ten-second periods.

    • If you have multiple queries running in more than one tab of the Galaxy query editor or other client, close down some of your queries and re-run the most important ones first.

    • If your site has many analysts submitting queries at the same time, ask for a temporary pause in your workgroup’s submissions until the queue drains.

    • Galaxy clusters support a maximum of 40 concurrent queries. We recommended staying at or below this concurrency limit for a single cluster. After 40 concurrent queries, queries begin to queue.

    • Add an additional cluster if you need to exceed the 40 concurrent queries limit.

    • Limit Galaxy cluster connection pools to 40. Above this limit, queries begin to queue.

    • If you have one or more sessions of the Trino web UI running, stop those until the queue slows down. The Trino UI can submit many small queries in order to monitor one complex query.

    • If your query client is dbt, run your query temporarily with another client. dbt can also submit many small queries to keep track of one complex query.