Enable autoscaling in SEP clusters #
Enabling cluster autoscaling in Starburst Enterprise platform (SEP) requires you to deploy and configure a cluster autoscaler and to install a metrics server to report on worker CPU utilization. This document assumes that a cluster autoscaler is currently deployed for your cluster:
Configure the cluster for autoscaling #
To prevent scale downs that interrupt in-flight query processing, you must set
the safe-to-evict
annotation for the cluster:
$ kubectl -n kube-system annotate deployment.apps/cluster-autoscaler
cluster-autoscaler.kubernetes.io/safe-to-evict="false"
The autoscaler manifest must be modified to include the following four lines in
the spec.containers.command
YAML node with appropriate indentation:
- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/<CLUSTER_NAME>
- --balance-similar-node-groups
- --skip-nodes-with-system-pods=false
- --nodes=min:max:<ASG_NAME>
To edit the manifest to include these lines, use the following command:
$ kubectl -n kube-system edit deployment.apps/cluster-autoscaler
Set the cluster-autoscaler
version. The specified
version must be the major version of your Kubernetes cluster. In the following
example, the version is set to v1.18.3
:
$ kubectl -n kube-system set image deployment.apps/cluster-autoscaler \
cluster-autoscaler=us.gcr.io/k8s-artifacts-prod/autoscaling/cluster-autoscaler:v1.18.3
See more node configurations for SEP in the worker properties section.
Deploy the metrics server #
Deploy the metrics server to report CPU utilization using the following command:
$ kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.6/components.yaml
Verify that the metrics server is deployed and is running:
$ kubectl get deployment metrics-server -n kube-system
Is the information on this page helpful?
Yes
No
Is the information on this page helpful?
Yes
No