Autoscaling Elasticsearch for Logs with a Kubernetes Operator
Summary
In this post, we’ll show you how to use a Kubernetes Operator to autoscale Elasticsearch clusters, going through the following with just a few commands: • Set up an initial index and an alias pointing to it, ready to handle writes. As the Elasticsearch cluster scales up or down, the operator will also maintain the configuration initially uploaded, such as the number of shards per index. This is what our next RBAC-focused YAML file will do, and we’re going to apply those permissions to a separate namespace called demo: Before we deploy the operator itself, we need to define the configuration it’s supposed to work on. In this case, we’ll define things like – when nodes reach X% disk space, we scale up the cluster: Now we’re ready to deploy the operator. Then, re-apply the configuration: If you’re still tailing the logs of the es-operator pod, you should see exactly what happens: • notices the actual disk usage is higher than the threshold Updating desired scaling for EDS demo/es-data1.
