Kubernetes controllers help automate container orchestration by implementing control loops that monitor a cluster state, compare it with the desired state defined in a configuration manifest, and apply changes (if necessary) to achieve that desired state. A Kubernetes operator extends the controller pattern by configuring, deploying, and orchestrating instances of stateful applications on behalf of a human actor. In addition to utilizing the operating pattern of controllers and resources, operators include domain-specific knowledge about managing the application lifecycle.
In this article, we’ll discuss how operators manage workloads in Kubernetes, the types of operators, typical use cases, and how operators can be deployed in a cluster.
What are Kubernetes operators?
Someone who’s managing cluster operations has an in-depth understanding of a specific domain, including how to deploy a service, the cluster’s normal behavior, and how to troubleshoot issues. The Kubernetes operator pattern was developed to automate intricate cluster operations that require iterative, manual efforts.
Kubernetes operators are built to automate scaling, upgrading, and configuration management for stateful applications. Stateful workloads such as monitoring systems and databases require domain-specific knowledge that isn’t natively offered by Kubernetes. To help with this, operators encode the domain knowledge into application extensions, which makes the management of stateful applications standardized, scalable, and repeatable.
How operators manage applications in Kubernetes
Operators in Kubernetes are custom controllers that rely on custom resources (CRs) to manage services and cluster components. Similar to controllers, operators use control loops to compare a cluster state with the desired state, and then to implement a state change if there’s a difference. High-level cluster configurations and directives are specified within the operator’s custom resource definition (CRD), where the operator translates these directives into cluster actions based on Kubernetes automation principles embedded within the operator’s logic.
Kubernetes operator use cases
Kubernetes operators eliminate tedious stateful application management tasks, simplifying the deployment and orchestration of crucial cluster services. In a typical Kubernetes ecosystem, operators can be used for implementing both Day 1 (installation and configuration) and Day 2 (maintenance, management and optimization) tasks. Uses of operators in Kubernetes clusters include:
Deploying apps on-demand
As operators encapsulate and package stateful applications for reuse, they can be deployed on demand across different stages of cluster operations. Operators also automate various post-provisioning tasks such as internal configuration, ingress and egress communication configuration, and capacity scaling.
Handling seamless code upgrades
Operators can be deployed to detect code changes in database schemas and application repositories, and to replicate the changes across containerized cluster workloads. Operators are also considered crucial in a continuous delivery framework that leverages declarative configuration upgrades to automate the implementation of changes to cluster configuration.
Enabling service discovery for unsupported Kubernetes APIs
Operator custom resources can be used to set up service-to-service communication by provisioning a pluggable policy layer. With operators, cluster administrators can configure APIs to implement access controls for seamless networking and service discovery of non-Kubernetes-native applications.
Performing backups and restoration
Cluster administrators can create CRs for volume snapshots to enable seamless backups and restoration of a cluster. This is typically achieved by configuring an operator to create a backup file and store it in a location configured within a Backup resource. A custom resource for restoration (Restore) can further be created with the backup file as a specification to automate service and data restoration during outages.
Hybrid environment management
When running Kubernetes clusters on multi-cloud instances, cluster administrators can leverage Kubernetes operators to simplify management and configuration tasks without acquiring platform expertise. In hybrid cloud clusters, operators are crucial components for complex cluster management, offering agility, reliability, and consistency while adhering to each platform’s best practices.
Using operators in a Kubernetes ecosystem
Operators in Kubernetes help with the efficient deployment and management of stateful applications, typically on behalf of a cluster administrator or site reliability engineer. An operator extends the Kubernetes API without requiring modifications to the Kubernetes code base. By bridging the gap between Kubernetes automation capabilities and external applications, operators ensure that all aspects of a software’s lifecycle are integrated into the cluster and invoked as needed.
Operator deployment options
There are two primary approaches to deploying operators in a Kubernetes cluster:
Third-party operators
These are pre-built solutions that extend the Kubernetes API for managing third-party applications as native objects of a Kubernetes cluster. Third-party operators can be installed with standard, pre-configured settings or can be modified by cluster administrators to suit specific use-cases. OperatorHub (project by Red Hat) and Artifact Hub (project by CNCF) are two community-driven public repositories that allow operators to be discovered, downloaded, or uploaded along with their base documentation and information on vendor support.
Custom operators
Cluster management teams can also build their own operators for use cases that don’t have a third-party operator available. Building custom operators is a complex undertaking, as it requires expert programming skills coupled with a deep knowledge of Kubernetes controllers and reconciliation loops. While these custom controllers can be implemented in any language that enables server-client communications, a few platforms are most commonly used:
- Go is the most preferred language as it’s also used for the Kubernetes code base.
- Helm Package Manager is used to develop custom operators for basic installation and upgrade capabilities.
- Ansible is used to build controllers for comprehensive automation and lifecycle management.
- The Kubernetes API aggregation layer is leveraged to build custom APIs that extend the control plane functionality beyond what is offered by core Kubernetes APIs.
6 popular Kubernetes operators
Prometheus
Prometheus is a widely adopted cloud-native monitoring framework. Similar to any other Kubernetes deployment object, the Prometheus operator enables the deployment and management of Prometheus monitoring services in a Kubernetes cluster. The operator comes preconfigured with some fundamental Prometheus components and services, such as:
- AlertManager
- Grafana
- Node exporter
- Versions
- Retention policies
- Metric data persistence
- Deployment replicas
The Prometheus operator also auto-generates Prometheus target configurations based on Kubernetes queries, eliminating the need to learn Prometheus Query Language (PromQL).
Grafana
Similar to the Prometheus operator, the Grafana operator simplifies the deployment, configuration, and management of Grafana instances in Kubernetes clusters. When compared with a standalone Grafana instance, the Grafana operator offers fine-grained control over managing Grafana service through Kubernetes resources such as ConfigMaps and CRDs.
The operator also supports advanced features for Grafana instances, including:
- Data source recovery
- Dashboard creation and configuration through YAML
- Ingress automation
- Provisioning notification channel
RBAC manager
Kubernetes relies on role-based access control (RBAC) to administer roles that are bound to containers for permission management. The RBAC manager operator uses the RBACDefinition custom resource to reduce operational overhead for managing role binding and service accounts. The operator uses declarative configuration of RBAC to eliminate errors in role management changes and simplifies scaling of complex clusters.
Elastic cloud on Kubernetes operator
The Elastic Kubernetes Operator is the official open-source operator for Elasticsearch, Logstash, and Kibana (also known as ELK or Elastic Stack). The Elastic Kubernetes Operator helps to automatically deploy and manage the Elastic Stack on Kubernetes clusters for the collection, formating, search, visualization, and analysis of data from any source. The operator is preconfigured to offer a number of advanced features, such as:
- Scheduled backups
- Upgrades
- Monitoring
- Dynamic scaling of storage
- Scaling cluster capacity
AWS controllers for Kubernetes (ACK)
ACK is a powerful tool that automates the management of AWS services directly using the Kubernetes API. The tool packages controllers for each AWS service in a separate container image within a publicly available repository. Each image corresponds to a single controller, making it easy to invoke supporting services such as message queues and databases. With the ACK operator, cluster administrators can take advantage of AWS features to deploy and configure resources for Kubernetes workloads without having to initiate the services outside the cluster.
Istio
Istio is a popular service mesh platform that automates cluster network functions in a flexible, transparent, and language-independent way. The Istio operator manages the installation, management, and upgrading of the Istio service mesh in a Kubernetes cluster, eliminating the manual overhead of using a Helm or Istioctl installation. The operator relies on the IstioOperator resource to detect configuration changes and updates the Istio installation based on the desired state of profile values.
Deploying an operator in Kubernetes
There are multiple ways to deploy operators into Kubernetes clusters. While most third-party solution vendors provide command line tools that expedite the installation of operators, experienced teams can also deploy operators manually using Helm templates to package CRDs and CRs for the installation and configuration of operators.
Although manual deployment of operators through Helm charts may sound tricky, the steps for doing so are typically straightforward. Here, we outline the steps for installing a Prometheus operator using Helm charts:
Prerequisites
- An existing Kubernetes cluster with Helm installed
- Helm CLI installed on your local machine
As the first step, download Prometheus templates into your operating cluster’s control plane by running the command:
Which returns the response:
To ensure that the repository is using the updated version, update the repository using the command:
Which returns an output similar to:
Install Prometheus into the cluster using the command:
Which returns the following output upon successful installation:
Check the status of the Prometheus installation by running the command:
If the installation is successful, the output for the above command displays pods running various Prometheus services, similar to:
Quick note: The above steps install various preconfigured components of Prometheus to help with basic monitoring of the operating cluster. These components include:
- Prometheus operator
- Alertmanager
- Custom Resources
- Node exporters
- Grafana
Final thoughts
Kubernetes allows you to automate intricate cluster tasks that are required beyond the native automation capabilities offered by the platform. Kubernetes operators extend Kubernetes automation capabilities by packaging, configuring, and managing stateful applications using custom resources. The operator pattern builds upon Kubernetes controller concepts by introducing bespoke object types whose definitions are handled by the Kubernetes API.
While automation is one of the key benefits of using operators in a Kubernetes ecosystem, operators also simplify application management and prevent human errors. However, keep in mind that it’s important to choose the correct operator for your specific use case.
If you're looking for a serverless, maintenance-free platform for automating your most critical operations, then you should consider using Airplane. Airplane is the developer platform for building custom internal tools. With Airplane, you can transform scripts, queries, APIs, and more into powerful workflows and UIs. Airplane offers strong job scheduling capabilities that make it simple to automate your most important processes. Airplane also offers built-ins, such as audit logs and permissions setting, that makes it efficient to track your automated jobs.
To build your first job and automate it within seconds, sign up for a free account or book a demo.
