This is part five of our on-going series exploring tools and practices that make it easier to manage and operate cloud native environments. Part 1 and 2 reviewed cloud native development and CICD tools, part 3 outlined cloud native network tools and part 4 evaluated service mesh tools. We reviewed tools in each category based on the feature set they provide and their ease of deployment and continued operations in Kubernetes based cloud native environments.
In this installment we will outline and evaluate cloud native storage tools. We will start off with a quick look at cloud native storage tools and why they are needed before moving on to a comparison of cloud native storage tools from the CNCF cloud native landscape.
Cloud native storage is storage technology geared towards use in cloud native environments. Cloud native environments are dynamic by nature and represent the convergence of multiple trends in the broader IT landscape including orchestration, containerisation, microservices, service meshes, DevOps and CICD. As such storage needs to evolve too in order to keep pace with these technologies.
Cloud native storage platforms enable comprehensive data management for stateful applications and provide solutions to the problem of persistent data storage in Kubernetes based cloud native environments.
Most cloud native storage solutions emulate the characteristics of cloud native tools and environments outlined by the CNCF. Some of these characteristics include, scalability, high availability, vendor-neutrality and being natively secure, resilient, manageable, observable, declarative in nature and API driven.
Let's now move on to a comparison of the top most rated cloud native storage solutions on the CNCF cloud native landscape.
OpenEbs is an opensource cloud native storage solution with the fourth highest rating on the CNCF landscape. It has a broad integration footprint with Kubernetes and can be easily installed and configured using Helm or Kubectl.
OpenEbs creates a software-defined storage infrastructure for Kubernetes applications and acts as an abstraction layer between those applications and the disparate underlying storage providers. In the process it reduces maintenance overhead, cuts costs and makes it easier to manage storage for stateful applications.
OpenEbs uses the container attached storage (CAS) model to abstract storage controllers as Kubernetes pods. Each storage volume has a dedicated pod as well as a set of replica pods. This allows storage volumes to be orchestrated and deployed similar to any other container or microservice.
Since CAS containerises storage and abstracts storage disks and pools as Kubernetes CRDs, it can be easily integrated with other cloud native solutions for management, provisioning and monitoring. This also allows for ease of orchestration and deployment using Kubernetes across cloud, bare metal and on-premises.
Once deployed, the OpenEbs architecture comprises a control plane, a data plane and a node disk manager (NDM). The control plane handles volume provisioning, snapshots, replication, cloning, storage policies and metric export. The data plane implements the IO path using multiple pluggable storage engines including Jiva and cStor.
The NDM component makes it easier to manage node-attached disks by detecting and loading them as Kubernetes custom resources called BlockDevice objects. It then provides an easy to access inventory of block devices, predicts block device failures, and allows for attaching/detaching block devices to pods without the need for restarts.
OpenEbs supports synchronous replication of data volumes across availability zones for high availability. This feature is especially useful when building highly available stateful applications that use public cloud provider local disks.
It also allows DevOps to create instantaneous snapshots and manage them using native Kubectl commands in the process enabling workload portability and data migration. OpenEbs also integrates with Velero using a native velero plugin to enable volume backup and restore.
Another useful feature is the support for creating, updating and managing granular storage policies. DevOps can create and manage policies for each individual volume including e.g. controlling the amount of volume replicas (ReplicaCount), and automatically launching a side-car to export Prometheus metrics (VolumeMonitor).
Rook is another open source cloud native storage solution with the third highest rating on the CNCF landscape. Rook is a storage orchestrator, allowing DevOps to offload the management of multiple storage backends to it. It can be easily installed in Kubernetes environments using the dedicated Kubernetes operator for each supported storage provider.
Rook makes it easier to manage distributed storage systems by automating deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management. Out of the box support for multiple storage backends including Ceph, EdgeFs and Cassondra ensures DevOps can pick and choose storage technologies based on their specific use case without having to worry about how well they integrate and run on Kubernetes.
Installing and operating these storage backends on Kubernetes is also simpler with Rook. Ceph, a distributed storage solution for block and object storage, and shared file systems is one good example.
Ceph can be installed on Kubernetes using kubectl or a dedicated operator. Once deployed DevOps can mount block and object storage or shared file systems for their applications using native Kubernetes primitives. For example block storage can be provisioned using a StorageClass and CephBlockPool (K8s CRD) definitions and consumed using the Ceph-CSI driver that automatically mounts storage to pods.
In addition the Rook operator manages CRDs for storage pools, object storage and filesystems. It also spins up and monitors Ceph monitor pods, responsible for maintaining a copy of the cluster map. Since Rook spins up monitor pods as part of ReplicaSets, they are always restarted by Kubernetes in case of failure. Rook can also terminate a monitor pod that fails to restart and replace it with a new one.
Rook also provides a dedicated dashboard for Ceph clusters where DevOps can monitor cluster health and status of cluster resources. Ceph clusters provisioned using Rook also support monitoring via Prometheus and Grafana. Once installed DevOps can monitor ceph cluster metrics, create alerts and graph them using Grafana or use one of the prebuilt Grafana dashboards.
Longhorn is a lightweight, easy-to-to-use block storage system for Kubernetes open sourced by Rancher. It can be easily deployed in Kubernetes clusters using a single command.
Longhorn works by creating dedicated storage controllers for each block device volume. These storage controllers then create a resilient block storage system by pooling together resources from shared local disks or network storage.
Spinning up individual storage controllers for each storage volume, radically simplifies controller design and allows controllers themselves to be scheduled and scaled using Kubernetes. Since individual controllers manage single volumes, it also helps contain controller failures to only that specific volume and allows seamless upgrades of controllers without impacting the normal operation of the storage system.
In Kubernetes environments, Longhorn makes it easier to deploy highly available persistent block storage for pods and containers. It does this using either the Longhorn Container Storage Interface (CSI) driver or the Longhorn Flexvolume drivers. Devops can specify the required driver in the deployment yaml file, which is then automatically deployed by Longhorn.
Once deployed they can then use Kubernetes StorageClass to provision storage volumes by specifying the size of the volume, IOPS requirements and the number of required replicas.
Longhorn also provides a broad feature set including snapshots, backup, restore and health monitoring and repair. Once provisioned, Longhorn monitors the health of each provisioned volume replica and repairs it whenever required. DevOps can also attach multiple storage frointends for each volume including a Linux kernel device or an iSCSI target
It also supports the creation of instant volume snapshots. Snapshots represent volume state at a specific time and are stored in the same location as the volume data. Snapshots can be restored using the Longhorn UI or backed up to NFS or S3 compatible secondary storage outside of the Longhorn system. The UI also provides an overview of the system and allows Kubernetes admins to monitor volume and backup/restore operations.
DevOps can also configure recurring snapshots/backups by specifying the time and the number of required snapshots/backups. Once configured, Longhorn will automatically create snapshots and/or backups for the specified volumes.
Want to dig deeper into Kubernetes based cloud native environments? Download the Complete CIOs Guide to Kubernetes:
Fan of all things cloud, containers and micro-services!
Part 4 of our Introduction to FinOps for Kubernetes: Challenges and Best Practices article series, which outlines a comprehensive list of best practices aimed at implementing FinOps processes for cloud native Kubernetes environments.
August 26, 2021
7 min read
In a recent report, CNCF identified "a more granular and active Kubernetes cost-monitoring strategy" as a primary means of reducing K8s cost. In this article we identify major takeaways from the report and outline the contours of a comprehensive Kubernetes cost monitoring strategy.
August 12, 2021
7 min read
Part 3 of our Introduction to FinOps for Kubernetes: Challenges and Best Practices article series, which outlines a comprehensive list of best practices aimed at implementing FinOps processes for cloud native Kubernetes environments.
July 12, 2021
7 min read