EKS Other Storage

Before we wrap up storage on EKS, let’s explore additional AWS-managed file and object stores beyond Amazon EBS and Amazon EFS. AWS offers FSx for Lustre, S3 (via API or CSI), and the node’s local instance storage. While powerful, these may require manual provisioning or mounting rather than automatic CSI-based dynamic provisioning.

1. Amazon FSx for Lustre

Amazon FSx for Lustre delivers a high-performance, POSIX-compliant file system ideal for workloads demanding massive throughput and low latency. You must create the file system in AWS and then mount it into your pods by defining a PersistentVolume and PersistentVolumeClaim.

Example PV and PVC

apiVersion: v1
kind: PersistentVolume
metadata:
  name: fsx-lustre-pv
spec:
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  csi:
    driver: fsx.csi.aws.com
    volumeHandle: fs-0123456789abcdef0
    volumeAttributes:
      dnsname: fs-0123456789abcdef0.fsx.us-west-2.amazonaws.com
  persistentVolumeReclaimPolicy: Retain
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: fsx-lustre-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Ti
  volumeName: fsx-lustre-pv

2. Amazon S3

Amazon S3 is an object store accessed via its API. Typically, applications use AWS SDKs or CLI to interact with buckets. AWS also provides the aws-s3-csi-driver that exposes S3 buckets as file systems.

Mounting S3 via CSI

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: s3-sc
provisioner: s3.csi.aws.com
reclaimPolicy: Retain
parameters:
  bucket: my-eks-bucket
  region: us-west-2
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: s3-pvc
spec:
  storageClassName: s3-sc
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

Keep in mind the following filesystem limitations:

• S3 is not POSIX-compliant; the CSI driver maps object APIs to file operations.
• Renaming an object involves a copy-and-delete under the hood:

• “Directories” are key prefixes; empty folders don’t exist unless objects are present.

3. Local Instance Storage

Each EC2 node includes either HDD or NVMe instance storage. It delivers blazing-fast I/O but is ephemeral—data is lost when the instance is stopped or terminated.

• No native CSI driver exists for instance store.
• You must manually manage mount points and node-level configuration.
• For persistence, attach an EBS volume or snapshot local data before shutdown:

Lifecycle Management

• Backup and restore logic must be custom-built (e.g., cronJobs, AWS DataSync).
• Consider combining with EBS for snapshots and durability.

4. In-Cluster Storage Solutions

Running storage natively within Kubernetes eliminates external dependencies like IAM, VPC routing, and separate APIs. Popular CNCF-backed options include Ceph, Longhorn, OpenEBS, and Rook. These platforms deploy operators and CRDs to manage storage across nodes.

Key benefits:

• Storage provisioning via Kubernetes APIs (no external calls).
• Built-in replication, healing, and snapshots managed by operators.
• Cloud-agnostic—support on any Kubernetes cluster, on-prem or in another cloud.

Comparative Overview

Conclusion

Your choice hinges on throughput requirements, data durability, and operational overhead. Whether you use FSx for Lustre’s HPC file system, object-based S3, node-local volumes, or a self-contained in-cluster solution, plan for backups, snapshots, and access controls to keep your data safe and available.