Kubernetes CRDs & Operators: Extending the API, Episode 3
What Is a CRD? · CRDs You Already Use · CRD Anatomy · Write Your First CRD · CEL Validation · Controller Loop · Build an Operator · CRD Versioning · Admission Webhooks · CRDs in Production
TL;DR
- The Kubernetes CRD schema is defined in
spec.versions[].schema.openAPIV3Schema— the API server uses it to validate every custom resource create and update before storing in etcd
(OpenAPI v3 schema = a JSON Schema dialect that describes the structure, types, and constraints of your resource’s fields) spec.versionsis a list — CRDs can serve multiple API versions simultaneously; exactly one version must havestorage: truescope: Namespacedvsscope: Clustercontrols whether custom resources live inside a namespace or at cluster level (likePersistentVolumevsPersistentVolumeClaim)spec.namesdefines the plural, singular, kind, and optional shortNames used inkubectland RBAC- The status subresource (
subresources.status: {}) separates user writes (spec) from controller writes (status) — enabling optimistic concurrency andkubectl statussupport - The scale subresource (
subresources.scale) makes your custom resource compatible withkubectl scaleand the HorizontalPodAutoscaler
The Big Picture
ANATOMY OF A CUSTOMRESOURCEDEFINITION
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: {plural}.{group} ← MUST be exactly this format
spec:
group: {group} ← API group (e.g. storage.example.com)
scope: Namespaced | Cluster ← where instances live
names: ← how kubectl refers to this resource
plural: backuppolicies
singular: backuppolicy
kind: BackupPolicy
shortNames: [bp]
versions: ← can be a list; one must have storage: true
- name: v1alpha1
served: true ← API server responds to this version
storage: true ← etcd stores objects in this version
schema:
openAPIV3Schema: ← validation schema for ALL objects of this type
type: object
properties:
spec: {...}
status: {...}
subresources:
status: {} ← enables separate status write path
scale: ← enables kubectl scale + HPA
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
additionalPrinterColumns: ← extra columns in kubectl get output
- name: Schedule
type: string
jsonPath: .spec.schedule
Understanding the Kubernetes CRD schema is the prerequisite for writing a CRD that behaves correctly in production — validation catches bad data at the API boundary, the status subresource prevents controller race conditions, and scope determines your entire RBAC and multi-tenancy model.
spec.group and metadata.name
The group is a reverse-DNS identifier for your API. Convention:
storage.example.com ← domain you control + functional area
monitoring.myteam.io
databases.platform.company.com
The CRD’s metadata.name must be exactly {plural}.{group}:
metadata:
name: backuppolicies.storage.example.com
spec:
group: storage.example.com
names:
plural: backuppolicies
If these do not match, the API server rejects the CRD with a validation error. This is the most common first-timer mistake.
spec.scope: Namespaced vs Cluster
SCOPE DETERMINES WHERE INSTANCES LIVE
Namespaced (scope: Namespaced) Cluster (scope: Cluster)
───────────────────────────── ──────────────────────────
kubectl get backuppolicies -n prod kubectl get clusterbackuppolicies
kubectl get backuppolicies -A (no -n flag, no namespace)
Analogous to: Pod, Deployment, Analogous to: PersistentVolume,
ConfigMap ClusterRole, Node
Namespaced: Use when instances are per-tenant or per-application. Users with namespace-scoped RBAC can manage their own instances without cluster-admin. Most CRDs should be namespaced.
Cluster-scoped: Use when instances represent cluster-wide configuration — a ClusterIssuer (cert-manager), ClusterSecretStore (ESO), a StorageClass-like concept. Requires cluster-level RBAC to create/modify.
You cannot change scope after a CRD is created without deleting and recreating it (which deletes all instances). Choose carefully.
spec.versions: Serving Multiple API Versions
spec:
versions:
- name: v1alpha1
served: true
storage: false # not stored; converted on read
schema:
openAPIV3Schema: {...}
- name: v1beta1
served: true
storage: false
schema:
openAPIV3Schema: {...}
- name: v1
served: true
storage: true # etcd stores in this version
schema:
openAPIV3Schema: {...}
Rules:
– served: true means the API server accepts requests at this version
– served: false means the API server returns 404 for that version — use to deprecate
– Exactly one version must have storage: true — this is what gets written to etcd
– When a client requests a non-storage version, the API server converts on the fly (or calls your conversion webhook — see EP08)
Early in development, start with v1alpha1 storage: true. Promote to v1 when the schema is stable. EP08 covers how to do this without losing data.
spec.names: What kubectl Sees
spec:
names:
plural: backuppolicies # kubectl get backuppolicies
singular: backuppolicy # kubectl get backuppolicy (also works)
kind: BackupPolicy # used in YAML apiVersion/kind
listKind: BackupPolicyList # optional; auto-derived if omitted
shortNames: # kubectl get bp
- bp
categories: # kubectl get all includes this type
- all
categories is worth noting: if you add all to categories, your custom resources appear when someone runs kubectl get all -n mynamespace. Most CRDs deliberately do not add this — it clutters get all output. Only add it if your resource is a primary operational concern.
schema.openAPIV3Schema: Validation
The schema is where you define field types, required fields, constraints, and descriptions. The API server validates every create and update against this schema before writing to etcd.
schema:
openAPIV3Schema:
type: object
required: ["spec"]
properties:
spec:
type: object
required: ["schedule", "retentionDays"]
properties:
schedule:
type: string
description: "Cron expression for backup schedule"
pattern: '^(\*|[0-9,\-\/]+)\s+(\*|[0-9,\-\/]+)\s+(\*|[0-9,\-\/]+)\s+(\*|[0-9,\-\/]+)\s+(\*|[0-9,\-\/]+)$'
retentionDays:
type: integer
minimum: 1
maximum: 365
storageClass:
type: string
default: "standard" # default value (Kubernetes 1.17+)
targets:
type: array
maxItems: 10
items:
type: object
required: ["name"]
properties:
name:
type: string
namespace:
type: string
default: "default"
status:
type: object
x-kubernetes-preserve-unknown-fields: true # controllers write arbitrary status
Field types available
| Type | Usage |
|---|---|
string |
Text values; supports format, pattern, enum, minLength, maxLength |
integer |
Whole numbers; supports minimum, maximum |
number |
Floating point |
boolean |
true/false |
object |
Nested structure; use properties to define fields |
array |
List; use items to define element schema; supports minItems, maxItems |
x-kubernetes-preserve-unknown-fields: true
This tells the API server not to prune fields it does not know about. Use it on status (controllers write whatever they need) and on fields that are intentionally free-form (like a config field that accepts arbitrary YAML). Avoid it on spec — it bypasses validation.
Validation behavior in practice
# This will fail with a clear error:
kubectl apply -f - <<EOF
apiVersion: storage.example.com/v1alpha1
kind: BackupPolicy
metadata:
name: bad
namespace: default
spec:
schedule: "not-a-cron" # fails pattern validation
retentionDays: 500 # fails maximum: 365
EOF
The BackupPolicy "bad" is invalid:
spec.schedule: Invalid value: "not-a-cron": spec.schedule in body should match
'^(\*|[0-9,\-\/]+)\s+...'
spec.retentionDays: Invalid value: 500: spec.retentionDays in body should be
less than or equal to 365
Schema validation catches configuration mistakes at apply time, not at runtime inside a pod. This is one of the core advantages of expressing domain configuration as CRDs rather than ConfigMaps.
additionalPrinterColumns: What kubectl get Shows
By default, kubectl get backuppolicies shows only NAME and AGE. You can add columns:
additionalPrinterColumns:
- name: Schedule
type: string
jsonPath: .spec.schedule
description: Cron schedule for backups
- name: Retention
type: integer
jsonPath: .spec.retentionDays
priority: 1 # 0 = always shown; 1 = only with -o wide
- name: Ready
type: string
jsonPath: .status.conditions[?(@.type=='Ready')].status
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
Result:
NAME SCHEDULE READY AGE
nightly 0 2 * * * True 3d
weekly 0 0 * * 0 False 7d
Good printer columns turn kubectl get into a useful operational dashboard. Include Ready (from status conditions) so operators can immediately see which custom resources are healthy without running kubectl describe.
The Status Subresource
subresources:
status: {}
Without the status subresource, spec and status are part of the same object. Any user with update permission on the CRD can modify both. Controllers write status through the same path as users write spec.
With the status subresource enabled:
– kubectl apply / kubectl patch only update spec — the status block is stripped
– Controllers use the /status subresource endpoint to write status
– RBAC can grant update on backuppolicies (spec) independently from update on backuppolicies/status
WITHOUT status subresource: WITH status subresource:
───────────────────────── ──────────────────────────
PUT /backuppolicies/nightly PUT /backuppolicies/nightly
→ updates spec AND status → updates spec only
PUT /backuppolicies/nightly/status
→ updates status only (controller path)
Always enable the status subresource on production CRDs. The split between spec and status is fundamental to the Kubernetes API contract. Without it, a controller updating status can accidentally overwrite spec changes made by a user at the same time.
The Scale Subresource
subresources:
scale:
specReplicasPath: .spec.replicas
statusReplicasPath: .status.replicas
labelSelectorPath: .status.labelSelector
This makes your custom resource compatible with:
kubectl scale backuppolicy nightly --replicas=3
And with HorizontalPodAutoscaler targeting your custom resource. If your CRD manages something replica-based (workers, shards, connections), enabling the scale subresource lets it plug into the standard Kubernetes autoscaling ecosystem without extra plumbing.
⚠ Common Mistakes
Forgetting x-kubernetes-preserve-unknown-fields: true on status. If you validate the status field with a strict schema but do not add this, the API server will prune any status fields the controller writes that are not in the schema. The controller’s status updates will silently lose fields. Either define the full status schema or use x-kubernetes-preserve-unknown-fields: true.
Using scope: Cluster for resources that should be namespaced. Once a CRD is created as cluster-scoped, you cannot make it namespaced without deleting and recreating it. Plan scope before deploying to production.
Not enabling the status subresource. Without it, controllers writing status can race with users updating spec. It also means kubectl patch --subresource=status does not work and some tooling behaves unexpectedly. Enable it from the start.
Loose schema with no required fields. An openAPIV3Schema with no required constraint accepts objects with empty spec. This usually means your controller gets called with a resource that is missing mandatory configuration. Define required fields and validate them at the API boundary, not inside the controller.
Quick Reference
# Inspect the full schema of a CRD
kubectl get crd backuppolicies.storage.example.com -o yaml | \
yq '.spec.versions[0].schema'
# Check what subresources are enabled
kubectl get crd certificates.cert-manager.io -o jsonpath=\
'{.spec.versions[0].subresources}'
# See all served versions for a CRD
kubectl get crd prometheuses.monitoring.coreos.com \
-o jsonpath='{.spec.versions[*].name}'
# Check which version is the storage version
kubectl get crd certificates.cert-manager.io \
-o jsonpath='{.spec.versions[?(@.storage==true)].name}'
# Describe the printer columns for a CRD
kubectl get crd scaledobjects.keda.sh \
-o jsonpath='{.spec.versions[0].additionalPrinterColumns}'
Key Takeaways
spec.versionsallows serving and storing multiple API versions; only one version hasstorage: truescope(Namespaced vs Cluster) cannot be changed after creation — choose deliberatelyopenAPIV3Schemavalidates every CR at the API boundary, before etcd storage- The status subresource separates the user write path (spec) from the controller write path (status) — always enable it
additionalPrinterColumnsmakeskubectl getoperationally useful; include aReadycolumn from status conditions
What’s Next
EP04: Write Your First Kubernetes CRD puts the anatomy into practice — a complete hands-on walkthrough building a BackupPolicy CRD from scratch, applying it to a cluster, creating instances, and verifying validation, RBAC, and status behavior.
Get EP04 in your inbox when it publishes → subscribe at linuxcent.com
