Autoscaling

Kubernetes Ecosystem: From User to Contributor, Episode 8
← EP07: Crossplane vs Terraform · EP08: Karpenter · EP09: Karpenter vs Cluster Autoscaler →

11 min read

TL;DR

• Karpenter node provisioning means no pre-defined node groups at all — it looks at pending pods’ actual resource requests and provisions the specific instance type and size that fits, directly
• NodePool and NodeClass are Karpenter’s two core CRDs: NodePool declares provisioning constraints and instance-type flexibility, NodeClass declares the cloud-specific details (AMI, subnets, security groups)
• Consolidation is Karpenter’s continuous bin-packing behavior — it doesn’t just scale up when pods are pending, it actively replaces underutilized nodes with better-fitting ones to reduce cost
• Karpenter handles spot interruption notices natively, draining gracefully before the two-minute warning expires, rather than relying on a separate spot-handling daemon
• Originally AWS-only, Karpenter has been donated to Kubernetes SIGs specifically to become a cross-cloud project — provider parity for GKE, AKS, and others is real, current, in-progress work
• Contribution opportunity: non-AWS provider feature parity is an explicitly open area with active upstream tracking — a genuinely current place to contribute

The Big Picture

CLUSTER AUTOSCALER MODEL                    KARPENTER MODEL
─────────────────────────                    ────────────────
Pre-defined node groups                      No node groups
(ASG A: m5.large, ASG B: m5.xlarge, ...)     Pending pod: needs 2 vCPU, 4Gi
        │                                            │
Pod pending, no capacity                     Karpenter evaluates: cheapest
        │                                    instance type that actually
Scale UP the node group                      fits, from a flexible list —
that (roughly) fits                          could be any instance family
        │                                    allowed by the NodePool
New node joins — may be                              │
oversized or undersized                      Provisions exactly that instance
for the actual pod                           — right-sized to the real
                                              pending workload

Karpenter node provisioning removes the middle abstraction layer entirely — instead of scaling a pre-sized group and hoping the group’s instance type roughly matches what’s pending, it computes the actual best-fit instance for the actual pending pods, every time.

NodePool and NodeClass: Karpenter’s Core CRDs

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      requirements:
      - key: karpenter.k8s.aws/instance-category
        operator: In
        values: ["c", "m", "r"]      # flexible across instance families
      - key: karpenter.k8s.aws/instance-generation
        operator: Gt
        values: ["4"]
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h
---
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2023
  subnetSelectorTerms:
  - tags: {karpenter.sh/discovery: my-cluster}
  securityGroupSelectorTerms:
  - tags: {karpenter.sh/discovery: my-cluster}

NodePool says “here’s the range of instance types you’re allowed to choose from, and here’s the disruption policy” — it’s about scheduling flexibility. EC2NodeClass (or the equivalent for other providers) says “here’s the actual cloud-specific detail” — AMI, subnets, security groups. Splitting these two concerns is deliberate: a platform team can offer multiple NodePools with different cost/performance trade-offs, all referencing the same underlying NodeClass.

How Karpenter Actually Picks an Instance Type

$ kubectl get nodeclaims
NAME            TYPE          ZONE         NODE               READY   AGE
general-x7k2l   c6a.xlarge    us-east-1a   ip-10-0-1-42...    True    45s

$ kubectl describe nodeclaim general-x7k2l
...
Events:
  Reason              Message
  ------              -------
  Launched            Launched instance: i-0abc123... c6a.xlarge
  #                    ^^^^^^^^^^ — chosen because it was the cheapest
  #                    instance type in the allowed range that fit
  #                    the pending pods' actual CPU/memory requests

A NodeClaim is the record of one provisioning decision — it shows exactly which instance type Karpenter chose and why, unlike a node-group scale-up event, which just tells you the group’s already-fixed instance type was used again regardless of fit.

Consolidation: Karpenter’s Continuous Bin-Packing

# Karpenter continuously evaluates whether existing nodes could be
# consolidated into fewer, better-utilized nodes
$ kubectl get nodeclaims -o wide
NAME            TYPE         CPU-UTIL   MEM-UTIL
node-a          m5.2xlarge   15%        20%
node-b          m5.2xlarge   18%        22%
#                                             both underutilized — Karpenter
#                                             may consolidate these two onto
#                                             a single, smaller instance

This is the behavior that most differentiates Karpenter from a traditional autoscaler: it doesn’t just react to pending pods by scaling up. It continuously looks for opportunities to replace a set of underutilized nodes with fewer, better-fitting ones — actively working to reduce cost, not just meet demand.

Interruption Handling: Spot Instances Done Right

# Karpenter watches for AWS's spot interruption notice natively
$ kubectl get events --field-selector reason=DisruptionTerminating
LAST SEEN   REASON                  MESSAGE
5s          DisruptionTerminating   Node terminating due to spot interruption,
                                     draining pods gracefully before 2-minute deadline

Before Karpenter, handling spot interruptions gracefully typically meant running a separate tool (like AWS Node Termination Handler) alongside your autoscaler. Karpenter builds this in directly — it’s part of the same controller making the original provisioning decision, not a bolted-on separate system watching for the same signal independently.

⚠ Production Gotchas

Aggressive consolidation without a properly configured PodDisruptionBudget can cause more pod churn than teams expect. Karpenter respects PDBs, but if you haven’t set them, consolidation can evict pods more freely than a team used to Cluster Autoscaler’s more conservative default behavior anticipated.

A misconfigured NodeClass (wrong subnet tags, wrong security group selector) fails silently from the scheduler’s point of view — pods just stay pending, and the actual error is in Karpenter’s controller logs or NodeClaim events, not anywhere the standard kubectl get pods workflow surfaces by default.

Karpenter’s own controller needs real resource requests and, ideally, its own dedicated nodes or a stable node pool — running the thing that provisions your nodes on a node that might itself get consolidated away is a bootstrapping problem worth designing around explicitly.

Quick Reference

kubectl get nodepools                       # provisioning policies defined
kubectl get nodeclasses                     # cloud-specific node configuration
kubectl get nodeclaims                      # individual provisioning decisions
kubectl describe nodeclaim <name>            # why this specific instance was chosen
kubectl get events --field-selector reason=DisruptionTerminating   # interruption/consolidation activity

Contribution Opportunity: Closing Non-AWS Provider Feature Parity

The limitation: Karpenter started as an AWS-specific project and has since been donated to Kubernetes SIGs specifically to become a genuinely cross-cloud tool. The GKE provider and others are real and actively developed, but feature parity with the mature AWS provider — specific instance-selection heuristics, certain disruption/consolidation behaviors, provider-specific NodeClass capabilities — isn’t complete yet, and this is openly tracked, not hidden.

Why it’s hard to fix: Each cloud’s instance-provisioning API, spot-interruption signaling mechanism, and networking model differs meaningfully — replicating AWS provider behavior on GCP or Azure isn’t a port, it’s a re-implementation against a different API with different constraints and different edge cases, done by a provider team with less historical runtime than the original AWS implementation had.

What a contribution-shaped fix looks like: The kubernetes-sigs/karpenter-provider-gcp (and other provider) repositories maintain their own issue trackers with specific, scoped feature-parity gaps against the AWS implementation — this isn’t a vague “make it better,” it’s a list of concrete, individually-tractable items. Picking one specific parity gap, understanding how the AWS provider solved the equivalent problem, and implementing the analogous behavior for the target cloud is real, wanted, trackable upstream work — precisely the shape of contribution this series has been pointing at throughout.

Key Takeaways

• Karpenter provisions the actual best-fit instance for pending pods directly, with no pre-defined node-group middle layer
• NodePool (scheduling flexibility) and NodeClass (cloud-specific detail) are deliberately separated concerns
• Consolidation is active, continuous bin-packing — Karpenter looks for cost savings, not just capacity needs
• Native spot interruption handling removes the need for a separate termination-handling tool
• Non-AWS provider feature parity is explicitly open, tracked work — a real, current, well-scoped contribution opportunity in a project under active cross-cloud expansion

What’s Next

EP09 puts Karpenter head-to-head against the tool it’s increasingly replacing — Cluster Autoscaler — and gives a clear recommendation for when the older, node-group model is still the right choice.

Next: EP09 — Karpenter vs Cluster Autoscaler: Why AWS Built Its Own Scaler

Get EP09 in your inbox when it publishes → linuxcent.com/subscribe