OS Hardening as Code, Episode 6
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening · Automated OpenSCAP Compliance · CI/CD Compliance Gate · BakeX Platform**

Note: this series was written when the project was called Stratum. It was renamed to
BakeX at v0.6.0 — same project, same Apache 2.0 license, same team. The old
github.com/rrskris/Stratum URL redirects here, and pip install stratumoss is retired in
favour of pip install bakex. Current home:
github.com/invicton/bakex.

TL;DR

• BakeX is open-source under Apache 2.0 — the engine, blueprint format, scanner, and Pipeline API are all in the repository
• Self-hostable end to end: nothing is locked to a hosted service, and there is no paid tier gating the pipeline
• Two real extension points: provider plugins (drop-in .py or a bakex.providers entry point) and blueprints (pure YAML, no code)
• Architecture: Blueprint YAML → Engine → Provider Layer → Ansible-Lockdown → OpenSCAP → Golden Image → Pipeline API
• The series taught the user-facing interface for five episodes; EP06 covers what’s underneath and how to build on it
• Installation is git clone + docker compose up, or pip install bakex for the CLI and web app

The Series Arc, Inverted

EP01 showed that default cloud AMIs arrive pre-broken. By the time you reach EP06, that problem has a complete solution:

EP01 — The problem:
  Default AMI → Production → Security audit finds gaps
  (unknown OS baseline, unverified hardening, no evidence)

EP06 — The solution:
  HardeningBlueprint YAML
           ↓
    bakex validate          ← EP02 (blueprint as code)
    bakex build             ← EP02
      one file per provider ← EP03 (multi-cloud)
           ↓
    OpenSCAP scan           ← EP04 (compliance grading)
    Grade: A (score 94)
           ↓
    POST /api/pipeline/scan ← EP05 (CI/CD gate)
    passed: true
           ↓
    Production deployment
    (Grade A, SARIF attached, blueprint version-controlled)

For five episodes, you’ve used BakeX as a user. This episode covers what it looks like to run it yourself, extend it, and build on it.

I’ve spent years watching infrastructure teams solve the same OS hardening problem in slightly different ways. Custom scripts that drift. OpenSCAP runs that produce evidence no one reads. Compliance checklists completed by humans who have competing priorities.

The tools exist. ansible-lockdown applies CIS controls reliably. OpenSCAP verifies them accurately. The CI/CD systems can enforce anything you can express as a pass/fail. The gap isn’t the tooling — it’s the integration layer that ties them together into a reproducible, auditable pipeline.

BakeX is that integration layer, open-sourced.

The philosophy is the same as Terraform applied to OS security posture: declare the desired state in a version-controlled file, apply it reproducibly, and verify it automatically. The skip-at-2am problem disappears not because engineers are more careful, but because there’s no step to skip.

The Architecture

┌─────────────────────────────────────────────────────────┐
│                 HardeningBlueprint YAML                  │
│         (version-controlled, provider-agnostic)          │
└─────────────────────┬───────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────┐
│                    BakeX Engine                          │
│                  (Apache 2.0, OSS)                       │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │  Blueprint  │  │   Provider   │  │    Scheduler   │  │
│  │   Parser    │  │    Layer     │  │  (parallel     │  │
│  │             │  │  AWS  GCP    │  │   multi-cloud  │  │
│  │  Validates  │  │  Azure DO    │  │   builds)      │  │
│  │  schema +   │  │  Linode      │  │                │  │
│  │  overrides  │  │  Proxmox     │  │                │  │
│  └─────────────┘  └──────────────┘  └────────────────┘  │
└─────────────────────┬───────────────────────────────────┘
                      │
           ┌──────────┴──────────┐
           ▼                     ▼
  ┌─────────────────┐   ┌─────────────────┐
  │ Ansible-Lockdown │   │  OpenSCAP       │
  │  Runner          │   │  Scanner        │
  │                  │   │                 │
  │  UBUNTU22-CIS    │   │  A-F grade      │
  │  RHEL8-STIG      │   │  SARIF export   │
  │  Custom roles    │   │  Drift detect   │
  └────────┬─────────┘   └────────┬────────┘
           │                      │
           └──────────┬───────────┘
                      │
                      ▼
         ┌─────────────────────────┐
         │   Golden Image          │
         │   (AMI / GCP / Azure)   │
         │   + compliance metadata │
         └────────────┬────────────┘
                      │
                      ▼
         ┌─────────────────────────┐
         │   Pipeline API          │
         │   (Apache 2.0, OSS)     │
         │                         │
         │  POST /api/pipeline/scan │
         │  ← CI/CD gate           │
         └─────────────────────────┘

Every component is open-source under Apache 2.0. The engine, provider layer, Ansible runner, OpenSCAP scanner, and Pipeline API are all in the repository. Nothing is locked to a hosted service.

Installation

Three ways in, depending on how much you want installed on the host.

Docker Compose — recommended, everything preinstalled:

git clone https://github.com/invicton/bakex.git
cd bakex
docker compose up

Open http://localhost:8001. Log in with any username and the admin token as the password —
it’s generated on first start and written to data/.admin_token. Set BAKEX_ADMIN_TOKEN and
BAKEX_SECRET_KEY in docker-compose.yml if you want logins that survive a rebuild.

Compose mounts ~/.aws, ~/.config/gcloud, and ~/.ssh read-only, plus persistent ./data,
./profiles, and ./plugins/providers. That last mount is the one to notice — it’s the
drop-in directory for provider plugins, which matters in the next section.

Published image:

docker run -p 8000:8000 rrskris/bakex:latest

PyPI — CLI and web app:

pip install "bakex[all-providers]"   # or pick extras: aws, gcp, azure, linode, digitalocean, proxmox
bakex serve --port 8000

One caveat worth stating plainly rather than letting you discover it: the extras install each
provider’s cloud SDK, and Ansible and OpenSCAP must be present on the host for real builds. If you
want the batteries-included path, use Compose. bakex validate works anywhere with no host
dependencies at all.

There is no Helm chart. BakeX is a build tool that talks to cloud APIs, not a cluster workload —
it does not need to live in Kubernetes to harden images for it.

The Three Extension Points

1. Blueprints — the extension point with no code in it

The highest-leverage way to extend BakeX isn’t Python. It’s a YAML file.

A blueprint is a complete, self-contained description of a hardened OS on a specific provider,
and the library ships 18 of them. Adding the nineteenth — say Ubuntu 24.04, or CIS Level 2 for a
distro that only has Level 1 today — requires no engine changes, because the benchmark, profile,
and datastream are just strings handed to oscap.

The full format is published as a JSON Schema (Draft 2020-12) at
docs/schema/hardening-blueprint.schema.json. Point your editor at it for autocomplete and
inline validation, or hand it to an LLM and let it draft the blueprint — the schema was published
partly so that agents could write these correctly without reading the source.

The loop is short enough to run in a coffee break:

$EDITOR blueprints/ubuntu/24.04/cis-l1-aws.yaml
bakex validate blueprints/ubuntu/24.04/cis-l1-aws.yaml

Validation is offline and checks more than syntax — it rejects OS/provider combinations the
catalog doesn’t support, so you find out that a distro isn’t available on your target cloud in
milliseconds rather than fifteen minutes into a paid build.

2. Provider Plugins

Adding a cloud means implementing four methods. That’s the whole interface
(bakex/plugins/base_provider.py):

# plugins/providers/my_cloud.py
from bakex.plugins.base_provider import BaseProvider, ProviderResult
from bakex.core.models import ComplianceProfile

class MyCloudProvider(BaseProvider):
    name = "my-cloud"          # matches target.provider in a blueprint

    def provision(self, profile: ComplianceProfile, **kwargs) -> str:
        """Launch a build instance; return its instance ID."""
        ...

    def run_ansible(self, instance_id: str, profile: ComplianceProfile) -> None:
        """Apply the Ansible-Lockdown hardening roles."""
        ...

    def snapshot(self, instance_id: str, profile: ComplianceProfile) -> ProviderResult:
        """Capture the golden image; return the artifact ID."""
        ...

    def teardown(self, instance_id: str) -> None:
        """Destroy the ephemeral build instance."""
        ...

There is no registration command. The loader (bakex/plugins/loader.py) is hybrid and finds
plugins two ways:

1. Drop-in — put the .py file in plugins/providers/. That directory is a Compose volume
   mount, so a plugin dropped there is live in the container without rebuilding an image.
2. Entry point — ship a pip-installable package declaring a bakex.providers entry point.
   This is how a third party distributes a provider without touching the BakeX repo.

Entry points load first and drop-ins load second, so a local file deliberately shadows an
installed package of the same name — which is exactly what you want when debugging someone
else’s provider.

The plugin becomes usable by writing provider: my-cloud in a blueprint’s target block. There
is no --provider flag to pass, because there is no --provider flag anywhere.

One honest note on the validation interaction from EP02: the compatibility check only objects
when both the OS and the provider are in the catalog. An unknown provider is assumed to be a
valid third-party plugin rather than an error — existence is the plugin registry’s call at build
time, compatibility is validation’s. That’s what makes shipping a provider out-of-tree possible
at all.

3. Pipeline Integrations

Beyond the curl-based gate from EP05, BakeX has a webhook system. Webhooks are registered through
the API rather than a config file, so they can be managed by the same automation that manages
everything else:

curl -X POST http://localhost:8001/api/webhooks \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://hooks.slack.com/services/…",
    "events": ["build.failed", "scan.grade_change"],
    "label": "platform-security alerts"
  }'

Five events fire: build.complete, build.failed, scan.complete, scan.failed, and
scan.grade_change. Registering an unknown event name is a 422 rather than a silent no-op —
a small thing that saves an afternoon.

scan.grade_change is the one to wire up first. A build that fails is loud on its own; a base
image that quietly slid from A to B between two scans is the signal nobody notices.

Deliveries are signed. The registration response returns a secret once, and every request
carries an X-BakeX-Signature: sha256=… HMAC so the receiver can verify the payload came from
your BakeX instance and not from anyone who guessed the endpoint URL.

There’s a defensive detail here that’s worth calling out, because it’s the kind of thing that
usually ships broken: webhook target URLs are checked against loopback, private, link-local, and
reserved ranges — including 169.254.169.254 — and they’re re-resolved at send time, not just
at registration. A user-configurable URL that the server will fetch is a textbook SSRF into the
cloud metadata endpoint, and on a tool that holds cloud credentials that would be a very bad day.

The Open-Core Model

BakeX sits alongside the tools that became infrastructure standards by being genuinely usable
before they were commercial:

Everything taught in this series — the blueprint format, the build pipeline, the compliance
grading, the CI/CD gate — is in the repository. There is no feature held back, because there is
currently nothing to hold it back for. Self-host it, extend it, fork it.

Worth being straight about where the project actually is: BakeX is young. It has signed releases,
SBOMs and provenance attestations, an OpenSSF Scorecard, a published JSON Schema, and over a
thousand tests — the engineering is in good order. What it does not yet have is users. If you’re
reading this and the shape of the tool fits your problem, you would be early, and early is when
your opinion changes the design.

The repository is at: github.com/invicton/bakex

What This Series Taught

EP01 — EP06 in one view:

What’s Next

This series closes the OS hardening gap. The same principle — declare desired state, build
reproducibly, verify automatically — applies to every layer of your infrastructure.

Write the next blueprint

The most useful thing you can do with what this series taught is add a blueprint, and it is
genuinely pure YAML — no Python, no engine changes, no build system to learn.

You’ve spent five episodes on Ubuntu 22.04 CIS Level 1. The natural next one is Level 2 for the
same OS: #1 — Ubuntu 22.04 CIS Level 2. The
issue carries the acceptance criteria and the exact verify command, and the review loop is
bakex validate returning 0.

If a different distro is closer to what you actually run, the whole set is filed and labelled:
good first issues, blueprint label.
RHEL 9, AlmaLinux 9, Rocky 9, Debian 12, and Amazon Linux 2023 all have gaps. Each one is one
file, and each is the sort of contribution that takes an evening.

GitHub: github.com/invicton/bakex

Elsewhere on the blog

If you’ve been following the eBPF: From Kernel to Cloud series,
EP10 covers what happens when you combine kernel-level observability with the hardened base BakeX
produces: every connection, every process spawn, every file access — visible from the host kernel,
on an OS baseline you can verify.

The next series is the Purple Team Playbook — real attack paths against cloud and Kubernetes
infrastructure, how they’re detected, and how they’re closed.

Get new episodes in your inbox → linuxcent.com/subscribe

The post BakeX — OS Hardening as a Platform appeared first on Linuxcent.

OS Hardening as Code, Episode 5
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening · Automated OpenSCAP Compliance · CI/CD Compliance Gate**

Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the current bakex
CLI. If you arrived here looking for stratum or pip install stratumoss, you’re in the
right place: github.com/invicton/bakex.

TL;DR

• A CI/CD compliance gate turns an OS hardening grade from a report into a build constraint — unhardened images fail the pipeline before they can be deployed
• POST /api/pipeline/scan scores an image against a pass_threshold and a severity_threshold, and returns a passed boolean
• The endpoint returns HTTP 200 even when the gate fails. curl -sf will not catch it — you must parse .passed. This is the single most important detail on this page
• The gate is two-dimensional: a score floor and a severity ceiling, so one critical finding blocks a release that scores 94
• GitHub Actions, GitLab CI, Jenkins, and Tekton integrations are one curl plus one jq
• The structural guarantee: an image that doesn’t pass the gate doesn’t reach the deploy job

The Problem: A Grade No One Checks Is Decoration

Pipeline without compliance gate:
  Build → Test → Security scan (results to dashboard) → Deploy

What actually happens:
  Build → Test → Security scan → "C grade, but we need to ship" → Deploy anyway
                                           │
                                           └─ Dashboard shows C grade
                                              Nobody is paged
                                              Deployment succeeds

A CI/CD compliance gate means the pipeline can’t continue if the grade is below threshold.

EP04 showed that automated OpenSCAP compliance gives every image a verified, reproducible grade before deployment. What it assumed is that someone checks the grade before deploying. They don’t — not under deadline pressure, not when the image has been “working fine for months,” not at 2am.

The same problem that made hardening runbooks skippable applies to compliance grades: if checking the grade is a discretionary step, it will be skipped.

A new microservice was deployed from an unhardened base image. The team had built it quickly during a sprint, used a community AMI as the base, and planned to harden it “in the next sprint.”

Three weeks later, a penetration test found it. SSH password authentication enabled. Three unnecessary services running — one of them with a known CVE. The finding: the instance had full inbound access from the VPC and was reachable from a compromised adjacent instance.

The deployment had gone through the normal CI/CD pipeline. Unit tests passed. Integration tests passed. A vulnerability scan ran. The scan produced a report that went to a dashboard. Nobody had a gate set up to fail the build if the image was unhardened.

The hardening work from the “next sprint” plan would have taken four hours. The pentest remediation took a week, plus the time to investigate what had been exposed during the three weeks the instance was running.

The CI/CD pipeline had every check except the one that would have caught the base image problem before the first deployment.

The Pipeline API

The Pipeline API is a single HTTP endpoint that takes an image ID, scans it, and returns a verdict:

curl -s -X POST https://bakex.yourdomain.com/api/pipeline/scan \
  -H "X-API-Key: ${BAKEX_TOKEN}" \
  -H "Content-Type: application/json" \
  -d '{
    "image_id": "ami-0a7f3c9e82d1b4c05",
    "provider": "aws",
    "region": "us-east-1",
    "pass_threshold": 75.0,
    "severity_threshold": "high",
    "wait": true
  }'

Authentication takes either X-API-Key or Authorization: Bearer; keys are created at
/settings/api-keys. With wait: true the request blocks until the scan completes — which is what
you want in CI, where a job that returns before the answer exists is worse than a slow one. There’s
a timeout_seconds (default 900) for when it doesn’t.

The response is the same shape whether you passed or failed:

{
  "job_id": "7f3c9e82-4d1b-4c05-a7f3-c9e82d1b4c05",
  "status": "complete",
  "passed": false,
  "grade": "C",
  "score_pct": 72.0,
  "severity_counts": { "critical": 0, "high": 2, "medium": 5, "low": 11 },
  "threshold_violations": ["high"],
  "pass_threshold": 75.0,
  "severity_threshold": "high",
  "image_id": "ami-0c9d5e3f81a2b6e07",
  "sarif_url": ".../api/auditor/scan-image/7f3c9e82.../report?fmt=sarif",
  "html_report_url": ".../api/auditor/scan-image/7f3c9e82.../report"
}

The detail that will silently break your gate

A failed gate still returns HTTP 200. There is no 4xx on failure — the verdict is in the
passed field, not the status code.

That means the pattern everyone reaches for first is wrong:

# WRONG — this never fails. -f only reacts to HTTP >= 400,
# and a failed gate returns 200.
curl -sf -X POST .../api/pipeline/scan -d '...' || exit 1

You have to read the body:

# RIGHT
RESULT=$(curl -s -X POST "${BAKEX_URL}/api/pipeline/scan" \
  -H "X-API-Key: ${BAKEX_TOKEN}" \
  -H "Content-Type: application/json" \
  -d "{\"image_id\": \"${AMI_ID}\", \"pass_threshold\": 75.0, \"severity_threshold\": \"high\"}")

echo "$RESULT" | jq -r '"grade=\(.grade) score=\(.score_pct) passed=\(.passed)"'

if [ "$(echo "$RESULT" | jq -r '.passed')" != "true" ]; then
  echo "Compliance gate failed — violations: $(echo "$RESULT" | jq -c '.threshold_violations')"
  echo "Report: $(echo "$RESULT" | jq -r '.html_report_url')"
  exit 1
fi

A gate that reports failure and exits 0 is worse than no gate, because it produces a green
pipeline and the belief that something was checked.

Two thresholds, not one

passed is the AND of two independent conditions:

passed = (score_pct >= pass_threshold) AND (no findings at or above severity_threshold)

severity_threshold: "high" means any critical or high finding fails the build regardless of
score. An image can score 94 — a comfortable A — and still fail on a single critical finding. That
is the right default: scores average away the thing that gets you breached.

GitHub Actions Integration

# .github/workflows/deploy.yml

jobs:
  build-image:
    runs-on: ubuntu-latest
    outputs:
      ami_id: ${{ steps.build.outputs.ami_id }}
    steps:
      - name: Build hardened AMI
        id: build
        run: |
          AMI_ID=$(bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml --json \
            | jq -r '.artifact_id')
          echo "ami_id=${AMI_ID}" >> $GITHUB_OUTPUT

  compliance-gate:
    runs-on: ubuntu-latest
    needs: build-image
    steps:
      - name: BakeX compliance gate
        run: |
          RESULT=$(curl -s -X POST ${{ vars.BAKEX_URL }}/api/pipeline/scan \
            -H "X-API-Key: ${{ secrets.BAKEX_TOKEN }}" \
            -H "Content-Type: application/json" \
            -d "{\"image_id\": \"${{ needs.build-image.outputs.ami_id }}\",
                 \"pass_threshold\": 75.0, \"severity_threshold\": \"high\"}")

          echo "$RESULT" | jq -r '"grade=\(.grade) score=\(.score_pct)"'

          # Must check .passed — the endpoint returns 200 on failure
          if [ "$(echo "$RESULT" | jq -r '.passed')" != "true" ]; then
            echo "::error::Compliance gate failed: $(echo "$RESULT" | jq -c '.threshold_violations')"
            exit 1
          fi

      - name: Upload SARIF to code scanning
        if: always()
        run: |
          curl -s -o bakex.sarif "$(echo "$RESULT" | jq -r '.sarif_url')"
      - uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: bakex.sarif

  deploy:
    runs-on: ubuntu-latest
    needs: [build-image, compliance-gate]
    steps:
      - name: Deploy to staging
        run: |
          aws autoscaling update-auto-scaling-group \
            --auto-scaling-group-name my-asg \
            --launch-template "ImageId=${{ needs.build-image.outputs.ami_id }}"

The deploy job only runs if compliance-gate passes. The AMI doesn’t reach the autoscaling group if it doesn’t meet the grade threshold.

GitLab CI Integration

# .gitlab-ci.yml

stages:
  - build
  - compliance
  - deploy

build-image:
  stage: build
  script:
    - |
      AMI_ID=$(bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml --json \
        | jq -r '.artifact_id')
      echo "AMI_ID=${AMI_ID}" >> build.env
  artifacts:
    reports:
      dotenv: build.env

compliance-gate:
  stage: compliance
  needs: [build-image]
  script:
    - |
      RESULT=$(curl -s -X POST ${BAKEX_URL}/api/pipeline/scan \
        -H "X-API-Key: ${BAKEX_TOKEN}" \
        -H "Content-Type: application/json" \
        -d "{\"image_id\": \"${AMI_ID}\", \"pass_threshold\": 75.0,
             \"severity_threshold\": \"high\"}")
      echo "$RESULT" | jq -r '"grade=\(.grade) score=\(.score_pct) passed=\(.passed)"'
      test "$(echo "$RESULT" | jq -r '.passed')" = "true"

deploy:
  stage: deploy
  needs: [build-image, compliance-gate]
  script:
    - ./deploy.sh ${AMI_ID}

What the Failed Gate Tells You

The value of the CI/CD compliance gate is not just that it blocks bad images — it’s that the failure output tells engineers what to fix.

The response carries three things an engineer can act on immediately:

$ echo "$RESULT" | jq '{grade, score_pct, threshold_violations, severity_counts}'
{
  "grade": "C",
  "score_pct": 72.0,
  "threshold_violations": ["high"],
  "severity_counts": { "critical": 0, "high": 2, "medium": 5, "low": 11 }
}

threshold_violations names the severities that broke the gate — here, two high findings, not the
score. That distinction matters: an engineer who reads “grade C” starts a broad hardening project,
while one who reads “two high findings” goes and fixes two things.

For the rule-level detail, follow sarif_url. Pushing that SARIF into GitHub code scanning (as in
the workflow above) puts each finding on the pull request diff, which is where someone will actually
read it — a link to a dashboard in a CI log is a link nobody clicks.

Thresholds by Environment

Not all environments need the same bar, and both dimensions are per-request — so the environment
distinction lives in your pipeline, not in BakeX config:

# Production — high score floor, nothing high or above
PASS=90.0 ; SEV=high

# Staging — lower floor, still no criticals
PASS=75.0 ; SEV=critical

# Development — score only, severity effectively off
PASS=60.0 ; SEV=low

curl -s -X POST "${BAKEX_URL}/api/pipeline/scan" \
  -H "X-API-Key: ${BAKEX_TOKEN}" -H "Content-Type: application/json" \
  -d "{\"image_id\": \"${AMI_ID}\", \"pass_threshold\": ${PASS}, \"severity_threshold\": \"${SEV}\"}"

Note that severity_threshold gets stricter as it goes down the list: low fails on any finding
at all, critical fails only on criticals. It reads backwards the first time. Development wanting a
permissive gate wants critical, not low.

Production Gotchas

The 200-on-failure behaviour is the whole ballgame. Repeating it because it is the one thing that
turns this page from useful to harmful if missed: check .passed. Never rely on curl -f, and never
rely on the HTTP status.

Scans take minutes, and wait: true blocks. The endpoint provisions an instance from the image
and scans it. With wait: true your CI job blocks for the duration; timeout_seconds defaults to
900. Set your CI step timeout above that, or use wait: false and poll GET /api/pipeline/scan/{job_id}.

Token rotation. The API key should rotate on the same schedule as other service credentials, and
environments should use different keys — a leaked staging key must not be able to satisfy a
production gate.

The gate needs a reachable BakeX server. This is an HTTP API, not a self-contained action: the
runner must reach the BakeX instance, and that instance needs cloud credentials for the provider
whose image it is scanning.

Key Takeaways

• A CI/CD compliance gate turns a compliance grade from a dashboard metric into a pipeline constraint — the image doesn’t deploy if it doesn’t pass
• POST /api/pipeline/scan is a single HTTP call that any CI/CD system can make — no agent, no plugin, no SDK required
• The endpoint returns 200 even when the gate fails. Parse .passed; curl -sf || exit 1 produces a green pipeline and a false sense of security
• The verdict is two-dimensional — a score floor AND a severity ceiling — so a single critical finding blocks an image that scores 94
• threshold_violations tells an engineer why it failed, which is the difference between “fix two high findings” and “start a hardening project”
• Push the sarif_url into GitHub code scanning so findings land on the pull request, not in a CI log

What’s Next

The CI/CD compliance gate closes the final gap: even if an unhardened image gets built, it can’t deploy. EP05 is the bookmark episode — this is the point where OS hardening becomes structurally enforced rather than procedurally expected.

EP06 is the series closer. For five episodes, you’ve been using BakeX as a user. What does it look like to run it yourself — extend it with a custom provider, deploy it in your own infrastructure, or contribute a blueprint back?

BakeX is Apache 2.0. EP06 is the architecture reveal, the deployment guide, and the extension points for everything the series taught.

Next: BakeX — open-source OS hardening platform for multi-cloud infrastructure

Get EP06 in your inbox when it publishes → linuxcent.com/subscribe

The post The Pipeline Gate — Hardened Images as a CI/CD Build Constraint appeared first on Linuxcent.

OS Hardening as Code, Episode 4
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening · Automated OpenSCAP Compliance**

Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the current bakex
CLI. If you arrived here looking for stratum or pip install stratumoss, you’re in the
right place: github.com/invicton/bakex.

TL;DR

• “We use CIS L1” means nothing without a verified grade — automated OpenSCAP compliance provides one before any instance is deployed
• BakeX runs OpenSCAP as a stage of every build, and the scan result carries a letter grade A–F
• The grade is OpenSCAP’s own XCCDF score mapped to a letter: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F below that
• SARIF output is machine-readable — importable directly into GitHub Advanced Security, Jira, or any SIEM
• Scanning and baseline comparison live in the web UI and HTTP API, not the CLI — the CLI is validate and build
• A build whose scan fails the blueprint’s threshold ends in Status: failed with exit code 1, and no image is snapshotted

The Problem: A Grade That’s Never Been Verified Is Not a Grade

Security audit request:
"Provide CIS L1 compliance evidence for all production instances"

Team response:
  Instance A: "CIS L1 hardened" — OpenSCAP last run: 4 months ago
  Instance B: "CIS L1 hardened" — OpenSCAP last run: never
  Instance C: "CIS L1 hardened" — OpenSCAP version: 1.2 (current: 1.3.8)
  Instance D: "CIS L1 hardened" — manual scan output: "87% passing"
  Instance E: "CIS L1 hardened" — manual scan output: "91% passing"

"Which profile was used for D and E? Are they comparable?"
"Were they scanned before or after a recent kernel update?"
"Why is C running an old OpenSCAP version?"

Automated OpenSCAP compliance means the grade is generated the same way, on every image, every time, before the image is ever deployed.

EP03 showed that the same HardeningBlueprint YAML builds consistent OS images across six cloud providers. What it left open is the question every auditor eventually asks: how do you know the Ansible hardening actually did what you think it did? Running Ansible-Lockdown successfully means the tasks ran. It does not mean every CIS control is satisfied — some controls can’t be applied by Ansible alone, some require manual verification, and some interact with the environment in unexpected ways.

A compliance team requested CIS L2 evidence for a SOC 2 Type II audit. The security team had been running OpenSCAP scans — but manually, on-demand, using slightly different profiles across teams, with no standard for how to store or compare results.

The audit found four problems:
1. Two instances had been scanned with CIS L1, not L2, despite being labeled “CIS L2”
2. Three instances hadn’t been scanned in over six months
3. The scan outputs from different teams were in different formats (HTML vs XML vs text)
4. Two instances showed “91% passing” and “89% passing” — with no documentation of whether those were acceptable thresholds or what the failing controls were

The audit took two weeks to resolve. The finding wasn’t a security failure — it was a documentation and process failure. But it consumed two weeks of engineering time and appeared in the audit report as a gap.

The root cause: compliance scanning was a manual step that produced inconsistent output in an inconsistent format.

How Automated OpenSCAP Compliance Works

Scanning is a stage of the build, not an afterthought you remember to run:

bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml
      │
      ├─ Provisioning via aws
      │
      ├─ Applying pre-hardening system configuration
      │    (hostname, filesystem, users)
      │
      ├─ Applying Ansible-Lockdown hardening roles
      │
      ├─ Running OpenSCAP compliance scan
      │    ├── benchmark:  xccdf_org.ssgproject.content_benchmark_UBUNTU2204
      │    ├── profile:    ...content_profile_cis_level1_server
      │    └── datastream: ssg-ubuntu2204-ds.xml
      │
      ├─ Snapshotting golden image
      │
      └─ Image ready: ami-0a7f3c9e82d1b4c05

All three compliance identifiers come from the blueprint’s compliance block, and they are full
XCCDF strings rather than friendly names like cis-l1 — they’re handed to oscap unmodified, so
there is no name-mapping layer that can silently pick the wrong profile. That single detail
answers the audit question “which profile was actually used?” without anyone having to remember.

Ubuntu is a special case worth knowing: it ships no SCAP content package in the archive, so BakeX
downloads the matching datastream from a ComplianceAsCode release and checksum-verifies it rather
than failing or silently scanning nothing.

The A-F Grade Calculation

The grade is deliberately boring, and that is the point. BakeX does not invent a scoring model —
it takes OpenSCAP’s own XCCDF score and maps it to a letter:

def score_to_grade(score: float) -> str:
    if score >= 90: return "A"
    if score >= 75: return "B"
    if score >= 60: return "C"
    if score >= 40: return "D"
    return "F"

The thresholds are fixed, not per-blueprint tunables. That is a defensible choice: a grade you can
adjust in the file being graded is not evidence, it’s decoration. If an A means ≥ 90 everywhere,
two teams’ grades are comparable without reading their blueprints — which was exactly the failure
in the audit story above.

What is configurable is when the build refuses to continue:

compliance:
  benchmark: xccdf_org.ssgproject.content_benchmark_UBUNTU2204
  profile: xccdf_org.ssgproject.content_profile_cis_level1_server
  datastream: /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
  fail_on_findings: true      # findings at/above the threshold fail the build
  severity_threshold: medium  # critical | high | medium | low

fail_on_findings with a severity_threshold is severity-based rather than score-based, which
tends to match how people actually reason about risk: one critical finding should block a release
even when 94% of rules pass. When it trips, the build ends in Status: failed, exit code 1, and
no image is snapshotted.

Where the Scan Surface Actually Lives

Worth being blunt about this, because it is the most common wrong assumption: there is no
bakex scan command. The CLI is two verbs — validate and build. Scanning, history, and
baseline comparison live in the web app and its HTTP API, because scan results need somewhere to
persist and something to render them.

Start the server and the whole surface is there:

bakex serve --port 8000

The auditor API is mounted at /api/auditor:

SARIF Export

The report endpoint speaks three formats, selected by query parameter:

# Human-readable — printable HTML, print-to-PDF from the browser
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=html"

# Machine-readable job dict
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=json"

# SARIF 2.1.0 — the one that matters for CI
curl -o scan.sarif.json \
  "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=sarif"

SARIF 2.1.0 is the standard interchange format for security scan results, which means the OpenSCAP
findings land wherever your other scanners’ findings already land:

• GitHub Advanced Security — upload with github/codeql-action/upload-sarif; findings appear in the Security tab, annotated on the PR
• Azure DevOps — native SARIF viewer
• Splunk / SIEM — structured JSON, parseable as events
• AWS Security Hub — importable as findings via the Security Hub API

For audit purposes the SARIF file is the evidence artifact: it carries every rule result, the
profile that was used, and the timestamp. “91% passing” in a spreadsheet is a claim. A SARIF file
in the Security tab is a record.

The badge endpoint is the small touch that gets used most — badge.svg renders the letter grade,
so a repo’s README can show the compliance grade of the image it builds, next to the CI badge.

Drift: Comparing Against a Baseline

The comparison endpoint takes two job IDs — a current scan and a stored baseline — and reports the
delta, including the change in score:

curl "http://localhost:8000/api/auditor/jobs/$CURRENT/compare/$BASELINE"

That is the mechanism behind “what changed since we built this.” You scan the image at build time,
keep that job as the baseline, and re-scan later; the comparison tells you which rules moved and
which direction the score went. It is how you find the instance somebody modified “temporarily”
and never reverted.

The honest limitation: this compares scan jobs, so drift detection is as good as your discipline
about scanning on a schedule. Nothing re-scans your fleet for you.

What Controls Typically Block an A Grade

For Ubuntu 22.04 CIS L1 builds in most cloud environments, these are the controls that most commonly prevent an A grade:

The first two (separate audit partition, AppArmor bootloader) are the most common A→B blockers and often require architecture decisions about how volumes are provisioned at launch versus build time.

Key Takeaways

• Automated OpenSCAP compliance means every image has a verified, reproducible grade generated by the same scanner with the same profile, before it’s ever deployed
• The grade is OpenSCAP’s own XCCDF score mapped to a fixed scale (A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40) — fixed on purpose, so grades from two teams are comparable without reading their blueprints
• The build gate is severity-based, not score-based: fail_on_findings plus severity_threshold blocks a release on one critical finding even when most rules pass
• SARIF 2.1.0 export makes scan results importable into GitHub Advanced Security, Azure DevOps, SIEM, and audit tooling — the SARIF file is the evidence artifact
• Scanning and baseline comparison are HTTP API surfaces, not CLI commands; the CLI is validate and build

What’s Next

Automated OpenSCAP compliance gives every image a verified grade before deployment. What EP04 left open is what happens after the grade is known — specifically, what prevents an engineer from deploying a C-grade image to production “just this once.”

The Pipeline API is the answer. EP05 covers the CI/CD compliance gate: POST /api/pipeline/scan fails the build if the image grade is below threshold. The unhardened image never reaches production — not because engineers are disciplined, but because the pipeline won’t let it through.

Next: CI/CD compliance gate — block unhardened images before they reach production

Get EP05 in your inbox when it publishes → linuxcent.com/subscribe

The post Compliance Grading — Automated OpenSCAP with A-F Scores Before Deployment appeared first on Linuxcent.

OS Hardening as Code, Episode 3
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening**

Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the current bakex
CLI. If you arrived here looking for stratum or pip install stratumoss, you’re in the
right place: github.com/invicton/bakex.

TL;DR

• Multi-cloud OS hardening with separate scripts per provider means three scripts that drift within weeks
• A HardeningBlueprint YAML separates compliance intent (portable) from provider details (handled by BakeX’s provider layer)
• You keep one blueprint file per provider — and every section except target stays byte-identical across all six. The diff below proves it
• Provider-specific differences — disk names, cloud-init ordering, base image identifiers — are abstracted away from the blueprint author
• The compliance posture becomes reviewable in a pull request: a control change touches six files identically, and a reviewer can see that at a glance
• Six providers ship as working blueprints today: AWS, GCP, Azure, DigitalOcean, Linode, Proxmox

The Problem: Three Clouds, Three Scripts, Three Ways to Drift

AWS hardening script          GCP hardening script          Azure hardening script
├── /dev/xvd* disk refs       ├── /dev/sda* disk refs       ├── /dev/sda* disk refs
├── 169.254.169.254 IMDS      ├── 169.254.169.254 IMDS      ├── 169.254.169.254 IMDS
├── cloud-init order A        ├── cloud-init order B        ├── cloud-init order C
└── Updated: Jan 2025         └── Updated: Aug 2024         └── Updated: Mar 2024
                                         │
                                         └─ 5 months behind
                                            on CIS updates

Multi-cloud OS hardening starts as a copy-paste of the AWS script. Within a month, the clouds diverge.

EP02 showed that a HardeningBlueprint YAML eliminates the skip-at-2am problem by making hardening a build artifact. What it assumed — quietly — is that you’re building for one provider. The moment you expand to a second cloud, the provider-specific details in the blueprint become a problem: disk names differ, cloud-init fires in a different order, and AWS-specific assumptions break silently on GCP.

We expanded from AWS to GCP six months ago. The EC2 hardening script had been working reliably for over a year. The GCP engineer took the AWS script, made some quick changes, and started building images.

The first GCP images had a subtle problem: the /tmp and /home separate partition entries in /etc/fstab referenced /dev/xvdb — an AWS disk naming convention. GCP uses /dev/sdb. The fstab entries were silently ignored. The mounts existed but weren’t restricted. The CIS controls for separate filesystem partitions were listed as passing in the scan output because the Ansible task had “run successfully” — it just hadn’t done what we thought.

It took a pentest three months later to catch it. The finding: six production GCP instances with /tmp not mounted with noexec, nosuid, nodev — despite our “CIS L1 hardened” label.

The root cause wasn’t the engineer. It was a hardening approach that required cloud-specific knowledge embedded in the script rather than in a provider abstraction layer.

How BakeX Separates Compliance Intent from Provider Details

Multi-cloud OS hardening works when the compliance intent and the provider details are kept strictly separate.

HardeningBlueprint YAML
(compliance intent — portable)
         │
         ▼
  BakeX Provider Layer
  ┌─────────────────────────────────────────────┐
  │  AWS         │  GCP         │  Azure        │
  │  /dev/xvd*   │  /dev/sda*   │  /dev/sda*    │
  │  IMDS v2     │  GCP IMDS    │  Azure IMDS   │
  │  cloud-init  │  cloud-init  │  waagent       │
  │  order A     │  order B     │  order C       │
  └─────────────────────────────────────────────┘
         │
         ▼
  Ansible-Lockdown + Provider-Aware Configuration
         │
         ▼
  OpenSCAP Scan
         │
         ▼
  Golden Image (AMI / GCP Image / Azure Image)

The blueprint author declares what should be true about the OS. BakeX’s provider layer handles how that’s achieved on each cloud.

The disk naming, cloud-init sequencing, metadata endpoint configuration, and provider-specific package repositories are all abstracted into the provider layer. They never appear in the blueprint file.

The Same Blueprint Across Six Providers

Here is the part people expect to be a flag, and isn’t. There is no --provider switch. The
provider is a field inside the blueprint, so you keep one file per target:

$ ls blueprints/ubuntu/22.04/
cis-l1-aws.yaml           cis-l1-digitalocean.yaml  cis-l1-linode.yaml
cis-l1-azure.yaml         cis-l1-gcp.yaml           cis-l1-proxmox.yaml

# Validate all six at once — offline, no cloud API calls
$ bakex validate blueprints/ubuntu/22.04/*.yaml
OK    blueprints/ubuntu/22.04/cis-l1-aws.yaml  (HardeningBlueprint 'ubuntu22-cis-l1-aws')
...
6/6 blueprint(s) valid.

# Build one
$ bakex build blueprints/ubuntu/22.04/cis-l1-gcp.yaml
Building 'ubuntu22-cis-l1-gcp' (gcp) → job 7f3c9e82-…

That design choice looks like more files, and it is. What you get for it is that a blueprint
is completely self-describing: the file names its own cloud and its own base image, so it
builds the same way on your laptop, in CI, and on a colleague’s machine with no flags to
forget and no environment to match.

The claim worth testing: how much actually differs between those six files?

I parsed all six and compared every section except metadata and target:

compliance    identical across all 6
controls      identical across all 6   (same rules, same enable state)
filesystem    identical across all 6
users         identical across all 6
system        identical across all 6

Only the target block changes, and it changes in exactly the way you’d expect:

Six wildly different ways of naming “Ubuntu 22.04 LTS” and sizing a 2 vCPU / 4 GB box. That
is the entire provider-specific surface. The CIS benchmark, the profile, the datastream, the
mount options, the locked root account, the control overrides and their justifications — all
byte-identical.

One honest caveat, because I checked rather than assumed: the six files are semantically
identical but not textually so. The justification string on one disabled SELinux rule is
worded slightly differently between files — same rule, same enabled: false, same meaning,
different prose. It’s a cosmetic inconsistency in the shipped library, not a behavioural one,
and it’s the kind of thing you only find by diffing rather than trusting the header comment.

If you change the compliance posture — add a control override, tighten a mount option — you
change it identically in six files and rebuild. A reviewer sees six identical hunks in the
diff. A sixth hunk that looks different is a bug, and it’s visible in code review rather than
three months later in a pentest.

What the Provider Layer Handles

The provider layer is where the cloud-specific knowledge lives, so the blueprint author doesn’t have to carry it:

Disk naming:

The CIS controls for separate /tmp and /home partitions reference disk paths that differ across these providers. The provider layer translates the blueprint’s filesystem.tmp declaration into the correct fstab entries for the target cloud.

Cloud-init ordering:

Different providers initialize services in different orders. On AWS, the network is available before cloud-init runs most tasks. On GCP, some network configuration happens after cloud-init starts. On Azure, the waagent handles some configuration that cloud-init handles elsewhere.

The provider layer sequences the hardening steps to run in the correct order for each provider — specifically, it waits for network availability before applying network-level hardening, and ensures the package manager is configured before running Ansible roles that require package installation.

Metadata endpoint configuration:

CIS controls include restrictions on access to the instance metadata service (IMDSv2 enforcement on AWS, equivalent controls on GCP/Azure). The provider layer applies the correct restriction for each cloud — the blueprint just declares compliance: benchmark: cis-l1.

Building Every Provider

There is no built-in fan-out flag, and honestly none is needed — the CLI is exit-code shaped,
so the shell already does this well:

# Validate everything first; stop before spending money if anything is wrong
bakex validate blueprints/ubuntu/22.04/*.yaml || exit 1

# Then build each target
for bp in blueprints/ubuntu/22.04/cis-l1-*.yaml; do
  bakex build "$bp" --json > "builds/$(basename "$bp" .yaml).json" &
done
wait

--json emits the job record — id, profile name, provider, status, artifact ID, error — which
is what you want when six builds are writing to six files at once. Every build either lands a
complete status with an artifact ID, or a failed status with the reason. Nothing produces
a half-hardened image.

The validate-then-build ordering matters more than it looks. Validation is offline and takes
milliseconds; a build takes 15–25 minutes and costs money. Catching a malformed blueprint or an
unsupported OS/provider pair in the first step means you never launch the instance.

Blueprint Versioning and Drift

Version-controlling the blueprint file solves a problem multi-cloud environments hit
consistently: knowing what your OS security posture was six months ago. The blueprint is the
answer — it’s a file, in git, with a commit history and a reviewer’s name on every change.

Re-scanning a running instance against the posture that built it is a separate job, and it
does not live in the CLI. bakex validate and bakex build are the two CLI verbs; scanning
and drift comparison are in the web UI and the HTTP API, where the scan results have somewhere
to live. EP04 covers that surface in detail — the A–F grade, the SARIF export, and comparing a
current scan against a stored baseline.

The useful discipline in the meantime is unglamorous: rebuild from the blueprint rather than
patching running instances. An instance that drifted is a symptom; the blueprint is the cure,
and re-baking is cheaper than reconciling.

Production Gotchas

Provider-specific CIS controls exist. CIS AWS Foundations Benchmark and CIS GCP Benchmark include cloud-specific controls (VPC flow logs, CloudTrail, etc.) that are separate from the OS-level CIS controls. The blueprint handles OS-level controls. Cloud-level controls (IAM, logging, network configuration) belong in your cloud security posture management tooling.

Build costs vary by provider. On AWS, the build instance is a t3.medium for 15–20 minutes (~$0.02). On GCP and Azure, equivalent pricing applies. For multi-provider builds, run them in regions close to your primary workloads to minimize image transfer time.

Proxmox is a template VMID, not an image name. Every cloud provider names its base image with a string; Proxmox names it with a number — the VE template’s VMID (9000 in the shipped blueprint). The provider talks to the Proxmox API remotely via proxmoxer, so no agent on the host is required, but it does need host credentials and it discovers the built VM’s IP through the QEMU guest agent inside the VM. If the guest agent isn’t installed in your template, the build will provision and then hang waiting for an address.

KVM and Proxmox are deliberately different providers. They look interchangeable and aren’t: a KVM target names a downloadable cloud image, a Proxmox target names a VE template that already exists on your host. Don’t assume a blueprint written for one works on the other.

GCP image sharing across projects requires explicit IAM. GCP machine images aren’t automatically available to other projects in the organization. BakeX builds the image; sharing it is a GCP IAM operation you configure at the project or organization level — there’s no BakeX command that grants cross-project access for you.

Key Takeaways

• Multi-cloud OS hardening with separate scripts per provider creates inevitable drift; a provider-abstracted blueprint eliminates it
• BakeX ships working blueprints for AWS, GCP, Azure, DigitalOcean, Linode, and Proxmox — one file per provider, with target as the only section that differs
• The provider is a field in the blueprint, not a CLI flag: every file is self-describing and builds identically in CI, locally, or on a teammate’s machine
• Fan-out is a shell loop over exit codes, not a framework feature — validate all six offline first, then build in parallel with --json
• Blueprint version control is the single source of truth for OS security posture history — and a compliance change that isn’t identical across all six providers shows up as an odd hunk in code review

What’s Next

Six providers, one compliance posture, and a diff that proves it. EP03 showed that the multi-cloud drift problem disappears when provider details are confined to a single block of the blueprint.

What neither EP02 nor EP03 answered is the auditor’s question: how do you know the image is actually compliant? “We ran CIS L1” is not an answer. “Grade A, 98/100 controls, SARIF export attached” is.

EP04 covers automated OpenSCAP compliance: the post-build scan in detail — how the A-F grade is calculated, what controls block an A grade, how SARIF exports work, and how drift detection catches what changed after deployment.

Next: automated OpenSCAP compliance — CIS benchmark grading before deployment

Get EP04 in your inbox when it publishes → linuxcent.com/subscribe

The post One Blueprint, Six Clouds — Multi-Provider OS Image Builds appeared first on Linuxcent.

OS Hardening as Code, Episode 2
Cloud AMI Security Risks · Linux Hardening as Code**

Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the current bakex
CLI. If you arrived here looking for stratum or pip install stratumoss, you’re in the
right place: github.com/invicton/bakex.

TL;DR

• A hardening runbook is a list of steps someone runs. A HardeningBlueprint YAML is a build artifact — if it wasn’t applied, the image doesn’t exist
• Linux hardening as code means declaring your entire OS security baseline in a single YAML file and building it reproducibly across any provider
• bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml either produces a hardened image or fails — there is no partial state
• The blueprint includes: target OS/provider/base image, compliance benchmark, filesystem layout, users, and per-rule overrides with documented justifications
• One blueprint file = one source of truth for your hardening posture, version-controlled and reviewable like any other infrastructure code
• Post-build OpenSCAP scan runs automatically — the image only snapshots if it passes

The Problem: A Runbook That Gets Skipped Once Is a Runbook That Gets Skipped

Hardening runbook
       │
       ▼
  Human executes
  steps manually
       │
       ├─── 47 deployments: followed correctly
       │
       └─── 1 deployment at 2am: step 12 skipped
                    │
                    ▼
           Instance in production
           without audit logging,
           SSH password auth enabled,
           unnecessary services running

Linux hardening as code eliminates the human decision point. If the blueprint wasn’t applied, the image doesn’t exist.

EP01 showed that default cloud AMIs arrive pre-broken — unnecessary services, no audit logging, weak kernel parameters, SSH configured for convenience not security. The obvious response is a hardening script. But a script run by a human is still a process step. It can be skipped. It can be done halfway. It can drift across different engineers who each interpret “run the hardening script” slightly differently.

A production deployment last year. The platform team had a solid CIS L1 hardening runbook — 68 steps, well-documented, followed consistently. Then a critical incident at 2am required three new instances to be deployed on short notice. The engineer on call ran the provisioning script and, under pressure, skipped the hardening step with the intention of running it the next morning.

They didn’t. The three instances stayed in production unhardened for six weeks before an automated scan caught them. Audit logging wasn’t configured. SSH was accepting password authentication. Two unnecessary services were running that weren’t in the approved software list.

Nothing was breached. But the finding went into the next compliance report as a gap, the team spent a week remediating, and the post-mortem conclusion was “we need better runbook discipline.”

That’s the wrong conclusion. The runbook isn’t the problem. The problem is that hardening was a process step instead of a build constraint.

What Linux Hardening as Code Actually Means

Linux hardening as code is the same principle as infrastructure as code applied to OS security posture: the desired state is declared in a file, the file is the source of truth, and the execution is deterministic and repeatable.

HardeningBlueprint YAML
         │
         ▼
  bakex build
         │
  ┌──────┴──────────────────┐
  │  Provider Layer          │
  │  (cloud-init, disk       │
  │   names, metadata        │
  │   endpoint per provider) │
  └──────┬──────────────────┘
         │
  ┌──────┴──────────────────┐
  │  Ansible-Lockdown        │
  │  (CIS L1/L2, STIG —      │
  │   the hardening steps)   │
  └──────┬──────────────────┘
         │
  ┌──────┴──────────────────┐
  │  OpenSCAP Scanner        │
  │  (post-build verify)     │
  └──────┬──────────────────┘
         │
         ▼
  Golden Image (AMI/GCP image/Azure image)
  + Compliance grade in image metadata

The YAML file is what you write. BakeX handles the rest.

The HardeningBlueprint YAML

The blueprint is the complete, auditable declaration of your OS security posture:

# blueprints/ubuntu/22.04/cis-l1-aws.yaml
bakex_version: "0.6.0"
kind: HardeningBlueprint

metadata:
  name: ubuntu22-cis-l1-aws
  version: "1.0.0"
  description: >
    CIS Ubuntu Linux 22.04 LTS Benchmark — Level 1 Server profile for AWS.
  tags: [ubuntu, ubuntu22.04, cis, level1, server, aws]

target:
  os: ubuntu22.04
  arch: x86_64
  provider: aws
  base_image: ami-0c7217cdde317cfec   # Ubuntu 22.04 LTS, us-east-1 (x86_64)
  instance_type: t3.medium
  root_volume_size_gb: 20

system:
  hostname: hardened-node
  timezone: UTC
  locale: en_US.UTF-8
  selinux_mode: null              # Ubuntu uses AppArmor; null skips SELinux

filesystem:                       # CIS 1.1.x: separate mounts with noexec/nosuid/nodev
  - device: tmpfs
    mountpoint: /tmp
    fstype: tmpfs
    options: [rw, nosuid, nodev, noexec, relatime]
    size: 2G

  - device: tmpfs
    mountpoint: /dev/shm
    fstype: tmpfs
    options: [rw, nosuid, nodev, noexec, relatime]

users:
  root:
    lock: true                    # CIS 5.4.2: lock the root account
  accounts:
    - name: bakex-admin
      groups: [sudo]
      shell: /bin/bash
      ssh_authorized_keys: []     # Add: - "ssh-ed25519 AAAA..."

compliance:
  benchmark: xccdf_org.ssgproject.content_benchmark_UBUNTU2204
  profile: xccdf_org.ssgproject.content_profile_cis_level1_server
  datastream: /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
  fail_on_findings: true
  severity_threshold: medium

controls:                         # per-rule overrides, keyed by XCCDF rule ID
  xccdf_org.ssgproject.content_rule_grub2_enable_selinux:
    enabled: false
    justification: >
      Ubuntu Linux uses AppArmor as the mandatory access control framework.
      SELinux is not installed or applicable on this distribution.

  xccdf_org.ssgproject.content_rule_package_telnet_removed: true

That is not a simplified illustration — it is the shipped cis-l1-aws.yaml, trimmed of comments. Each section is explicit:

target — which OS, which provider, and which base image to harden. This is the only provider-specific section. Everything below it is portable.

compliance — the OpenSCAP benchmark, profile, and datastream. These are full XCCDF identifiers, not friendly names like cis-l1, because they’re passed straight to oscap. Ubuntu ships no SCAP content package, so BakeX downloads the matching datastream from a ComplianceAsCode release and checksum-verifies it.

filesystem — a list of mounts, each with its CIS-required options.

users — root lock state and the service accounts baked into the image.

controls — documented exceptions, keyed by XCCDF rule ID. Not suppressions — overrides with a recorded justification. A bare true enforces a rule; a map with enabled: false disables it and demands a reason. This is the difference between “we turned off this control” and “this control is not applicable, documented here.”

Building the Image

# Validate the blueprint before building — exit 0 if valid, 1 if not
bakex validate blueprints/ubuntu/22.04/cis-l1-aws.yaml

# Build — this will take 15-25 minutes
bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml

Validation is fast and offline:

OK    blueprints/ubuntu/22.04/cis-l1-aws.yaml  (HardeningBlueprint 'ubuntu22-cis-l1-aws')

1/1 blueprint(s) valid.

The build walks five stages, and the provider is read from the blueprint — there is no --provider flag to get wrong:

Building 'ubuntu22-cis-l1-aws' (aws) → job 7f3c9e82-4d1b-4c05-a7f3-c9e82d1b4c05
[2026-07-27T15:42:01] Provisioning via aws
[2026-07-27T15:42:45] Applying pre-hardening system configuration (hostname, filesystem, users)
[2026-07-27T15:43:12] Applying Ansible-Lockdown hardening roles
[2026-07-27T15:52:11] Running OpenSCAP compliance scan
[2026-07-27T15:54:10] Snapshotting golden image
[2026-07-27T15:54:47] Image ready: ami-0a7f3c9e82d1b4c05

Status: complete

Two things worth noticing. bakex build also accepts a bundled profile name, so bakex build ubuntu22-cis-l1-aws does the same thing without a path. And --json emits the job record instead of the log, which is what you want in CI.

If the post-build scan comes back below the configured threshold, the build fails — no AMI is created. The instance is terminated. The image does not exist. Exit code is 1, and Status: failed carries the reason.

That is the structural guarantee. You cannot skip a build step at 2am because at 2am you’re calling bakex build, not running steps manually.

The Control Override Mechanism

The override mechanism is what separates this from checkbox compliance.

Every security benchmark has controls that conflict with how production environments actually work. CIS L1 recommends /tmp on a separate partition. Many cloud instances use tmpfs with equivalent nodev, nosuid, noexec mount options. The intent of the control is satisfied. The literal implementation differs.

Without an override mechanism, you have two bad options: fail the scan (noisy, meaningless), or configure the scanner to ignore the control (undocumented, invisible to auditors).

The blueprint’s controls section gives you a third option: record the override and its justification in the same version-controlled artifact that produced the image.

controls:
  xccdf_org.ssgproject.content_rule_grub2_enable_selinux:
    enabled: false
    justification: >
      Ubuntu Linux uses AppArmor as the mandatory access control framework.
      SELinux is not installed or applicable on this distribution.

Note that the key is the full XCCDF rule ID, not a CIS section number. That is deliberate — the rule ID is what OpenSCAP reports, so the override and the scanner finding line up exactly with no translation table in between.

Be precise about what this does today. As of v0.6.0 the controls block is declarative:
it records intent alongside the blueprint, and the UI reads it back, but it is not yet compiled
into an OpenSCAP tailoring file. The scan still evaluates the full profile, so an overridden
rule still shows up as a finding and still counts against the score. What you get right now is
provenance — the justification lives in git, next to the thing that built the image, reviewable
in a pull request — not automatic score adjustment.

That is a smaller claim than “documented exceptions are counted as compliant,” and it’s the
true one. Wiring overrides through to scan tailoring is the obvious next step, and it’s the
kind of well-scoped gap that makes a good first contribution.

What the Blueprint Gives You That a Script Doesn’t

The last row is the one that matters. A script is skippable because there’s a human in the loop. A blueprint is a build artifact — you can’t deploy the image without the blueprint having been applied, because the image is what the blueprint produces.

Validating a Blueprint Before Building

# Schema validation — one file
bakex validate blueprints/ubuntu/22.04/cis-l1-aws.yaml

# Or the whole library at once
bakex validate blueprints/**/*.yaml

# Machine-readable, for CI and agents
bakex validate blueprints/ubuntu/22.04/cis-l1-aws.yaml --json

bakex validate exits 0 when every file is valid and 1 when any file fails, which makes it a one-line CI gate. It never touches a cloud API — it is pure schema and cross-field checking, so it runs in your pipeline before you’ve paid for a build instance.

That cross-field part matters more than it sounds. Validation rejects OS/provider combinations the catalog doesn’t support, so a blueprint asking for an OS that provider can’t supply fails at validation time rather than fifteen minutes into a build.

If you want to generate or check blueprints from something other than the CLI, the format is published as a JSON Schema (Draft 2020-12) at docs/schema/hardening-blueprint.schema.json. Point your editor at it for autocomplete, or hand it to an LLM and let it write the blueprint.

Production Gotchas

Build time is 15–25 minutes. Ansible-Lockdown applies 144+ tasks for CIS L1. Build this into your pipeline timing — don’t expect golden images in 3 minutes.

Cloud-init ordering matters. On AWS, certain hardening steps (sysctl tuning, PAM configuration) interact with cloud-init. The BakeX provider layer handles sequencing — but if you add custom hardening roles, test the cloud-init interaction explicitly.

Some CIS controls conflict with managed service requirements. AWS Systems Manager Session Manager requires specific SSH configuration. RDS requires specific networking settings. Use the controls override section to document these — don’t suppress them silently.

Kernel parameter hardening requires a reboot. Controls in the 3.x (network parameters) and 1.5.x (kernel modules) sections apply sysctl changes that take effect on reboot. The BakeX build process reboots the instance before the OpenSCAP scan — don’t skip the reboot if you’re building manually.

Key Takeaways

• Linux hardening as code means the blueprint YAML is the build artifact — the image either exists and is hardened, or it doesn’t exist
• The controls override mechanism is the difference between undocumented suppressions and auditable, reasoned exceptions
• Post-build OpenSCAP scan runs automatically — a failing grade blocks image creation
• bakex validate is an offline, exit-code-shaped CI gate — it catches unsupported OS/provider pairs before a build instance is ever launched
• The compliance sections are portable across providers (EP03 covers this): swap the target block, and the benchmark, filesystem, users, and control overrides stay byte-identical
• Version-controlling the blueprint gives you a complete history of what your OS security posture was at any point in time — the same way Terraform state tracks infrastructure

What’s Next

One blueprint, one provider. EP02 showed that the skip-at-2am problem is solved when hardening is a build artifact rather than a process step.

What it didn’t address: what happens when you expand to a second cloud. GCP uses different disk names. Azure cloud-init fires in a different order. The AWS metadata endpoint IP is different from every other provider. If you maintain separate hardening scripts per cloud, they drift within a month.

EP03 covers multi-cloud OS hardening: one compliance posture, six providers, no drift — and it shows the diff that proves it.

Next: multi-cloud OS hardening — one blueprint for AWS, GCP, and Azure

Get EP03 in your inbox when it publishes → linuxcent.com/subscribe

The post Hardening Blueprint as Code — Declare Your OS Baseline in YAML appeared first on Linuxcent.