OS Hardening as Code, Episode 4
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening · Automated OpenSCAP Compliance**

Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the current bakex
CLI. If you arrived here looking for stratum or pip install stratumoss, you’re in the
right place: github.com/invicton/bakex.

TL;DR

• “We use CIS L1” means nothing without a verified grade — automated OpenSCAP compliance provides one before any instance is deployed
• BakeX runs OpenSCAP as a stage of every build, and the scan result carries a letter grade A–F
• The grade is OpenSCAP’s own XCCDF score mapped to a letter: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F below that
• SARIF output is machine-readable — importable directly into GitHub Advanced Security, Jira, or any SIEM
• Scanning and baseline comparison live in the web UI and HTTP API, not the CLI — the CLI is validate and build
• A build whose scan fails the blueprint’s threshold ends in Status: failed with exit code 1, and no image is snapshotted

The Problem: A Grade That’s Never Been Verified Is Not a Grade

Security audit request:
"Provide CIS L1 compliance evidence for all production instances"

Team response:
  Instance A: "CIS L1 hardened" — OpenSCAP last run: 4 months ago
  Instance B: "CIS L1 hardened" — OpenSCAP last run: never
  Instance C: "CIS L1 hardened" — OpenSCAP version: 1.2 (current: 1.3.8)
  Instance D: "CIS L1 hardened" — manual scan output: "87% passing"
  Instance E: "CIS L1 hardened" — manual scan output: "91% passing"

"Which profile was used for D and E? Are they comparable?"
"Were they scanned before or after a recent kernel update?"
"Why is C running an old OpenSCAP version?"

Automated OpenSCAP compliance means the grade is generated the same way, on every image, every time, before the image is ever deployed.

EP03 showed that the same HardeningBlueprint YAML builds consistent OS images across six cloud providers. What it left open is the question every auditor eventually asks: how do you know the Ansible hardening actually did what you think it did? Running Ansible-Lockdown successfully means the tasks ran. It does not mean every CIS control is satisfied — some controls can’t be applied by Ansible alone, some require manual verification, and some interact with the environment in unexpected ways.

A compliance team requested CIS L2 evidence for a SOC 2 Type II audit. The security team had been running OpenSCAP scans — but manually, on-demand, using slightly different profiles across teams, with no standard for how to store or compare results.

The audit found four problems:
1. Two instances had been scanned with CIS L1, not L2, despite being labeled “CIS L2”
2. Three instances hadn’t been scanned in over six months
3. The scan outputs from different teams were in different formats (HTML vs XML vs text)
4. Two instances showed “91% passing” and “89% passing” — with no documentation of whether those were acceptable thresholds or what the failing controls were

The audit took two weeks to resolve. The finding wasn’t a security failure — it was a documentation and process failure. But it consumed two weeks of engineering time and appeared in the audit report as a gap.

The root cause: compliance scanning was a manual step that produced inconsistent output in an inconsistent format.

How Automated OpenSCAP Compliance Works

Scanning is a stage of the build, not an afterthought you remember to run:

bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml
      │
      ├─ Provisioning via aws
      │
      ├─ Applying pre-hardening system configuration
      │    (hostname, filesystem, users)
      │
      ├─ Applying Ansible-Lockdown hardening roles
      │
      ├─ Running OpenSCAP compliance scan
      │    ├── benchmark:  xccdf_org.ssgproject.content_benchmark_UBUNTU2204
      │    ├── profile:    ...content_profile_cis_level1_server
      │    └── datastream: ssg-ubuntu2204-ds.xml
      │
      ├─ Snapshotting golden image
      │
      └─ Image ready: ami-0a7f3c9e82d1b4c05

All three compliance identifiers come from the blueprint’s compliance block, and they are full
XCCDF strings rather than friendly names like cis-l1 — they’re handed to oscap unmodified, so
there is no name-mapping layer that can silently pick the wrong profile. That single detail
answers the audit question “which profile was actually used?” without anyone having to remember.

Ubuntu is a special case worth knowing: it ships no SCAP content package in the archive, so BakeX
downloads the matching datastream from a ComplianceAsCode release and checksum-verifies it rather
than failing or silently scanning nothing.

The A-F Grade Calculation

The grade is deliberately boring, and that is the point. BakeX does not invent a scoring model —
it takes OpenSCAP’s own XCCDF score and maps it to a letter:

def score_to_grade(score: float) -> str:
    if score >= 90: return "A"
    if score >= 75: return "B"
    if score >= 60: return "C"
    if score >= 40: return "D"
    return "F"

The thresholds are fixed, not per-blueprint tunables. That is a defensible choice: a grade you can
adjust in the file being graded is not evidence, it’s decoration. If an A means ≥ 90 everywhere,
two teams’ grades are comparable without reading their blueprints — which was exactly the failure
in the audit story above.

What is configurable is when the build refuses to continue:

compliance:
  benchmark: xccdf_org.ssgproject.content_benchmark_UBUNTU2204
  profile: xccdf_org.ssgproject.content_profile_cis_level1_server
  datastream: /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
  fail_on_findings: true      # findings at/above the threshold fail the build
  severity_threshold: medium  # critical | high | medium | low

fail_on_findings with a severity_threshold is severity-based rather than score-based, which
tends to match how people actually reason about risk: one critical finding should block a release
even when 94% of rules pass. When it trips, the build ends in Status: failed, exit code 1, and
no image is snapshotted.

Where the Scan Surface Actually Lives

Worth being blunt about this, because it is the most common wrong assumption: there is no
bakex scan command. The CLI is two verbs — validate and build. Scanning, history, and
baseline comparison live in the web app and its HTTP API, because scan results need somewhere to
persist and something to render them.

Start the server and the whole surface is there:

bakex serve --port 8000

The auditor API is mounted at /api/auditor:

SARIF Export

The report endpoint speaks three formats, selected by query parameter:

# Human-readable — printable HTML, print-to-PDF from the browser
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=html"

# Machine-readable job dict
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=json"

# SARIF 2.1.0 — the one that matters for CI
curl -o scan.sarif.json \
  "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=sarif"

SARIF 2.1.0 is the standard interchange format for security scan results, which means the OpenSCAP
findings land wherever your other scanners’ findings already land:

• GitHub Advanced Security — upload with github/codeql-action/upload-sarif; findings appear in the Security tab, annotated on the PR
• Azure DevOps — native SARIF viewer
• Splunk / SIEM — structured JSON, parseable as events
• AWS Security Hub — importable as findings via the Security Hub API

For audit purposes the SARIF file is the evidence artifact: it carries every rule result, the
profile that was used, and the timestamp. “91% passing” in a spreadsheet is a claim. A SARIF file
in the Security tab is a record.

The badge endpoint is the small touch that gets used most — badge.svg renders the letter grade,
so a repo’s README can show the compliance grade of the image it builds, next to the CI badge.

Drift: Comparing Against a Baseline

The comparison endpoint takes two job IDs — a current scan and a stored baseline — and reports the
delta, including the change in score:

curl "http://localhost:8000/api/auditor/jobs/$CURRENT/compare/$BASELINE"

That is the mechanism behind “what changed since we built this.” You scan the image at build time,
keep that job as the baseline, and re-scan later; the comparison tells you which rules moved and
which direction the score went. It is how you find the instance somebody modified “temporarily”
and never reverted.

The honest limitation: this compares scan jobs, so drift detection is as good as your discipline
about scanning on a schedule. Nothing re-scans your fleet for you.

What Controls Typically Block an A Grade

For Ubuntu 22.04 CIS L1 builds in most cloud environments, these are the controls that most commonly prevent an A grade:

The first two (separate audit partition, AppArmor bootloader) are the most common A→B blockers and often require architecture decisions about how volumes are provisioned at launch versus build time.

Key Takeaways

• Automated OpenSCAP compliance means every image has a verified, reproducible grade generated by the same scanner with the same profile, before it’s ever deployed
• The grade is OpenSCAP’s own XCCDF score mapped to a fixed scale (A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40) — fixed on purpose, so grades from two teams are comparable without reading their blueprints
• The build gate is severity-based, not score-based: fail_on_findings plus severity_threshold blocks a release on one critical finding even when most rules pass
• SARIF 2.1.0 export makes scan results importable into GitHub Advanced Security, Azure DevOps, SIEM, and audit tooling — the SARIF file is the evidence artifact
• Scanning and baseline comparison are HTTP API surfaces, not CLI commands; the CLI is validate and build

What’s Next

Automated OpenSCAP compliance gives every image a verified grade before deployment. What EP04 left open is what happens after the grade is known — specifically, what prevents an engineer from deploying a C-grade image to production “just this once.”

The Pipeline API is the answer. EP05 covers the CI/CD compliance gate: POST /api/pipeline/scan fails the build if the image grade is below threshold. The unhardened image never reaches production — not because engineers are disciplined, but because the pipeline won’t let it through.

Next: CI/CD compliance gate — block unhardened images before they reach production

Get EP05 in your inbox when it publishes → linuxcent.com/subscribe

The post Compliance Grading — Automated OpenSCAP with A-F Scores Before Deployment appeared first on Linuxcent.