OS Hardening as Code, Episode 4
Cloud AMI Security Risks · Linux Hardening as Code · Multi-Cloud OS Hardening · Automated OpenSCAP Compliance**
Note: the tool in this series was released as Stratum and renamed to BakeX at
v0.6.0 — same project, same license, same team. Commands below use the currentbakex
CLI. If you arrived here looking forstratumorpip install stratumoss, you’re in the
right place: github.com/invicton/bakex.
TL;DR
- “We use CIS L1” means nothing without a verified grade — automated OpenSCAP compliance provides one before any instance is deployed
- BakeX runs OpenSCAP as a stage of every build, and the scan result carries a letter grade A–F
- The grade is OpenSCAP’s own XCCDF score mapped to a letter: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F below that
- SARIF output is machine-readable — importable directly into GitHub Advanced Security, Jira, or any SIEM
- Scanning and baseline comparison live in the web UI and HTTP API, not the CLI — the CLI is
validateandbuild - A build whose scan fails the blueprint’s threshold ends in
Status: failedwith exit code 1, and no image is snapshotted
The Problem: A Grade That’s Never Been Verified Is Not a Grade
Security audit request:
"Provide CIS L1 compliance evidence for all production instances"
Team response:
Instance A: "CIS L1 hardened" — OpenSCAP last run: 4 months ago
Instance B: "CIS L1 hardened" — OpenSCAP last run: never
Instance C: "CIS L1 hardened" — OpenSCAP version: 1.2 (current: 1.3.8)
Instance D: "CIS L1 hardened" — manual scan output: "87% passing"
Instance E: "CIS L1 hardened" — manual scan output: "91% passing"
"Which profile was used for D and E? Are they comparable?"
"Were they scanned before or after a recent kernel update?"
"Why is C running an old OpenSCAP version?"
Automated OpenSCAP compliance means the grade is generated the same way, on every image, every time, before the image is ever deployed.
EP03 showed that the same HardeningBlueprint YAML builds consistent OS images across six cloud providers. What it left open is the question every auditor eventually asks: how do you know the Ansible hardening actually did what you think it did? Running Ansible-Lockdown successfully means the tasks ran. It does not mean every CIS control is satisfied — some controls can’t be applied by Ansible alone, some require manual verification, and some interact with the environment in unexpected ways.
A compliance team requested CIS L2 evidence for a SOC 2 Type II audit. The security team had been running OpenSCAP scans — but manually, on-demand, using slightly different profiles across teams, with no standard for how to store or compare results.
The audit found four problems:
1. Two instances had been scanned with CIS L1, not L2, despite being labeled “CIS L2”
2. Three instances hadn’t been scanned in over six months
3. The scan outputs from different teams were in different formats (HTML vs XML vs text)
4. Two instances showed “91% passing” and “89% passing” — with no documentation of whether those were acceptable thresholds or what the failing controls were
The audit took two weeks to resolve. The finding wasn’t a security failure — it was a documentation and process failure. But it consumed two weeks of engineering time and appeared in the audit report as a gap.
The root cause: compliance scanning was a manual step that produced inconsistent output in an inconsistent format.
How Automated OpenSCAP Compliance Works
Scanning is a stage of the build, not an afterthought you remember to run:
bakex build blueprints/ubuntu/22.04/cis-l1-aws.yaml
│
├─ Provisioning via aws
│
├─ Applying pre-hardening system configuration
│ (hostname, filesystem, users)
│
├─ Applying Ansible-Lockdown hardening roles
│
├─ Running OpenSCAP compliance scan
│ ├── benchmark: xccdf_org.ssgproject.content_benchmark_UBUNTU2204
│ ├── profile: ...content_profile_cis_level1_server
│ └── datastream: ssg-ubuntu2204-ds.xml
│
├─ Snapshotting golden image
│
└─ Image ready: ami-0a7f3c9e82d1b4c05
All three compliance identifiers come from the blueprint’s compliance block, and they are full
XCCDF strings rather than friendly names like cis-l1 — they’re handed to oscap unmodified, so
there is no name-mapping layer that can silently pick the wrong profile. That single detail
answers the audit question “which profile was actually used?” without anyone having to remember.
Ubuntu is a special case worth knowing: it ships no SCAP content package in the archive, so BakeX
downloads the matching datastream from a ComplianceAsCode release and checksum-verifies it rather
than failing or silently scanning nothing.
The A-F Grade Calculation
The grade is deliberately boring, and that is the point. BakeX does not invent a scoring model —
it takes OpenSCAP’s own XCCDF score and maps it to a letter:
def score_to_grade(score: float) -> str:
if score >= 90: return "A"
if score >= 75: return "B"
if score >= 60: return "C"
if score >= 40: return "D"
return "F"
| Grade | Score | Meaning |
|---|---|---|
| A | ≥ 90 | Production-ready, minimal exceptions |
| B | ≥ 75 | Acceptable with documented exceptions |
| C | ≥ 60 | Below standard — deploy with caution |
| D | ≥ 40 | Significant gaps — do not deploy to production |
| F | < 40 | Hardening failed |
The thresholds are fixed, not per-blueprint tunables. That is a defensible choice: a grade you can
adjust in the file being graded is not evidence, it’s decoration. If an A means ≥ 90 everywhere,
two teams’ grades are comparable without reading their blueprints — which was exactly the failure
in the audit story above.
What is configurable is when the build refuses to continue:
compliance:
benchmark: xccdf_org.ssgproject.content_benchmark_UBUNTU2204
profile: xccdf_org.ssgproject.content_profile_cis_level1_server
datastream: /usr/share/xml/scap/ssg/content/ssg-ubuntu2204-ds.xml
fail_on_findings: true # findings at/above the threshold fail the build
severity_threshold: medium # critical | high | medium | low
fail_on_findings with a severity_threshold is severity-based rather than score-based, which
tends to match how people actually reason about risk: one critical finding should block a release
even when 94% of rules pass. When it trips, the build ends in Status: failed, exit code 1, and
no image is snapshotted.
Where the Scan Surface Actually Lives
Worth being blunt about this, because it is the most common wrong assumption: there is no
bakex scan command. The CLI is two verbs — validate and build. Scanning, history, and
baseline comparison live in the web app and its HTTP API, because scan results need somewhere to
persist and something to render them.
Start the server and the whole surface is there:
bakex serve --port 8000
The auditor API is mounted at /api/auditor:
| Endpoint | What it does |
|---|---|
POST /api/auditor/scan-image |
Scan an image and return a job |
POST /api/auditor/scan-container |
Same, for a container image |
GET /api/auditor/jobs |
List scan jobs |
GET /api/auditor/jobs/{job_id} |
One job, with grade and severity counts |
GET /api/auditor/jobs/{job_id}/compare/{baseline_id} |
Diff a scan against a baseline |
GET /api/auditor/scan-image/{job_id}/report?fmt=… |
Export the report |
GET /api/auditor/scan-image/{job_id}/badge.svg |
Grade badge for a README |
SARIF Export
The report endpoint speaks three formats, selected by query parameter:
# Human-readable — printable HTML, print-to-PDF from the browser
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=html"
# Machine-readable job dict
curl "http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=json"
# SARIF 2.1.0 — the one that matters for CI
curl -o scan.sarif.json \
"http://localhost:8000/api/auditor/scan-image/$JOB/report?fmt=sarif"
SARIF 2.1.0 is the standard interchange format for security scan results, which means the OpenSCAP
findings land wherever your other scanners’ findings already land:
- GitHub Advanced Security — upload with
github/codeql-action/upload-sarif; findings appear in the Security tab, annotated on the PR - Azure DevOps — native SARIF viewer
- Splunk / SIEM — structured JSON, parseable as events
- AWS Security Hub — importable as findings via the Security Hub API
For audit purposes the SARIF file is the evidence artifact: it carries every rule result, the
profile that was used, and the timestamp. “91% passing” in a spreadsheet is a claim. A SARIF file
in the Security tab is a record.
The badge endpoint is the small touch that gets used most — badge.svg renders the letter grade,
so a repo’s README can show the compliance grade of the image it builds, next to the CI badge.
Drift: Comparing Against a Baseline
The comparison endpoint takes two job IDs — a current scan and a stored baseline — and reports the
delta, including the change in score:
curl "http://localhost:8000/api/auditor/jobs/$CURRENT/compare/$BASELINE"
That is the mechanism behind “what changed since we built this.” You scan the image at build time,
keep that job as the baseline, and re-scan later; the comparison tells you which rules moved and
which direction the score went. It is how you find the instance somebody modified “temporarily”
and never reverted.
The honest limitation: this compares scan jobs, so drift detection is as good as your discipline
about scanning on a schedule. Nothing re-scans your fleet for you.
What Controls Typically Block an A Grade
For Ubuntu 22.04 CIS L1 builds in most cloud environments, these are the controls that most commonly prevent an A grade:
| Control | Why it often fails | Fix |
|---|---|---|
1.1.7 /var/log/audit separate partition |
Cloud images don’t have separate volumes at build time | Add EBS volume, configure at launch |
| 1.6.1 AppArmor bootloader config | GRUB parameters not set correctly | Update /etc/default/grub, run update-grub |
| 3.1.1 Disable IPv6 | Cloud networking sometimes requires IPv6 | Override with documented reason if intentional |
| 5.2.21 SSH MaxStartups | Default sshd_config not updated | Add MaxStartups 10:30:60 to sshd_config |
| 6.1.10 World-writable files | Some package installations leave world-writable files | Post-install cleanup in Ansible role |
The first two (separate audit partition, AppArmor bootloader) are the most common A→B blockers and often require architecture decisions about how volumes are provisioned at launch versus build time.
Key Takeaways
- Automated OpenSCAP compliance means every image has a verified, reproducible grade generated by the same scanner with the same profile, before it’s ever deployed
- The grade is OpenSCAP’s own XCCDF score mapped to a fixed scale (A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40) — fixed on purpose, so grades from two teams are comparable without reading their blueprints
- The build gate is severity-based, not score-based:
fail_on_findingsplusseverity_thresholdblocks a release on one critical finding even when most rules pass - SARIF 2.1.0 export makes scan results importable into GitHub Advanced Security, Azure DevOps, SIEM, and audit tooling — the SARIF file is the evidence artifact
- Scanning and baseline comparison are HTTP API surfaces, not CLI commands; the CLI is
validateandbuild
What’s Next
Automated OpenSCAP compliance gives every image a verified grade before deployment. What EP04 left open is what happens after the grade is known — specifically, what prevents an engineer from deploying a C-grade image to production “just this once.”
The Pipeline API is the answer. EP05 covers the CI/CD compliance gate: POST /api/pipeline/scan fails the build if the image grade is below threshold. The unhardened image never reaches production — not because engineers are disciplined, but because the pipeline won’t let it through.
Next: CI/CD compliance gate — block unhardened images before they reach production
Get EP05 in your inbox when it publishes → linuxcent.com/subscribe