a poor mans qemu orchestrator

This replaces the earlier SSH/SCP-based flow with:

• server/ — a Rust HTTP server (attested-builder-server, built with axum + tokio) that runs as a systemd service on the AMD SEV-SNP host. It accepts a build job over HTTP, runs it via job_runner.sh (the same fuse2fs/QEMU logic that was previously invoked over SSH), and serves back logs + artifacts.

job_runner.sh is installed once on the host at deploy time — it is never uploaded per job. Only the five per-job build artifacts (VM_IMAGE/KERNEL/INITRD/CMDLINE/OVMF) are sent with each request.

1. Deployment

Prerequisites on the AMD host (already covered in earlier setup): the qemu-deploy-user account, kvm group membership, the /dev/sev udev rule, and fuse2fs/fusermount installed.

# Build
nix build

# Install
sudo mkdir -p /opt/attested-builder /var/lib/attested-builder
sudo cp result/bin/attested-builder-server job_runner.sh /opt/attested-builder/
sudo chmod +x /opt/attested-builder/job_runner.sh
sudo chown -R qemu-deploy-user:qemu-deploy-user /opt/attested-builder /var/lib/attested-builder

sudo cp ../systemd/attested-builder.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now attested-builder
sudo systemctl status attested-builder

Sanity check:

curl http://localhost:8443/healthz   # -> ok

Cross-compiling from a dev machine instead of building on the host:

rustup target add x86_64-unknown-linux-musl
cargo build --release --target x86_64-unknown-linux-musl
scp target/x86_64-unknown-linux-musl/release/attested-builder-server \
    qemu-deploy-user@amd-securityserver:/tmp/

Config (all via Environment= in the unit, or an EnvironmentFile)

Var	Default	Meaning
LISTEN_ADDR	:8443	HTTP listen address (:PORT shorthand for 0.0.0.0:PORT is accepted)
DATA_DIR	/var/lib/attested-builder	where job dirs/artifacts live
JOB_RUNNER_SCRIPT	/opt/attested-builder/job_runner.sh	path to the runner script
WORKERS	1	how many jobs run concurrently — leave at 1 unless you've confirmed the host can run more than one SEV-SNP guest at a time
DEFAULT_MEM / DEFAULT_CPUS / DEFAULT_TIMEOUT	36G / 8 / 900	used when the client doesn't override them
JOB_TTL_MINUTES	120	finished jobs a client never fetched/deleted are reaped after this long
AUTH_TOKEN	unset	see below
RUST_LOG	unset	e.g. info, debug — controls tracing log verbosity

2. Auth: off today, extensible later

Right now AUTH_TOKEN is unset, so the server accepts requests from anyone who can reach it — fine given it's on a trusted private network.

To turn on bearer-token auth (no code changes, no client-protocol changes): set AUTH_TOKEN in the systemd unit (ideally via EnvironmentFile=-/etc/attested-builder/auth-token.env so the secret isn't in the unit file itself), restart the service, and set the same value as AUTH_TOKEN in the GitLab CI variable for the client. Requests without a matching Authorization: Bearer <token> header get 401.

To move to mTLS later, the auth check is isolated in one function (check_auth in server/src/main.rs) and the listener is built in one place (main(), currently axum::serve over a plain TcpListener). Switching to client-cert auth means terminating TLS in front of the axum::Router — either with a reverse proxy (nginx/Caddy configured for ssl_verify_client) or in-process via a crate like axum-server with its RustlsConfig, requiring and verifying a client certificate, and reading the peer certificate in check_auth instead of (or in addition to) the bearer token. The rest of the server — job model, streaming, artifact handling — doesn't need to change.

3. GitLab CI usage

attested-build:
  stage: build
  variables:
    SERVER_URL: "http://amd-securityserver:8443"
    # AUTH_TOKEN: set as a masked/protected CI variable once auth is turned on
  script:
    - ./client/run_attested_builder.sh "$CI_PROJECT_DIR" ./build.sh ./output-paths.txt ./out
  artifacts:
    paths:
      - out/

The client script:

1. Builds the attested image locally with Nix (unchanged from before).
2. Computes the expected SEV-SNP measurement locally (unchanged).
3. Uploads VM_IMAGE/KERNEL/INITRD/CMDLINE/OVMF to the server as one multipart POST to /v1/jobs.
4. Streams /v1/jobs/{id}/logs?follow=true live — this is what shows up in the GitLab job log in real time.
5. Polls final /v1/jobs/{id} state.
6. Downloads /v1/jobs/{id}/artifacts (a .tar.gz) and extracts it into $OUTPUT_DIR.
7. DELETEs the job on the server (the TTL reaper is just a backstop for jobs a client never got to clean up, e.g. a killed CI job).
8. Runs the same local snp-verify-report attestation check as before — completely unchanged, since it only needs the fetched attestation-report.bin and the locally-computed measurement.

4. API reference

Method	Path	Purpose
POST	/v1/jobs	Submit a job. Multipart fields: vm_image, kernel, initrd, cmdline, ovmf (files); mem, cpus, timeout (optional form fields). Returns {"job_id": "...", "state": "queued"}.
GET	/v1/jobs/{id}	{"job_id", "state", "exit_code", "error"}. state is one of queued/running/succeeded/failed.
GET	/v1/jobs/{id}/logs?follow=true	Plain-text log, chunked. Sends existing output first, then live output until the job finishes (or follow=false for just a snapshot).
GET	/v1/jobs/{id}/artifacts	.tar.gz of the job's artifacts/ dir. 409 if the job hasn't finished yet.
DELETE	/v1/jobs/{id}	Deletes the job dir immediately.
GET	/healthz	Liveness check.

5. Implementation notes

• Job execution model: each job runs job_runner.sh as a child process via tokio::process::Command. A configurable number of worker tasks (WORKERS, default 1) pull from a shared queue — same single-flight-by-default posture as before, since SEV-SNP guest launches on one host generally shouldn't overlap unless you've verified otherwise.
• Log streaming: each job has a tokio::sync::broadcast channel for live output plus a one-shot "done" broadcast so /logs?follow=true requests know when to stop waiting. A late subscriber always gets the full job.log from disk first, then joins the live stream — small chance of one duplicated line at the seam, never a gap.
• Artifacts endpoint: builds the .tar.gz fully in memory before responding, rather than streaming it directly to the socket the way /logs does. Fine for the current artifact sizes (a build tarball, a couple of logs, an attestation report); if that stops being true, swap it for a channel-backed std::io::Write feeding an axum::body::Body::from_stream, the same pattern already used for /logs.

6. Files in this repo

  src/main.rs          # HTTP server: job queue, log streaming, artifact packaging
  Cargo.toml
  Cargo.lock
  job_runner.sh         # fuse2fs/QEMU logic, invoked locally by main.rs
  attested-builder.service

7. License

This project is licensed under the EUPL License. See the LICENSE file for details.