Skip to content

Add onctl restart: cold-boot a Firecracker VM from its disk, optionally on a new kernel - #1140

Merged
cdalar merged 1 commit into
mainfrom
fc-restart
Sep 26, 2026
Merged

cdalar merged 1 commit into
mainfrom
fc-restart

Conversation

@cdalar

@cdalar cdalar commented Sep 26, 2026

Copy link
Copy Markdown
Owner

Why

Firecracker chooses the kernel at boot, and onctl had no way to boot an existing VM again:

  • resume restores a snapshot, with the old kernel and memory inside it;
  • create over a dead record (e.g. after a host reboot) wipes the VM directory, including its rootfs.

So moving a VM to a new kernel, or recovering one whose VMM died, meant losing its disk. This came up when boxctl-vms started booting boxes on its own kernel (cdalar/boxctl-vms#99, #100): existing boxes, including multi-node k3s clusters, could only get it by being recreated.

What

  • cloud.Restarter (optional interface, like ImageLister) + RestartOptions{KernelImage}.
  • ProviderFC.Restart: best-effort guest filesystem sync, stop the VMM if it's alive, recreate the TAP device, and boot a fresh VMM against the VM's existing rootfs (and cache disk) with the same IP, MAC, vCPUs, memory and kernel command line.
    • --kernel-image swaps the kernel and is saved in the metadata, so later restarts keep it.
    • Works for running, dead (host reboot) and paused VMs (the snapshot is discarded: a cold boot can't resume it, and it would bring back the old kernel).
    • A missing kernel/rootfs fails before the VM is touched; a failed boot records the VM as dead instead of leaving stale "running" metadata.
  • Deploy and Restart share kernelArgs(), so a restarted VM gets exactly the network settings it was deployed with.
  • onctl restart <name> [--kernel-image <vmlinux>] [-f] (fc only; other providers get a clear "not supported").
  • TESTING-fc.md: manual test line.

Testing

  • New unit tests: running VM + new kernel (same disk/IP/MAC/cmdline, new kernel persisted), default kernel kept, dead VM, paused VM (snapshot removed, cold boot not restore), missing kernel leaves the VM running, boot failure marks dead, not found; command flags/args.
  • go test ./pkg/cloud/ ./internal/... pass, go vet, golangci-lint clean.
  • cmd: TestReadConfig_NoConfigDirectory and TestCreateFlagsBindToViper fail locally on main as well (unrelated; they seem to depend on the local ~/.onctl).
  • Not yet run against real Firecracker. Suggested on aimax: sudo ONCTL_CLOUD=fc onctl -c /root/.onctl/onctl.yaml restart <vm> -f --kernel-image /opt/fc/images/boxctl-kernel.vmlinux, then uname -r / check the disk content survived.

Follow-up (not in this PR)

boxctl-vms could expose this (e.g. "restart box" / automatic kernel upgrade on restart), since its agent already dispatches onctl commands.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WubXQDJdsuKoncpCSToz93

…onally on a new kernel

Firecracker picks the kernel at boot, and onctl had no way to boot an
existing VM again: Resume restores a snapshot (old kernel and memory
included), and Deploy over a dead record wipes the VM directory, rootfs and
all. So moving a VM to a new kernel -- or recovering one after a host reboot
killed its VMM -- meant losing its disk.

- cloud.Restarter (optional interface, like ImageLister) + RestartOptions.
- ProviderFC.Restart: best-effort guest fs sync, stop the VMM if alive,
  recreate the TAP device and boot a fresh VMM against the VM's existing
  rootfs (and cache disk) with the same IP, MAC, vCPUs, memory and kernel
  command line. --kernel-image swaps the kernel and is recorded in the
  metadata so later restarts keep it. Works for running, dead (host reboot)
  and paused VMs (the snapshot is discarded). A missing kernel/rootfs fails
  before touching the VM; a failed boot records the VM as dead.
- Deploy and Restart share kernelArgs(), so a restarted VM comes back with
  exactly the network settings it was deployed with.
- `onctl restart <name> [--kernel-image path] [-f]`, fc only.

Tests: Restart on a running VM with a new kernel, default kernel, dead VM,
paused VM, missing kernel, boot failure, not found; command flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WubXQDJdsuKoncpCSToz93
@codecov

codecov Bot commented Sep 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 65.71429% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 34.54%. Comparing base (9d31e7a) to head (b9cabbf).

Files with missing lines Patch % Lines
pkg/cloud/fc.go 76.47% 20 Missing ⚠️
cmd/restart.go 20.00% 16 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1140      +/-   ##
==========================================
+ Coverage   33.99%   34.54%   +0.54%     
==========================================
  Files          50       51       +1     
  Lines        5509     5610     +101     
==========================================
+ Hits         1873     1938      +65     
- Misses       3636     3672      +36     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cdalar
cdalar merged commit 0e0c6de into main Sep 26, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址