From 7c6443c4161fdf4980d5049ca8400ece23221849 Mon Sep 17 00:00:00 2001 From: Anastassios Nanos Date: Sat, 26 Sep 2026 00:37:48 +0300 Subject: [PATCH] fix(install): Stop before the download when sudo needs a password For a user install on a host that is not prepared, install.sh printed the missing host settings as a warning and went on. It downloaded and unpacked the bundle, about 488 MB, and then brig-rootless-setup.sh asked for sudo. A user without sudo was left at `[sudo] password for :`, or without a terminal, with a failed install. brig's install guide says a user install asks for sudo at no point. The preflight now covers every sudo call the setup can make: the device grant, loading kvm and vhost_vsock, loading vhost_vsock at boot, the AppArmor profile, and home directories an earlier sudo run left root-owned. When any of them is missing and `sudo -n true` fails, it stops before the download. It prints what is missing and one `sudo sh` block that fixes it for this user and this prefix. An admin runs that block, and the next run of the installer needs no sudo. With sudo that runs without a password, nothing changes: the setup makes the settings at the end. Missing packages and a missing subuid range stop the install as before, since the setup never made them. The block now covers those too, and the subuid range it offers starts past every range already handed out. The setup no longer asks sudo to load a module the kernel does not have. That was a password prompt for nothing. This builds on the per-user grant file. The block writes 99-brig-kvm-.rules, and the setup reads the grant back from the devices. tests/install-preflight.sh runs install.sh against a stub sudo, getfacl, stat and sysctl, and points it at a bundle that does not exist. Without sudo, it checks that the install stops before the fetch, writes nothing under $HOME, and prints a valid block with the udev rule and the AppArmor profile for this user. With sudo, it checks that the install goes on. On a prepared host, it checks that no sudo is asked for. Against the previous install.sh it fails the first check: the install goes on to fetch the bundle. Checked live on an Ubuntu 24.04 host, with a user who has no sudo. The previous installer unpacked 488M into their home and then failed at sudo. This one stopped with the block and left nothing in $HOME. An admin ran the block as printed, and the user's next run installed without a single sudo call. A microVM then booted with its monitor running as that user. Signed-off-by: Anastassios Nanos --- docs/rootless.md | 30 +++++- install.sh | 194 +++++++++++++++++++++++++++++++------ scripts/build-bundle.sh | 9 +- tests/install-preflight.sh | 122 +++++++++++++++++++++++ 4 files changed, 320 insertions(+), 35 deletions(-) create mode 100755 tests/install-preflight.sh diff --git a/docs/rootless.md b/docs/rootless.md index f5554c8..ae24512 100644 --- a/docs/rootless.md +++ b/docs/rootless.md @@ -168,10 +168,32 @@ only. And the tree is one copy per user where a root install is shared, so with root and several users, `brig-ctl rootless` against one shared install is the better trade. -The host prerequisites above still apply. `install.sh --user` reports both -before it unpacks anything, and the rootless setup asks sudo once to fix them. -The AppArmor profile names this prefix's rootlesskit, so a profile written for -`/var/lib` does not cover a tree in `$HOME`. +The host prerequisites above still apply, and `install.sh --user` checks them +before it downloads anything. When sudo runs without a password, it goes on and +the rootless setup makes them with sudo at the end. When it does not, the +installer stops there. It prints the commands that prepare the host for this +user and this prefix, as one block for root: + +```console +$ curl -fsSL https://raw.githubusercontent.com/NOFireAI/brig-standalone-linux/main/install.sh | sh - +[brig-install] ERROR: this host is not set up for a rootless brig: + no access to /dev/kvm from inside the user namespace + no access to /dev/vhost-vsock from inside the user namespace + + Nothing was downloaded: sudo cannot run here without a password. + Run the commands below as root, or ask an admin to, and then run this + installer again. It needs no sudo after that. + +sudo sh -eu <<'BRIG_ROOT' +cat > /etc/udev/rules.d/99-brig-kvm-alice.rules <<'RULE' +... +BRIG_ROOT +``` + +Once an admin has run them, the same installer runs to the end without sudo. +A user with a sudo password can run `sudo -v` first instead. The AppArmor +profile names this prefix's rootlesskit, so a profile written for `/var/lib` +does not cover a tree in `$HOME`. Building for a prefix directly is still an option, for an air-gapped host or a layout of your own: diff --git a/install.sh b/install.sh index c6b1d0d..61acfba 100755 --- a/install.sh +++ b/install.sh @@ -254,60 +254,194 @@ verify_system() { command -v systemctl >/dev/null 2>&1 || HAVE_SYSTEMD=false } -# What a user install cannot do for itself, in two classes. Nothing here can -# fix the first, so the install stops; brig-rootless-setup.sh asks sudo for the -# second, so those are reported and left to it. Each is a confusing failure -# somewhere else if it goes unchecked: rootlesskit dies on its own re-exec -# without the AppArmor profile, KVM_CREATE_VM returns EPERM without access to -# the device, and newuidmap is setuid-root. +# What a user install cannot do for itself, in three classes. Each is a +# confusing failure somewhere else if it goes unchecked: rootlesskit dies on its +# own re-exec without the AppArmor profile, KVM_CREATE_VM returns EPERM without +# access to the device, and newuidmap is setuid-root. +# +# - No root can fix a missing login session or systemd, so those stop the +# install. +# - Packages and a subuid range are root's, and brig-rootless-setup.sh does +# not make them, so those stop the install too. +# - The rest brig-rootless-setup.sh makes with sudo at the end of the +# install. Without a sudo that runs without a password, it would stop at a +# prompt after the whole bundle is downloaded and unpacked. So the install +# stops here instead. +# +# Whatever needs root is printed as one block of commands for this user and +# this prefix. An admin runs it once, and the next install needs no sudo. The +# checks and the commands mirror brig-rootless-setup.sh; keep them in step. verify_rootless_prereqs() { + me="$(id -un)" miss="" - once="" + need="" + setup_can=true + fix="$TMP_DIR/root-commands.sh" + : > "$fix" + [ -d "${XDG_RUNTIME_DIR:-/run/user/$(id -u)}" ] \ || miss="$miss\n no ${XDG_RUNTIME_DIR:-/run/user/$(id -u)}: log in as this user first" [ "$HAVE_SYSTEMD" = "true" ] \ || miss="$miss\n no systemctl: a user install needs a systemd user session" + + pkgs="" command -v newuidmap >/dev/null 2>&1 \ - || miss="$miss\n newuidmap is missing: sudo apt install uidmap" + || { need="$need\n newuidmap is missing"; pkgs="$pkgs uidmap"; } command -v setfacl >/dev/null 2>&1 \ - || miss="$miss\n setfacl is missing: sudo apt install acl" - grep -q "^$(id -un):" /etc/subuid 2>/dev/null && grep -q "^$(id -un):" /etc/subgid 2>/dev/null \ - || miss="$miss\n no subuid range: sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $(id -un)" + || { need="$need\n setfacl is missing"; pkgs="$pkgs acl"; } + if [ -n "$pkgs" ]; then + setup_can=false + echo "DEBIAN_FRONTEND=noninteractive apt-get install -y$pkgs > "$fix" + fi + + # A range that overlaps another user's maps both users' containers onto the + # same host uids, so the range offered starts past every one handed out. + if ! grep -q "^$me:" /etc/subuid 2>/dev/null || ! grep -q "^$me:" /etc/subgid 2>/dev/null; then + setup_can=false + need="$need\n no subuid range for $me" + first=$(cat /etc/subuid /etc/subgid 2>/dev/null \ + | awk -F: '$2 + $3 > m {m = $2 + $3} END {print (m > 100000 ? m : 100000)}') + last=$((first + 65535)) + echo "usermod --add-subuids $first-$last --add-subgids $first-$last $me" >> "$fix" + fi + + # The profile names the binary by path, so the one a /var/lib install left + # behind does not cover a tree in $HOME. + if [ "$(sysctl -n kernel.apparmor_restrict_unprivileged_userns 2>/dev/null || echo 0)" = "1" ] \ + && ! grep -rqsF "$BIN_DIR/rootlesskit" /etc/apparmor.d/ 2>/dev/null; then + need="$need\n no AppArmor profile for $BIN_DIR/rootlesskit" + profile="/etc/apparmor.d/$(printf '%s' "${BIN_DIR#/}/rootlesskit" | tr / .)" + cat >> "$fix" < '$profile' <<'PROF' +abi , +include + +$BIN_DIR/rootlesskit flags=(unconfined) { + userns, + include if exists +} +PROF +apparmor_parser -r '$profile' +FIX + fi # Readable here is the wrong question. The monitor runs inside a user # namespace where this user's supplementary groups are gone, so the kvm # group that makes /dev/kvm openable at this prompt reaches nothing in # there -- testing r/w passes on exactly the hosts that then fail to boot a - # sandbox. Look for what the setup actually grants: this user by name, or a - # mode that covers everyone. + # sandbox. Look for what the setup grants: this user by name, or a mode + # that covers everyone. A module that is not loaded has no device yet, and + # its device needs the grant once it has one. + grant=false for d in kvm vhost-vsock; do [ -e "/dev/$d" ] || continue - getfacl -p "/dev/$d" 2>/dev/null | grep -q "^user:$(id -un):rw" && continue + getfacl -p "/dev/$d" 2>/dev/null | grep -q "^user:$me:rw" && continue [ "$(stat -c %a "/dev/$d" 2>/dev/null || echo 0)" = 666 ] && continue - once="$once\n no access to /dev/$d from inside the user namespace" + need="$need\n no access to /dev/$d from inside the user namespace" + grant=true + done + load="" + for m in kvm vhost_vsock; do + [ -d "/sys/module/$m" ] && continue + module_exists "$m" || continue + need="$need\n the $m module is not loaded" + load="$load $m" + grant=true + done + if [ "$grant" = true ]; then + cat >> "$fix" < /etc/udev/rules.d/99-brig-kvm-$me.rules <<'RULE' +# brig: a rootless brig runs the VMM as the invoking user, inside a user +# namespace that drops supplementary groups, so the kvm group never reaches +# the monitor. Grant the user directly -- narrower than mode 0666, and +# reapplied on every event for these devices, which a one-shot setfacl is not. +# An image whose user is not root needs --open-devices instead. +KERNEL=="kvm", SUBSYSTEM=="misc", MODE="0660", RUN+="/usr/bin/setfacl -m u:$me:rw /dev/kvm" +KERNEL=="vhost-vsock", SUBSYSTEM=="misc", MODE="0660", RUN+="/usr/bin/setfacl -m u:$me:rw /dev/vhost-vsock" +RULE +udevadm control --reload-rules +FIX + fi + for m in $load; do + echo "modprobe $m" >> "$fix" + done + # vhost_vsock does not autoload, so the host is told to load it at boot. + case " $load " in *" vhost_vsock "*) vsock=true ;; *) vsock=false ;; esac + [ -d /sys/module/vhost_vsock ] && vsock=true + if [ "$vsock" = true ] && ! grep -qsx 'vhost_vsock' /etc/modules-load.d/*.conf /etc/modules; then + need="$need\n vhost_vsock is not loaded at boot" + cat >> "$fix" <<'FIX' +cat > /etc/modules-load.d/brig.conf <<'CONF' +# brig: the vsock device the monitor gives the guest. Written by +# brig-rootless-setup.sh; the matching udev rule grants access to it. +vhost_vsock +CONF +FIX + fi + if [ "$grant" = true ]; then + cat >> "$fix" <<'FIX' +udevadm trigger --subsystem-match=misc --sysname-match=kvm +udevadm trigger --subsystem-match=misc --sysname-match=vhost-vsock +udevadm settle --timeout=10 || true +FIX + fi + + # Anything an earlier sudo run left root-owned in this home stops an + # unprivileged brig, and the setup takes it back with sudo. + for d in "$HOME/brig" "$HOME/.brig" "$HOME/.sigstore" "${XDG_CONFIG_HOME:-$HOME/.config}/brig"; do + [ -e "$d" ] || continue + [ "$(stat -c %u "$d" 2>/dev/null || echo -1)" = "$(id -u)" ] && continue + need="$need\n $d is not owned by $me" + echo "chown -R $(id -u):$(id -g) '$d'" >> "$fix" done - # The profile names the binary by path, so the one a /var/lib install left - # behind does not cover a tree in $HOME. - if [ "$(sysctl -n kernel.apparmor_restrict_unprivileged_userns 2>/dev/null || echo 0)" = "1" ] \ - && ! grep -rqsF "$BIN_DIR/rootlesskit" /etc/apparmor.d/ 2>/dev/null; then - once="$once\n no AppArmor profile for $BIN_DIR/rootlesskit" - fi - - if [ -n "$miss" ]; then - # shellcheck disable=SC2059 - printf "[brig-install] ERROR: this host is not set up for a rootless brig:$miss\n\n See docs/rootless.md.\n" >&2 - exit 1 + if [ -z "$miss" ] && [ -z "$need" ]; then + return 0 fi - if [ -n "$once" ]; then + if [ -z "$miss" ] && [ "$setup_can" = true ] && sudo -n true 2>/dev/null; then # Counted, not assumed: this said "two" whatever it had found, which # reads as a second thing the reader has missed. - n=$(printf '%b' "$once" | grep -c '^ ') + n=$(printf '%b' "$need" | grep -c '^ ') if [ "$n" = 1 ]; then what="one host setting is"; else what="$n host settings are"; fi # shellcheck disable=SC2059 - printf "[brig-install] WARNING: $what still missing:$once\n" >&2 - warn "brig-rootless-setup.sh will ask for sudo once to set those up" + printf "[brig-install] WARNING: $what still missing:$need\n" >&2 + warn "brig-rootless-setup.sh will make them with sudo" + return 0 + fi + + _break_bar + # shellcheck disable=SC2059 + printf "[brig-install] ERROR: this host is not set up for a rootless brig:$miss$need\n\n" >&2 + if [ -n "$need" ]; then + if [ "$setup_can" = true ]; then + why="sudo cannot run here without a password" + else + why="the installer does not install packages or add subuid ranges" + fi + cat >&2 <&2 + echo " makes these itself." >&2 + fi fi + echo " See docs/rootless.md." >&2 + exit 1 +} + +# Whether the running kernel has the module, loaded or not. Without modinfo +# there is no telling, and the answer that leads to a working host is yes. +module_exists() { + command -v modinfo >/dev/null 2>&1 || return 0 + modinfo "$1" >/dev/null 2>&1 } # An existing prefix is only ours if we stamped it. diff --git a/scripts/build-bundle.sh b/scripts/build-bundle.sh index dedc990..ef8cb6c 100755 --- a/scripts/build-bundle.sh +++ b/scripts/build-bundle.sh @@ -420,8 +420,15 @@ if [ "$OPEN_DEVICES" != true ]; then fatal "setfacl is missing (apt install acl); the device grant is made with it" fi fi +# A kernel without the module has nothing to load, and asking sudo to try is a +# password prompt for nothing. install.sh makes the same test before it +# downloads anything. for m in kvm vhost_vsock; do - [ -d "/sys/module/$m" ] || sudo modprobe "$m" >/dev/null 2>&1 || true + [ -d "/sys/module/$m" ] && continue + if command -v modinfo >/dev/null 2>&1 && ! modinfo "$m" >/dev/null 2>&1; then + continue + fi + sudo modprobe "$m" >/dev/null 2>&1 || true done # Loading it now only covers this boot. Nothing asks for vhost_vsock again on diff --git a/tests/install-preflight.sh b/tests/install-preflight.sh new file mode 100755 index 0000000..244c331 --- /dev/null +++ b/tests/install-preflight.sh @@ -0,0 +1,122 @@ +#!/bin/sh +# +# Run install.sh as a user install against stubs and check where it stops when +# the host is not prepared. The bundle it is pointed at does not exist, so a +# run that gets past the preflight fails on the fetch, and says so. +# +# Needs a normal user with a subuid range, on Linux. + +set -eu + +here="$(cd "$(dirname "$0")/.." && pwd)" +user="$(id -un)" + +skip() { echo "SKIP: $*"; exit 0; } +fail() { echo "FAIL: $*" >&2; exit 1; } +ok() { echo "ok - $*"; } + +[ "$(uname -s)" = Linux ] || skip "install.sh installs on Linux only" +[ "$(id -u)" -ne 0 ] || skip "a user install refuses root; run this as a normal user" +grep -q "^$user:" /etc/subuid 2>/dev/null || skip "no subuid range for $user" +grep -q "^$user:" /etc/subgid 2>/dev/null || skip "no subgid range for $user" + +T="$(mktemp -d)" +trap 'rm -rf "$T"' EXIT +mkdir -p "$T/stub" "$T/home" "$T/run" + +# sudo: only -n true can succeed, and only when the test says sudo works. +cat > "$T/stub/sudo" <> "$T/sudo.log" +[ "\$*" = "-n true" ] && [ -f "$T/sudo-ok" ] && exit 0 +exit 1 +STUB + +# getfacl and sysctl answer from files the test sets. +cat > "$T/stub/getfacl" < "$T/stub/sysctl" < "$T/stub/stat" <<'STUB' +#!/bin/sh +if [ "$1" = -c ] && [ "$2" = %a ]; then + case "$3" in /dev/*) echo 660; exit 0 ;; esac +fi +exec /usr/bin/stat "$@" +STUB + +# modinfo: no module is missing but loadable, so the result does not depend on +# what this kernel has loaded. +printf '#!/bin/sh\nexit 1\n' > "$T/stub/modinfo" +for b in newuidmap setfacl systemctl; do + printf '#!/bin/sh\nexit 0\n' > "$T/stub/$b" +done +chmod 0755 "$T/stub"/* + +run_install() { + : > "$T/sudo.log" + set +e + env -u XDG_CONFIG_HOME -u XDG_DATA_HOME \ + HOME="$T/home" XDG_RUNTIME_DIR="$T/run" PATH="$T/stub:$PATH" \ + INSTALL_BRIG_BUNDLE="$T/no-such-bundle.tar.gz" \ + sh "$here/install.sh" > "$T/out.log" 2>&1 + rc=$? + set -e +} + +prefix="$T/home/.local/share/brig/data" + +# 1. Not prepared, and no sudo: stop before anything is fetched, and print the +# root commands for this user and this prefix. +touch "$T/apparmor" +run_install +[ "$rc" -ne 0 ] || fail "the install succeeded on a host that is not prepared" +if grep -q "no-such-bundle" "$T/out.log"; then + cat "$T/out.log" >&2 + fail "the install went on to fetch the bundle" +fi +[ ! -e "$T/home/.local/share/brig" ] || fail "the install wrote $T/home/.local/share/brig" +grep -q "Nothing was downloaded" "$T/out.log" || { cat "$T/out.log" >&2; fail "no reason given"; } +awk '/^sudo sh -eu <<.BRIG_ROOT.$/ {f=1; next} /^BRIG_ROOT$/ {f=0} f' "$T/out.log" > "$T/root.sh" +[ -s "$T/root.sh" ] || { cat "$T/out.log" >&2; fail "no root commands printed"; } +sh -n "$T/root.sh" || fail "the root commands are not valid sh" +profile="/etc/apparmor.d/$(printf '%s' "${prefix#/}/bin/rootlesskit" | tr / .)" +if ! grep -qF "apparmor_parser -r '$profile'" "$T/root.sh" \ + || ! grep -qF "$prefix/bin/rootlesskit flags=(unconfined) {" "$T/root.sh"; then + cat "$T/root.sh" >&2 + fail "no AppArmor profile for $prefix/bin/rootlesskit" +fi +if [ -e /dev/kvm ]; then + grep -q "^cat > /etc/udev/rules.d/99-brig-kvm-$user.rules" "$T/root.sh" \ + || { cat "$T/root.sh" >&2; fail "no udev rule for $user"; } + grep -q "u:$user:rw /dev/kvm" "$T/root.sh" || fail "the udev rule does not grant $user" +fi +ok "without sudo the install stops before the download and prints the root commands" + +# 2. Not prepared, and sudo runs without a password: go on, and leave the +# host settings to the rootless setup. +touch "$T/sudo-ok" +run_install +grep -q "will make them with sudo" "$T/out.log" || { cat "$T/out.log" >&2; fail "no warning"; } +grep -q "no-such-bundle" "$T/out.log" || { cat "$T/out.log" >&2; fail "the install did not go on"; } +ok "with sudo the install goes on to the download" +rm -f "$T/sudo-ok" + +# 3. Prepared by an admin, and no sudo: go on without asking for it. +if [ -d /sys/module/vhost_vsock ] && ! grep -qsx vhost_vsock /etc/modules-load.d/*.conf /etc/modules; then + skip "vhost_vsock is loaded here but not at boot, which a prepared host has" +fi +rm -f "$T/apparmor" +touch "$T/granted" +run_install +grep -q "no-such-bundle" "$T/out.log" || { cat "$T/out.log" >&2; fail "a prepared host did not get past the preflight"; } +[ ! -s "$T/sudo.log" ] || { cat "$T/sudo.log" >&2; fail "a prepared host was asked for sudo"; } +ok "a prepared host needs no sudo"