From de7f3f12fd80387c7493452a00fad2b36acf63ad Mon Sep 17 00:00:00 2001 From: Zhe Yuan Date: Sat, 5 Sep 2026 20:31:26 -0400 Subject: [PATCH] Rewrite the prepare scripts for current OpenWrt and more distros prepare-openwrt-env.sh - Dispatch on the package manager instead of guessing from the distro version: apt, dnf/yum, pacman, zypper and apk are all supported. - Probe every package against the running release and skip the ones that no longer exist. Previously a single dropped package (on Ubuntu 26.04 arm64, gcc-multilib-s390x-linux-gnu) failed the whole apt command and set -e aborted the script, so nothing was installed at all. - Align the package lists with the official build system guide, adding libelf-dev, python3-dev, bc, xsltproc, zstd, u-boot-tools and others. - Make Go opt-in via --with-go rather than forced on arm64. - Add -y/--yes, -n/--dry-run and -h/--help; allow running as root. prepare-openwrt.sh - Split read-only discovery from mutation. Versions are resolved over git ls-remote before anything is moved, so aborting at the prompt leaves the helper checkout untouched. - Resume an interrupted first run instead of failing forever: a root with .git but no source tree was previously unrecoverable without a manual rm -rf. Also restore the worktree when HEAD already points at the target ref, where checkout is a no-op. - Detect the relocated layout by content rather than directory name, so renaming the root no longer triggers a second, nested relocation. - Replace the staged in-place relocation with a resumable content move, removing the window that could strand the checkout in a hidden dir. - Filter release candidates before sorting; sort -V orders v25.12.0-rc1 after v25.12.0, so the newest stable tag could be an rc. - Follow upstream: snapshots use main (master as fallback) and the dead -SNAPSHOT tag lookup is gone. - Use a partial clone (--filter=blob:none) by default. - Guard version switches on a dirty tree and back up .config first. - Add /helper/ to .git/info/exclude and warn that git clean -xdff still deletes it, since -x bypasses ignore rules by design. - Leave a symlink at the old checkout path so shells and editors whose working directory still points there keep working. - Add --stable/--branch/--snapshot plus -y, -n, --force, --full-clone, --allow-rc, --root, --skip-feeds, --skip-defconfig, --repo-url and --no-compat-link. Co-Authored-By: Claude Opus 5 --- README.md | 46 +- prepare-openwrt-env.sh | 420 ++++++++++---- prepare-openwrt.sh | 1213 +++++++++++++++++++++++++++++++--------- 3 files changed, 1303 insertions(+), 376 deletions(-) diff --git a/README.md b/README.md index 52f9ec8..9aa3840 100644 --- a/README.md +++ b/README.md @@ -5,14 +5,23 @@ This repository contains shell scripts that streamline preparing, configuring, a ## What’s included - prepare-openwrt-env.sh - - Installs required build dependencies via apt on Ubuntu 22.04/24.04/26.04 and Debian 12/13+. - - Handles x86_64 and ARM64 differences (Python/distutils, multilib notes, Go on ARM64). + - Installs the build dependencies listed by the official OpenWrt build system guide. + - Supports apt (Debian/Ubuntu/Mint), dnf/yum (Fedora/RHEL/Rocky/Alma), pacman (Arch/Manjaro), zypper (openSUSE) and apk (Alpine). + - Probes each package against the running release and skips the ones that no longer exist, so a renamed or dropped package cannot abort the whole install. + - Handles x86_64 and ARM64 differences (native vs. cross multilib). + - Options: `-y/--yes` (no prompt), `-n/--dry-run` (only print), `--with-go` (also install the Go toolchain). - prepare-openwrt.sh - Clones or updates the OpenWrt source repo in the top-level `openwrt/` root. - - Lets you select Stable, Beta, or Snapshot versions interactively. + - Lets you select a stable tag, a release branch, or the snapshot (`main`) interactively, or non-interactively via flags. - Uses the script location as the source of truth, so running it from another current working directory is supported. - - On first run from a helper checkout, relocates that checkout to `openwrt/helper/`. + - On first run from a helper checkout, relocates that checkout to `openwrt/helper/` and leaves a symlink behind at the old path. + - Prints a plan and asks for confirmation; nothing on disk is touched before you confirm. + - Uses a partial clone (`--filter=blob:none`) by default, so the first fetch is far smaller than full history. + - Resumes automatically if a previous run was interrupted. - Updates feeds and runs make defconfig. + - Options: `--stable []`, `--branch `, `--snapshot`, `-y/--yes`, `-n/--dry-run`, + `-f/--force`, `--full-clone`, `--allow-rc`, `--root `, `--skip-feeds`, + `--skip-defconfig`, `--repo-url `, `--no-compat-link`, `-h/--help`. - add-external-repos.sh - Clones or updates additional package repositories into package/. - Currently includes luci-app-netspeedtest and luci-app-easytier (adjust REPOS as needed). @@ -30,20 +39,24 @@ This repository contains shell scripts that streamline preparing, configuring, a - Fixes MAX macro naming collisions in multiple DAHDI modules for OpenWrt 24.10.4 builds. ## System requirements -- OS: Ubuntu 22.04/24.04/26.04 or Debian 12/13+ (others may work with adjustments) +- OS: Debian/Ubuntu, Fedora/RHEL, Arch, openSUSE or Alpine (the other scripts are developed and tested on Debian/Ubuntu) - Architectures: x86_64/amd64 and aarch64/arm64 - Tools: git, wget, curl, bash - Internet access for feeds and package downloads -- Optional: Go (required by some OpenWrt packages; auto-configured on ARM64 by prepare-openwrt-env.sh; configurable via update-go-path.sh) +- Optional: Go (required by some OpenWrt packages; install it with `./prepare-openwrt-env.sh --with-go`; configurable via update-go-path.sh) +- Note: with the default partial clone, switching to another tag or branch later fetches missing blobs on demand and therefore needs network access ## Quick start 1) Prepare the host machine - Run: ./prepare-openwrt-env.sh - This installs compilers, headers, Python tooling, and other build prerequisites. +- Preview without touching the system: ./prepare-openwrt-env.sh -n 2) Get OpenWrt sources - Run: ./prepare-openwrt.sh -- Choose Stable, Beta, or Snapshot. +- Choose a stable tag, a release branch, or the snapshot. +- Non-interactive: ./prepare-openwrt.sh --stable 25.12.5 -y +- Preview only, changes nothing: ./prepare-openwrt.sh --stable -n - The script: - Reuses or creates the top-level OpenWrt root at `./openwrt`. - Moves this helper checkout to `./openwrt/helper` on first run. @@ -120,11 +133,26 @@ This repository contains shell scripts that streamline preparing, configuring, a - Reuse or create `/openwrt/` as the OpenWrt source root. - Move the entire helper checkout, including its `.git`, to `/openwrt/helper/`. - Initialize or update the OpenWrt source tree directly in `/openwrt/`. -- If you later rerun `openwrt/helper/prepare-openwrt.sh`, it reuses the parent `openwrt/` directory and does not move directories again. + - Leave a symlink at the original checkout path pointing to `/openwrt/helper/`, so shells, editors and agent sessions whose working directory is still the old path keep working. Disable with `--no-compat-link`. +- If you later rerun `prepare-openwrt.sh` β€” from `openwrt/helper/` or through the symlink at the old path β€” it reuses the parent `openwrt/` directory and does not move anything again. +- The already-relocated layout is detected by content (the OpenWrt tree or its `origin` remote), not by directory name, so renaming `openwrt/` to something else does not trigger a second relocation. +- If a run is interrupted after the repository was created but before the source tree was checked out, the next run detects that state, reports `resuming it`, and completes. No manual cleanup is needed. - If the script is already located in a real OpenWrt source root, it treats that directory as the source root and does not relocate it just because the directory name is `openwrt`. +- Use `--root ` to point at a different OpenWrt root in any ambiguous situation. + +### Known hazard: `git clean -xdff` +`helper/` lives inside the OpenWrt Git worktree but is not tracked by it. The script +adds `/helper/` to `openwrt/.git/info/exclude`, which keeps `git status` clean and stops +`git add -A` from staging it β€” but that does **not** protect it from `git clean -xdff`: +`-x` deliberately includes ignored files and `-ff` removes Git's refusal to recurse into +a nested repository. Running `git clean -xdff` in the OpenWrt root will delete the helper +checkout including its `.git`. Use this instead: + + git clean -xdf -e /helper ## Security and safety -- Scripts use set -e to stop on errors. +- Scripts use set -euo pipefail to stop on errors. +- prepare-openwrt.sh backs up `.config` to `.config..bak` before switching versions, and refuses to switch a tree with modified tracked files unless you pass --force. - They modify .config and write under package/ and feeds/ inside your OpenWrt tree. - Backup your .config if you need a known baseline. diff --git a/prepare-openwrt-env.sh b/prepare-openwrt-env.sh index a3daa50..d341052 100755 --- a/prepare-openwrt-env.sh +++ b/prepare-openwrt-env.sh @@ -1,154 +1,370 @@ #!/bin/bash -# Automatically prepare OpenWRT build dependencies for Debian/Ubuntu -# Supports both x86_64 and arm64 -# Detects Ubuntu 22.04, 24.04, 26.04, Debian 12/13 and newer +# Prepare the OpenWrt build dependencies. +# +# Supported distributions: +# - Debian / Ubuntu / Linux Mint / Raspberry Pi OS (apt) +# - Fedora / RHEL / CentOS / Rocky / AlmaLinux (dnf, yum) +# - Arch / Manjaro / EndeavourOS (pacman) +# - openSUSE / SLES (zypper) +# - Alpine (apk) +# +# Package lists follow https://openwrt.org/docs/guide-developer/toolchain/install-buildsystem +# Packages that do not exist on the running release are skipped instead of +# aborting the whole installation. +# +# Usage: ./prepare-openwrt-env.sh [-y] [-n] [--with-go] +# -y, --yes do not ask for confirmation +# -n, --dry-run only print what would be installed +# --with-go also install the Go toolchain +# # Author: Zhe Yuan -set -e +set -euo pipefail umask 022 +ASSUME_YES=false +DRY_RUN=false +WITH_GO=false + +while [ $# -gt 0 ]; do + case "$1" in + -y|--yes) ASSUME_YES=true ;; + -n|--dry-run) DRY_RUN=true ;; + --with-go) WITH_GO=true ;; + -h|--help) + sed -n '2,20p' "$0" | sed 's/^# \{0,1\}//' + exit 0 + ;; + *) + echo "[ERROR] Unknown option: $1" >&2 + exit 1 + ;; + esac + shift +done + echo "========================================" -echo " OpenWRT Build Environment Preparation" +echo " OpenWrt Build Environment Preparation" echo "========================================" # --- Detect architecture --- -ARCH=$(uname -m) -echo "[INFO] Detected architecture: $ARCH" +ARCH="$(uname -m)" +echo "[INFO] Architecture: $ARCH" -# --- Detect OS version --- -if [ -f /etc/os-release ]; then +# --- Detect distribution --- +DISTRO="unknown" +DISTRO_LIKE="" +VERSION_ID="unknown" +PRETTY_NAME="unknown" + +if [ -r /etc/os-release ]; then + # shellcheck disable=SC1091 . /etc/os-release - DISTRO="$ID" + DISTRO="${ID:-unknown}" + DISTRO_LIKE="${ID_LIKE:-}" VERSION_ID="${VERSION_ID:-unknown}" + PRETTY_NAME="${PRETTY_NAME:-$DISTRO $VERSION_ID}" else - echo "[ERROR] Cannot detect system version (no /etc/os-release)" + echo "[ERROR] Cannot detect the distribution (no /etc/os-release)." >&2 exit 1 fi -echo "[INFO] Detected distribution: $DISTRO $VERSION_ID" +echo "[INFO] Distribution: $PRETTY_NAME (id=$DISTRO)" -# --- Version helpers --- -version_ge() { - local current="${1:-unknown}" - local minimum="$2" - - if [[ "$current" == "unknown" ]]; then - return 1 +# --- Detect package manager --- +PKG_MGR="" +for candidate in apt-get dnf yum pacman zypper apk; do + if command -v "$candidate" >/dev/null 2>&1; then + PKG_MGR="$candidate" + break fi +done - if command -v dpkg >/dev/null 2>&1; then - dpkg --compare-versions "$current" ge "$minimum" +if [ -z "$PKG_MGR" ]; then + echo "[ERROR] No supported package manager found." >&2 + echo "[ERROR] Supported: apt-get, dnf, yum, pacman, zypper, apk." >&2 + exit 1 +fi + +echo "[INFO] Package manager: $PKG_MGR" + +# --- Privilege escalation --- +SUDO="" +if [ "$(id -u)" -ne 0 ]; then + if command -v sudo >/dev/null 2>&1; then + SUDO="sudo" else - [[ "$(printf '%s\n%s\n' "$minimum" "$current" | sort -V | head -n1)" == "$minimum" ]] + echo "[ERROR] Not running as root and 'sudo' is not available." >&2 + exit 1 fi +fi + +is_x86_64() { + [ "$ARCH" = "x86_64" ] || [ "$ARCH" = "amd64" ] } -IS_UBUNTU_24_OR_NEWER=false -IS_UBUNTU_26_OR_NEWER=false -IS_DEBIAN_13_OR_NEWER=false +is_arm64() { + [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ] +} -if [[ "$DISTRO" == "ubuntu" ]] && version_ge "$VERSION_ID" "24.04"; then - IS_UBUNTU_24_OR_NEWER=true -fi +# --------------------------------------------------------------------------- +# Package lists +# +# REQUIRED_PACKAGES installation fails if one of them cannot be installed +# OPTIONAL_PACKAGES nice to have; silently skipped when unavailable +# --------------------------------------------------------------------------- +REQUIRED_PACKAGES="" +OPTIONAL_PACKAGES="" +REFRESH_CMD="" +INSTALL_CMD="" -if [[ "$DISTRO" == "ubuntu" ]] && version_ge "$VERSION_ID" "26.04"; then - IS_UBUNTU_26_OR_NEWER=true -fi +case "$PKG_MGR" in + apt-get) + REFRESH_CMD="$SUDO apt-get update" + INSTALL_CMD="$SUDO apt-get install -y --no-install-recommends" -if [[ "$DISTRO" == "debian" ]] && version_ge "$VERSION_ID" "13"; then - IS_DEBIAN_13_OR_NEWER=true -fi + REQUIRED_PACKAGES="build-essential clang flex bison g++ gawk gcc gettext git \ + file wget unzip rsync swig time python3 python3-dev python3-setuptools \ + libncurses-dev libssl-dev zlib1g-dev libelf-dev make patch diffutils" -# --- Base packages (common to all) --- -COMMON_PACKAGES="build-essential clang flex bison g++ gawk \ -gettext git libssl-dev rsync swig unzip zlib1g-dev file wget bzip2" + # Extras from the OpenWrt 25.xx / Debian Trixie list. Names come and go + # between releases, so everything here is filtered before installing. + OPTIONAL_PACKAGES="bc binutils-gold ccache ecj fastjar help2man libbsd-dev \ + liblzma-dev meson mold mtd-utils ninja-build pbzip2 pigz pkg-config \ + subversion texinfo u-boot-tools xsltproc xxd zstd bzip2 quilt asciidoc \ + device-tree-compiler java-propose-classpath python3-distutils \ + python3-distutils-extra libncurses5-dev" -# --- Determine specific package set --- -if [[ "$ARCH" == "x86_64" || "$ARCH" == "amd64" ]]; then - echo "[INFO] Preparing environment for x86_64." + if is_x86_64; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES gcc-multilib g++-multilib glibc-source" + elif is_arm64; then + # arm64 has no native multilib; the cross variants are optional and the + # available set differs a lot between releases (e.g. the s390x variants + # were dropped in Ubuntu 26.04). + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES \ + gcc-multilib-i686-linux-gnu g++-multilib-i686-linux-gnu \ + gcc-multilib-x86-64-linux-gnu g++-multilib-x86-64-linux-gnu \ + gcc-multilib-x86-64-linux-gnux32 g++-multilib-x86-64-linux-gnux32 \ + gcc-multilib-s390x-linux-gnu g++-multilib-s390x-linux-gnu \ + libc6-dev-i386-cross libc6-dev-i386-amd64-cross libc6-dev-i386-x32-cross \ + glibc-source" + fi - if [[ "$IS_UBUNTU_26_OR_NEWER" == "true" ]]; then - echo "[INFO] Detected Ubuntu 26.04+ (no python3-distutils; OpenWrt docs use libncurses-dev)." - EXTRA_PACKAGES="gcc-multilib g++-multilib glibc-source libncurses-dev python3-setuptools" - elif [[ "$IS_UBUNTU_24_OR_NEWER" == "true" ]]; then - echo "[INFO] Detected Ubuntu 24.04+ (Python 3.12+, no python3-distutils)." - EXTRA_PACKAGES="gcc-multilib g++-multilib libncurses5-dev python3-setuptools" - elif [[ "$IS_DEBIAN_13_OR_NEWER" == "true" ]]; then - echo "[INFO] Detected Debian 13+ (no python3-distutils)." - EXTRA_PACKAGES="gcc-multilib g++-multilib libncurses-dev python3-setuptools" - else - echo "[INFO] Detected older Ubuntu/Debian (with python3-distutils)." - EXTRA_PACKAGES="gcc-multilib g++-multilib libncurses-dev python3-distutils python3-setuptools" - fi + if [ "$WITH_GO" = true ]; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES golang golang-go" + fi + ;; - INSTALL_PACKAGES="$COMMON_PACKAGES $EXTRA_PACKAGES" + dnf|yum) + REFRESH_CMD="$SUDO $PKG_MGR makecache" + INSTALL_CMD="$SUDO $PKG_MGR install -y --setopt=install_weak_deps=False" -elif [[ "$ARCH" == "aarch64" || "$ARCH" == "arm64" ]]; then - echo "[INFO] Preparing environment for ARM64." + REQUIRED_PACKAGES="gcc gcc-c++ make git-core patch diffutils file wget \ + unzip rsync tar which bzip2 ncurses-devel openssl-devel zlib-devel \ + python3 python3-devel python3-setuptools" - # Note: arm64 has no native multilib; install cross multilib packages - MULTILIB_PACKAGES=" - gcc-multilib-i686-linux-gnu gcc-multilib-s390x-linux-gnu \ - gcc-multilib-x86-64-linux-gnu gcc-multilib-x86-64-linux-gnux32 \ - g++-multilib-i686-linux-gnu g++-multilib-s390x-linux-gnu \ - g++-multilib-x86-64-linux-gnu g++-multilib-x86-64-linux-gnux32 \ - libc6-dev-i386-amd64-cross libc6-dev-i386-cross libc6-dev-i386-x32-cross" + OPTIONAL_PACKAGES="bash-completion elfutils-libelf-devel zlib-static \ + glibc-static swig quilt ccache perl-base perl-Data-Dumper \ + perl-File-Compare perl-File-Copy perl-FindBin perl-IPC-Cmd perl-JSON-PP \ + perl-lib perl-Thread-Queue perl-Time-Piece libxslt bc texinfo \ + dtc zstd pkgconf-pkg-config" - if [[ "$IS_UBUNTU_26_OR_NEWER" == "true" ]]; then - EXTRA_PACKAGES="glibc-source libncurses-dev python3-setuptools" - elif [[ "$IS_UBUNTU_24_OR_NEWER" == "true" ]]; then - EXTRA_PACKAGES="libncurses5-dev python3-setuptools" - elif [[ "$IS_DEBIAN_13_OR_NEWER" == "true" ]]; then - EXTRA_PACKAGES="libncurses-dev python3-setuptools" - else - EXTRA_PACKAGES="libncurses-dev python3-distutils python3-setuptools" - fi + if [ "$WITH_GO" = true ]; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES golang" + fi + ;; - INSTALL_PACKAGES="$COMMON_PACKAGES $EXTRA_PACKAGES $MULTILIB_PACKAGES golang" + pacman) + REFRESH_CMD="$SUDO pacman -Sy" + INSTALL_CMD="$SUDO pacman -S --needed --noconfirm" + REQUIRED_PACKAGES="base-devel bash bzip2 git libelf libxslt ncurses openssl \ + rsync swig time unzip util-linux wget which zlib python python-setuptools" + + OPTIONAL_PACKAGES="ccache quilt pkgconf diffutils patch tar make gawk \ + subversion asciidoc dtc zstd" + + if [ "$WITH_GO" = true ]; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES go" + fi + ;; + + zypper) + REFRESH_CMD="$SUDO zypper --non-interactive refresh" + INSTALL_CMD="$SUDO zypper --non-interactive install --no-recommends" + + REQUIRED_PACKAGES="bash bc binutils bzip2 flex gawk gcc gcc-c++ \ + gettext-tools git-core make ncurses-devel patch rsync unzip util-linux \ + wget zlib-devel libopenssl-devel python3-devel python3-setuptools" + + OPTIONAL_PACKAGES="asciidoc fastjar intltool libxslt-tools mercurial \ + perl-ExtUtils-MakeMaker sdcc quilt ccache libelf-devel swig zstd dtc" + + if [ "$WITH_GO" = true ]; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES go" + fi + ;; + + apk) + REFRESH_CMD="$SUDO apk update" + INSTALL_CMD="$SUDO apk add" + + REQUIRED_PACKAGES="argp-standalone bash bc binutils bzip2 coreutils \ + diffutils elfutils-dev findutils flex g++ gawk gcc gettext git grep gzip \ + linux-headers make ncurses-dev openssl-dev patch perl python3-dev rsync \ + tar unzip util-linux wget zlib-dev" + + OPTIONAL_PACKAGES="asciidoc cdrkit intltool libxslt musl-fts-dev \ + musl-libintl musl-obstack-dev swig quilt ccache zstd" + + if [ "$WITH_GO" = true ]; then + OPTIONAL_PACKAGES="$OPTIONAL_PACKAGES go" + fi + ;; +esac + +# --------------------------------------------------------------------------- +# Availability filtering +# --------------------------------------------------------------------------- +pkg_available() { + local pkg="$1" + + case "$PKG_MGR" in + apt-get) + # apt-cache policy prints no "Candidate:" line for purely virtual names + # and "(none)" when no installable version exists in the repositories. + local policy="" + policy="$(apt-cache policy "$pkg" 2>/dev/null || true)" + case "$policy" in + *"Candidate: (none)"*) return 1 ;; + *"Candidate:"*) return 0 ;; + *) return 1 ;; + esac + ;; + dnf) dnf -q info "$pkg" >/dev/null 2>&1 ;; + yum) yum -q info "$pkg" >/dev/null 2>&1 ;; + pacman) pacman -Si "$pkg" >/dev/null 2>&1 || pacman -Sg "$pkg" >/dev/null 2>&1 ;; + zypper) zypper --non-interactive --quiet search --match-exact "$pkg" >/dev/null 2>&1 ;; + apk) [ -n "$(apk search -x "$pkg" 2>/dev/null)" ] ;; + *) return 0 ;; + esac +} + +filter_available() { + local pkg + local kept="" + local dropped="" + + for pkg in $1; do + if pkg_available "$pkg"; then + kept="$kept $pkg" + else + dropped="$dropped $pkg" + fi + done + + FILTER_KEPT="${kept# }" + FILTER_DROPPED="${dropped# }" +} + +echo +echo "[INFO] Refreshing package metadata..." +if [ "$DRY_RUN" = true ]; then + echo "[DRY-RUN] $REFRESH_CMD" else - echo "[WARN] Unknown architecture: $ARCH" - echo "[WARN] Attempting to use x86_64 package list." - if [[ "$IS_UBUNTU_26_OR_NEWER" == "true" ]]; then - INSTALL_PACKAGES="$COMMON_PACKAGES gcc-multilib g++-multilib glibc-source libncurses-dev python3-setuptools" - else - INSTALL_PACKAGES="$COMMON_PACKAGES gcc-multilib g++-multilib libncurses-dev python3-setuptools" - fi + $REFRESH_CMD +fi + +echo "[INFO] Checking package availability for this release..." + +filter_available "$REQUIRED_PACKAGES" +REQUIRED_AVAILABLE="$FILTER_KEPT" +REQUIRED_MISSING="$FILTER_DROPPED" + +filter_available "$OPTIONAL_PACKAGES" +OPTIONAL_AVAILABLE="$FILTER_KEPT" +OPTIONAL_MISSING="$FILTER_DROPPED" + +INSTALL_PACKAGES="$REQUIRED_AVAILABLE $OPTIONAL_AVAILABLE" +INSTALL_PACKAGES="$(echo "$INSTALL_PACKAGES" | tr -s '[:space:]' ' ' | sed 's/^ //;s/ $//')" + +if [ -z "$INSTALL_PACKAGES" ]; then + echo "[ERROR] No installable package found. Are the repositories configured?" >&2 + exit 1 fi -# --- Display package list --- echo echo "========================================" echo "πŸ“¦ Packages to install:" -echo "$INSTALL_PACKAGES" +echo "$INSTALL_PACKAGES" | tr ' ' '\n' | sed 's/^/ /' echo "========================================" -echo -echo "[INFO] Updating APT repositories..." -sudo apt update -y +if [ -n "$REQUIRED_MISSING" ]; then + echo + echo "[WARN] Required packages not offered by this release (build may fail):" + echo " $REQUIRED_MISSING" +fi -echo "[INFO] Installing packages..." -sudo apt install -y $INSTALL_PACKAGES +if [ -n "$OPTIONAL_MISSING" ]; then + echo + echo "[INFO] Optional packages unavailable here, skipped:" + echo " $OPTIONAL_MISSING" +fi -# --- Post installation notice --- -echo -echo "========================================" -echo "βœ… OpenWRT build environment setup complete!" -echo "========================================" -echo +if [ "$DRY_RUN" = true ]; then + echo + echo "[DRY-RUN] $INSTALL_CMD $INSTALL_PACKAGES" + exit 0 +fi -if [[ "$ARCH" == "x86_64" || "$ARCH" == "amd64" ]]; then - echo "πŸ‘‰ [NOTE] Go language is optional on x86_64." - echo " If you plan to compile packages requiring Go (e.g. firewall4, netifd), install it manually:" - echo " sudo apt install golang -y" -else - echo "πŸ‘‰ [INFO] Go language installed for ARM64 cross compilation." +if [ "$ASSUME_YES" != true ]; then + echo + read -r -p "Proceed with installation? [Y/n]: " reply + case "$reply" in + [nN]|[nN][oO]) echo "[INFO] Aborted by user."; exit 0 ;; + esac fi echo -echo "βš™οΈ After running 'make menuconfig', set Go toolchain path if required:" -echo " Example: /usr/lib/go-1.18" +echo "[INFO] Installing packages..." + +FAILED_PACKAGES="" +if ! $INSTALL_CMD $INSTALL_PACKAGES; then + echo + echo "[WARN] Bulk installation failed. Retrying package by package..." + for pkg in $INSTALL_PACKAGES; do + if ! $INSTALL_CMD "$pkg" >/dev/null 2>&1; then + FAILED_PACKAGES="$FAILED_PACKAGES $pkg" + echo " [FAIL] $pkg" + fi + done + FAILED_PACKAGES="${FAILED_PACKAGES# }" +fi + echo +echo "========================================" +if [ -n "$FAILED_PACKAGES" ]; then + echo "⚠️ Setup finished with errors." + echo " Failed packages: $FAILED_PACKAGES" +else + echo "βœ… OpenWrt build environment setup complete!" +fi +echo "========================================" +echo + +if [ "$WITH_GO" != true ]; then + echo "πŸ‘‰ [NOTE] The Go toolchain is only needed for Go based packages" + echo " (e.g. xray-core, v2ray). Install it with:" + echo " ./prepare-openwrt-env.sh --with-go" + echo +fi + +if is_arm64; then + echo "πŸ‘‰ [NOTE] arm64 has no native multilib. The cross-multilib packages are" + echo " optional and only needed for x86 targets; missing ones were skipped." + echo +fi + echo "πŸ‘‰ Next step:" -echo " ./prepare-openwrt.sh # to clone and set up OpenWRT source" +echo " ./prepare-openwrt.sh # clone and set up the OpenWrt source" echo diff --git a/prepare-openwrt.sh b/prepare-openwrt.sh index 92e1040..5d99979 100755 --- a/prepare-openwrt.sh +++ b/prepare-openwrt.sh @@ -1,341 +1,1024 @@ #!/bin/bash -# Automatically prepare OpenWRT build environment. -# - Clone or update repository -# - Support selecting release/beta/snapshot versions -# - Normalize helper checkout into openwrt/helper -# - Update and install feeds -# - Prepare .config for build +# Prepare an OpenWrt build tree. +# +# Layout produced by this script: +# before /openwrt-build-helper/... this helper checkout +# first run /openwrt/ the OpenWrt source root +# /openwrt/helper/... this checkout, moved in +# later runs the same /openwrt/ is updated in place +# +# Nothing on disk is touched before the plan is printed and confirmed, so +# aborting at the prompt leaves the helper checkout exactly where it was. +# +# Usage: ./prepare-openwrt.sh [options] +# +# Version selection (mutually exclusive; omit all for an interactive menu): +# --stable [] stable release tag, e.g. 25.12.5 or v25.12.5 +# without a value: newest stable tag (rc tags excluded) +# --branch release branch: 24.10, openwrt-24.10, main, ... +# --snapshot newest snapshot (the 'main' branch) +# +# Behaviour: +# -y, --yes non-interactive, accept defaults, never prompt +# -n, --dry-run print the plan and every action, change nothing +# -f, --force override the safety guards (dirty tree, stray files) +# --full-clone fetch full history (default: --filter=blob:none) +# --allow-rc let --stable with no value pick a release candidate +# --root use as the OpenWrt root instead of /openwrt +# --skip-feeds do not run scripts/feeds update/install +# --skip-defconfig do not run make defconfig +# --repo-url override the upstream URL (advanced / testing) +# --no-compat-link do not leave a symlink at the old checkout path +# -h, --help show this help +# +# After the first run the checkout lives at /openwrt/helper. A symlink +# is left behind at the original path so that shells, editors and agents whose +# working directory still points at /openwrt-build-helper keep working. +# +# Note: with the default partial clone, switching to another tag or branch +# later fetches the missing blobs on demand and therefore needs network. +# +# Author: Zhe Yuan -set -e +set -euo pipefail umask 022 +# --- Configuration --- REPO_URL="https://github.com/openwrt/openwrt.git" -SCRIPT_PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)/$(basename "${BASH_SOURCE[0]}")" -SCRIPT_DIR="$(dirname "$SCRIPT_PATH")" +SNAPSHOT_BRANCH="main" +SNAPSHOT_FALLBACK="master" + +# --- Options --- +VERSION_MODE="" +VERSION_INPUT="" +ASSUME_YES=false +DRY_RUN=false +FORCE=false +FULL_CLONE=false +ALLOW_RC=false +ROOT_OVERRIDE="" +SKIP_FEEDS=false +SKIP_DEFCONFIG=false +COMPAT_LINK=true + +# --- Discovered state --- +SCRIPT_INVOKED_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +# Physical path: layout decisions must see through a compatibility symlink, +# otherwise re-running through the old path would relocate an already +# relocated checkout. +SCRIPT_DIR="$(cd -- "$SCRIPT_INVOKED_DIR" && pwd -P)" +COMPAT_LINK_PATH="" WORK_ROOT="" HELPER_DIR="" +RELOCATE_MODE="none" # none | move | inplace +ROOT_STATE="" +TARGET_KIND="" # tag | branch +TARGET_REF="" +CURRENT_REF="" +REMOTE_TAGS="" +REMOTE_HEADS="" +LOCK_DIR="" +CONFIG_BACKUP="" + +# --- Output helpers --- +info() { echo "[INFO] $*"; } +warn() { echo "[WARN] $*"; } +err() { echo "[ERROR] $*" >&2; } + +# Wrap every mutating command so that --dry-run prints instead of running. +run() { + if [ "$DRY_RUN" = true ]; then + printf '[DRY-RUN]' + printf ' %q' "$@" + echo + return 0 + fi + "$@" +} + +usage() { + sed -n '2,42p' "$0" | sed 's/^# \{0,1\}//;s/^#$//' +} + +set_version_mode() { + if [ -n "$VERSION_MODE" ]; then + err "--stable, --branch and --snapshot are mutually exclusive." + exit 1 + fi + VERSION_MODE="$1" +} + +# --- Argument parsing --- +while [ $# -gt 0 ]; do + case "$1" in + --stable) + set_version_mode stable + # Optional value: only consume $2 when it is not another option. + if [ $# -gt 1 ] && [ -n "${2:-}" ] && [ "${2#-}" = "$2" ]; then + VERSION_INPUT="$2" + shift + fi + ;; + --branch) + set_version_mode branch + if [ $# -lt 2 ] || [ -z "${2:-}" ]; then + err "--branch requires a branch name." + exit 1 + fi + VERSION_INPUT="$2" + shift + ;; + --snapshot) set_version_mode snapshot ;; + -y|--yes) ASSUME_YES=true ;; + -n|--dry-run) DRY_RUN=true ;; + -f|--force) FORCE=true ;; + --full-clone) FULL_CLONE=true ;; + --allow-rc) ALLOW_RC=true ;; + --skip-feeds) SKIP_FEEDS=true ;; + --skip-defconfig) SKIP_DEFCONFIG=true ;; + --no-compat-link) COMPAT_LINK=false ;; + --root) + if [ $# -lt 2 ] || [ -z "${2:-}" ]; then + err "--root requires a directory." + exit 1 + fi + ROOT_OVERRIDE="$2" + shift + ;; + --repo-url) + if [ $# -lt 2 ] || [ -z "${2:-}" ]; then + err "--repo-url requires a URL." + exit 1 + fi + REPO_URL="$2" + shift + ;; + -h|--help) usage; exit 0 ;; + *) + err "Unknown option: $1" + echo "Run '$0 --help' for usage." >&2 + exit 1 + ;; + esac + shift +done + +# A closed stdin must not make `read` spin or hang. +if [ ! -t 0 ]; then + ASSUME_YES=true +fi echo "========================================" -echo " OpenWRT Build Environment Setup Script" +echo " OpenWrt Source Preparation" echo "========================================" -# --- Helpers --- +# --------------------------------------------------------------------------- +# Generic helpers +# --------------------------------------------------------------------------- + +# Resolve a directory to its physical path, returning non-zero instead of +# letting a bare `cd` abort the whole script under `set -e`. +abspath() { + local p="$1" + [ -d "$p" ] || return 1 + ( cd -- "$p" 2>/dev/null && pwd -P ) || return 1 +} + +is_git_root() { + local d="$1" top resolved + [ -e "$d/.git" ] || return 1 + top="$(git -C "$d" rev-parse --show-toplevel 2>/dev/null)" || return 1 + resolved="$(abspath "$d")" || return 1 + [ "$top" = "$resolved" ] +} + +# Content-only check: does not require the directory to be a git repository. +is_openwrt_tree() { + [ -f "$1/feeds.conf.default" ] && [ -f "$1/Makefile" ] && [ -d "$1/scripts" ] +} + is_openwrt_repo() { - local dir="$1" + is_git_root "$1" && is_openwrt_tree "$1" +} - [ -f "$dir/feeds.conf.default" ] && - [ -f "$dir/Makefile" ] && - git -C "$dir" rev-parse --is-inside-work-tree >/dev/null 2>&1 +has_openwrt_origin() { + local u + u="$(git -C "$1" remote get-url origin 2>/dev/null)" || return 1 + case "$u" in + *openwrt/openwrt*|"$REPO_URL") return 0 ;; + *) return 1 ;; + esac } is_helper_checkout() { - local dir="$1" + local d="$1" - [ -f "$dir/prepare-openwrt.sh" ] && - [ -f "$dir/prepare-openwrt-env.sh" ] && - [ -f "$dir/add-external-repos.sh" ] -} + [ -f "$d/prepare-openwrt.sh" ] || return 1 -has_remote_branch() { - local branch="$1" - git show-ref --verify --quiet "refs/remotes/origin/$branch" -} - -has_tag() { - local tag="$1" - git rev-parse --verify "refs/tags/$tag" >/dev/null 2>&1 -} - -ensure_helper_target_available() { - local target_helper="$1" - local source_dir="$2" - local existing_path="" - local source_path="" - - if [ ! -e "$target_helper" ]; then - return + # Complete checkout. + if [ -f "$d/prepare-openwrt-env.sh" ] && [ -f "$d/add-external-repos.sh" ]; then + return 0 fi - existing_path="$(cd "$target_helper" && pwd -P)" - source_path="$(cd "$source_dir" && pwd -P)" - - if [ "$existing_path" = "$source_path" ]; then - return + # Half-finished in-place move: some scripts already sit in helper/, so the + # full signature is gone. Recognizing this is what makes the move resumable. + if [ -d "$d/helper" ] && + { [ -f "$d/helper/prepare-openwrt-env.sh" ] || + [ -f "$d/helper/add-external-repos.sh" ]; }; then + return 0 fi - echo "[ERROR] Helper destination already exists at '$target_helper'." - echo "[ERROR] Refusing to overwrite a different helper checkout." - exit 1 + [ -f "$d/prepare-openwrt-env.sh" ] || [ -f "$d/add-external-repos.sh" ] } -ensure_target_root_ready() { - local target_root="$1" - local first_entry="" - - if [ ! -e "$target_root" ]; then - return - fi - - if [ ! -d "$target_root" ]; then - echo "[ERROR] Target OpenWrt root '$target_root' exists but is not a directory." +require_sort_v() { + if [ "$(printf 'v1.10\nv1.9\n' | sort -V | head -n 1)" != "v1.9" ]; then + err "This script needs GNU 'sort -V' for version ordering." exit 1 fi - - if is_openwrt_repo "$target_root"; then - return - fi - - first_entry="$(find "$target_root" -mindepth 1 -maxdepth 1 ! -name helper -print -quit 2>/dev/null || true)" - if [ -z "$first_entry" ]; then - return - fi - - echo "[ERROR] Target OpenWrt root '$target_root' already contains unexpected files." - echo "[ERROR] Refusing to merge the helper checkout into a non-OpenWrt directory." - exit 1 } -normalize_structure() { - local source_dir="$SCRIPT_DIR" - local source_name - local parent_dir - local target_root - local target_helper - local temp_dir +confirm() { + # confirm "" + local question="$1" default="$2" reply="" - source_name="$(basename "$source_dir")" - parent_dir="$(dirname "$source_dir")" - - if [ "$source_name" = "helper" ] && [ "$(basename "$parent_dir")" = "openwrt" ]; then - WORK_ROOT="$parent_dir" - HELPER_DIR="$source_dir" - cd "$WORK_ROOT" - echo "[INFO] Reusing existing normalized layout at '$WORK_ROOT'." - return + if [ "$ASSUME_YES" = true ]; then + return 0 fi - if is_openwrt_repo "$source_dir"; then - WORK_ROOT="$source_dir" - HELPER_DIR="$source_dir/helper" - cd "$WORK_ROOT" - echo "[INFO] Script is already running from an OpenWrt source root at '$WORK_ROOT'." - echo "[INFO] Skipping relocation to avoid moving an existing OpenWrt tree." - return - fi - - if ! is_helper_checkout "$source_dir"; then - echo "[ERROR] '$source_dir' does not look like a helper checkout." - echo "[ERROR] Refusing to relocate an unexpected directory." - exit 1 - fi - - target_root="$parent_dir/openwrt" - target_helper="$target_root/helper" - - if [ "$source_dir" != "$target_root" ]; then - ensure_target_root_ready "$target_root" - fi - ensure_helper_target_available "$target_helper" "$source_dir" - - if [ "$source_dir" = "$target_root" ]; then - echo "[INFO] Helper checkout already uses the target root name. Normalizing in place." - temp_dir="$parent_dir/.openwrt-helper-relocate-$$" - - if [ -e "$temp_dir" ]; then - echo "[ERROR] Temporary relocation path already exists: '$temp_dir'." - exit 1 - fi - - cd "$parent_dir" - mv "$source_dir" "$temp_dir" - mkdir -p "$target_root" - mv "$temp_dir" "$target_helper" + if [ "$default" = "yes" ]; then + read -r -p "$question [Y/n]: " reply || reply="" + case "${reply:-}" in + [nN]|[nN][oO]) return 1 ;; + *) return 0 ;; + esac else - echo "[INFO] Relocating helper checkout to '$target_helper'." - mkdir -p "$target_root" - cd "$parent_dir" - mv "$source_dir" "$target_helper" - fi - - WORK_ROOT="$target_root" - HELPER_DIR="$target_helper" - cd "$WORK_ROOT" - - echo "[INFO] OpenWrt source root: '$WORK_ROOT'." - echo "[INFO] Helper checkout: '$HELPER_DIR'." -} - -ensure_origin_remote() { - local current_url="" - - if git remote get-url origin >/dev/null 2>&1; then - current_url="$(git remote get-url origin)" - if [ "$current_url" != "$REPO_URL" ]; then - echo "[INFO] Updating git remote 'origin' to '$REPO_URL'." - git remote set-url origin "$REPO_URL" - fi - else - git remote add origin "$REPO_URL" + read -r -p "$question [y/N]: " reply || reply="" + case "${reply:-}" in + [yY]|[yY][eE][sS]) return 0 ;; + *) return 1 ;; + esac fi } -prepare_openwrt_repo() { - cd "$WORK_ROOT" - - if is_openwrt_repo "$WORK_ROOT"; then - echo "[INFO] Existing OpenWrt repository detected at '$WORK_ROOT'. Updating refs..." - ensure_origin_remote - git fetch --tags origin - return +release_lock() { + if [ -n "$LOCK_DIR" ] && [ -d "$LOCK_DIR" ]; then + rmdir "$LOCK_DIR" 2>/dev/null || true fi +} - if [ -e "$WORK_ROOT/.git" ]; then - echo "[ERROR] '$WORK_ROOT' already contains a Git repository that does not look like OpenWrt." +acquire_lock() { + [ "$DRY_RUN" = true ] && return 0 + LOCK_DIR="$WORK_ROOT/.prepare-openwrt.lock" + if ! mkdir "$LOCK_DIR" 2>/dev/null; then + LOCK_DIR="" + err "Another prepare-openwrt.sh seems to be running in '$WORK_ROOT'." + err "If that is wrong, remove '$WORK_ROOT/.prepare-openwrt.lock'." exit 1 fi - - if [ -f "$WORK_ROOT/feeds.conf.default" ] || [ -f "$WORK_ROOT/Makefile" ] || [ -d "$WORK_ROOT/package" ]; then - echo "[ERROR] '$WORK_ROOT' contains files that look like an incomplete OpenWrt tree." - echo "[ERROR] Refusing to initialize over an unexpected directory state." - exit 1 - fi - - echo "[INFO] Initializing OpenWrt repository in '$WORK_ROOT'..." - git init . - ensure_origin_remote - git fetch --tags origin + trap release_lock EXIT } -checkout_or_update_branch() { - local branch="$1" +# --------------------------------------------------------------------------- +# Phase A: layout discovery (read-only) +# --------------------------------------------------------------------------- - if [ ! "$(git rev-parse --abbrev-ref HEAD 2>/dev/null)" = "$branch" ]; then - if git show-ref --verify --quiet "refs/heads/$branch"; then - git checkout "$branch" +# Echoes one of: +# absent empty openwrt-complete openwrt-partial foreign-git nonempty +# notdir unreadable +classify_root() { + local d="$1" entry name + + [ -e "$d" ] || { echo absent; return; } + [ -d "$d" ] || { echo notdir; return; } + { [ -r "$d" ] && [ -x "$d" ]; } || { echo unreadable; return; } + + if is_openwrt_repo "$d"; then echo openwrt-complete; return; fi + + if is_git_root "$d"; then + if has_openwrt_origin "$d"; then + echo openwrt-partial else - git checkout -b "$branch" --track "origin/$branch" + echo foreign-git fi + return fi - git pull --ff-only origin "$branch" + # Any real content other than the helper checkout and well-known junk? + while IFS= read -r entry; do + name="$(basename "$entry")" + case "$name" in + helper|.DS_Store|Thumbs.db|.directory|.prepare-openwrt.lock) continue ;; + *) echo nonempty; return ;; + esac + done < <(find "$d" -mindepth 1 -maxdepth 1) + + echo empty } -# --- Function: choose version --- -choose_version() { - echo - echo "Select OpenWRT version type:" - echo " 1) Stable release (e.g., 24.10.4)" - echo " 2) Beta (e.g., 24.10-SNAPSHOT or openwrt-24.10)" - echo " 3) Latest snapshot (master branch)" - echo -n "Enter choice [1-3, default=1]: " - read -r choice +list_root_clutter() { + local d="$1" entry name + while IFS= read -r entry; do + name="$(basename "$entry")" + case "$name" in + helper|.DS_Store|Thumbs.db|.directory|.prepare-openwrt.lock) continue ;; + *) echo " $name" ;; + esac + done < <(find "$d" -mindepth 1 -maxdepth 1) +} - case "$choice" in +check_helper_target() { + local t="$HELPER_DIR" existing source + + [ -e "$t" ] || return 0 + + if [ ! -d "$t" ]; then + err "Helper destination '$t' exists but is not a directory." + err "Move or remove it, then re-run." + exit 1 + fi + + existing="$(abspath "$t")" || { err "Cannot resolve '$t'."; exit 1; } + source="$(abspath "$SCRIPT_DIR")" || { err "Cannot resolve '$SCRIPT_DIR'."; exit 1; } + [ "$existing" = "$source" ] && return 0 + + if is_helper_checkout "$t"; then + err "A different helper checkout already exists at '$t'." + else + err "'$t' exists and is not a helper checkout." + fi + err "Move or remove it, then re-run." + exit 1 +} + +discover_layout() { + local parent + parent="$(dirname -- "$SCRIPT_DIR")" + + if [ -n "$ROOT_OVERRIDE" ]; then + # Escape hatch for every ambiguous case. + if [ -d "$ROOT_OVERRIDE" ]; then + WORK_ROOT="$(abspath "$ROOT_OVERRIDE")" || + { err "Cannot resolve --root '$ROOT_OVERRIDE'."; exit 1; } + else + case "$ROOT_OVERRIDE" in + /*) WORK_ROOT="$ROOT_OVERRIDE" ;; + *) WORK_ROOT="$PWD/$ROOT_OVERRIDE" ;; + esac + fi + + if [ "$SCRIPT_DIR" = "$WORK_ROOT/helper" ] || ! is_helper_checkout "$SCRIPT_DIR"; then + HELPER_DIR="$WORK_ROOT/helper" + RELOCATE_MODE="none" + elif [ "$SCRIPT_DIR" = "$WORK_ROOT" ]; then + HELPER_DIR="$WORK_ROOT/helper" + RELOCATE_MODE="inplace" + else + HELPER_DIR="$WORK_ROOT/helper" + RELOCATE_MODE="move" + fi + + elif is_helper_checkout "$SCRIPT_DIR" && root_is_parent "$parent"; then + # Already normalized. Detected by CONTENT first, so renaming the parent + # to e.g. openwrt-24.10 does not trigger a second relocation. + WORK_ROOT="$parent" + HELPER_DIR="$SCRIPT_DIR" + RELOCATE_MODE="none" + info "Reusing the existing layout at '$WORK_ROOT'." + + elif is_openwrt_repo "$SCRIPT_DIR"; then + WORK_ROOT="$SCRIPT_DIR" + HELPER_DIR="$SCRIPT_DIR/helper" + RELOCATE_MODE="none" + warn "Running from inside an OpenWrt source root; not relocating anything." + + elif is_helper_checkout "$SCRIPT_DIR"; then + WORK_ROOT="$parent/openwrt" + HELPER_DIR="$WORK_ROOT/helper" + if [ "$SCRIPT_DIR" = "$WORK_ROOT" ]; then + RELOCATE_MODE="inplace" + else + RELOCATE_MODE="move" + fi + + else + err "'$SCRIPT_DIR' does not look like a helper checkout." + err "Refusing to relocate an unexpected directory. Use --root to override." + exit 1 + fi + + if [ "$RELOCATE_MODE" = "inplace" ]; then + # The root currently IS the helper checkout (its own git repo included). + # Classifying it would reject it as a foreign repository; after the + # contents move down into helper/ the root is empty by construction. + ROOT_STATE="inplace-helper" + else + ROOT_STATE="$(classify_root "$WORK_ROOT")" + fi + + case "$ROOT_STATE" in + notdir) + err "OpenWrt root '$WORK_ROOT' exists but is not a directory." + exit 1 + ;; + unreadable) + err "OpenWrt root '$WORK_ROOT' exists but cannot be read (permissions)." + exit 1 + ;; + foreign-git) + err "'$WORK_ROOT' is a Git repository, but its origin is not OpenWrt." + err "Choose another root with --root, or move that repository away." + exit 1 + ;; + openwrt-partial) + warn "'$WORK_ROOT' has a Git repository but no source tree." + warn "A previous run was interrupted; resuming it." + ;; + nonempty) + err "OpenWrt root '$WORK_ROOT' contains unexpected files:" + list_root_clutter "$WORK_ROOT" >&2 + if [ "$FORCE" != true ]; then + err "Refusing to use it. Re-run with --force to proceed anyway." + exit 1 + fi + warn "--force given; proceeding despite the files above." + ;; + esac + + case "$RELOCATE_MODE" in + move) + check_helper_target + ;; + inplace) + # helper/ is created by the move itself and may be partially populated + # from an interrupted run, so only rule out a non-directory here. + if [ -e "$HELPER_DIR" ] && [ ! -d "$HELPER_DIR" ]; then + err "Helper destination '$HELPER_DIR' exists but is not a directory." + exit 1 + fi + ;; + esac + + # A repo created inside another worktree confuses the outer repository. + local outer="" + if outer="$(git -C "$(dirname -- "$WORK_ROOT")" rev-parse --show-toplevel 2>/dev/null)"; then + if [ -n "$outer" ] && [ "$outer" != "$WORK_ROOT" ]; then + warn "'$WORK_ROOT' will live inside the Git worktree at '$outer'." + fi + fi +} + +# True when $1 is already the OpenWrt root for this helper checkout. +root_is_parent() { + local p="$1" + is_openwrt_repo "$p" || + has_openwrt_origin "$p" || + is_openwrt_tree "$p" || + [ "$(basename -- "$p")" = "openwrt" ] +} + +# --------------------------------------------------------------------------- +# Phase B: remote ref discovery (read-only, network, no local repo needed) +# --------------------------------------------------------------------------- +discover_remote_refs() { + info "Reading refs from $REPO_URL ..." + + if ! REMOTE_TAGS="$(git ls-remote --tags --refs "$REPO_URL" 'v*' 2>&1 | + awk '{print $2}' | sed 's#refs/tags/##')"; then + err "Cannot read tags from '$REPO_URL'." + err "$REMOTE_TAGS" + exit 1 + fi + + if ! REMOTE_HEADS="$(git ls-remote --heads "$REPO_URL" 2>&1 | + awk '{print $2}' | sed 's#refs/heads/##')"; then + err "Cannot read branches from '$REPO_URL'." + err "$REMOTE_HEADS" + exit 1 + fi + + if [ -z "$REMOTE_HEADS" ]; then + err "'$REPO_URL' reports no branches; is the URL correct?" + exit 1 + fi +} + +remote_has_head() { + printf '%s\n' "$REMOTE_HEADS" | grep -qxF "$1" +} + +remote_has_tag() { + printf '%s\n' "$REMOTE_TAGS" | grep -qxF "$1" +} + +latest_stable_tag() { + local pattern='^v[0-9]+\.[0-9]+\.[0-9]+$' + [ "$ALLOW_RC" = true ] && pattern='^v[0-9]+\.[0-9]+\.[0-9]+(-rc[0-9]+)?$' + # Filter release candidates BEFORE sorting: `sort -V` orders v25.12.0-rc1 + # after v25.12.0, so filtering afterwards would still let an rc win. + printf '%s\n' "$REMOTE_TAGS" | grep -E "$pattern" | sort -V | tail -n 1 +} + +list_release_branches() { + printf '%s\n' "$REMOTE_HEADS" | grep -E '^openwrt-[0-9]+\.[0-9]+$' | sort -V +} + +# --------------------------------------------------------------------------- +# Phase C: version selection and ref resolution +# --------------------------------------------------------------------------- +choose_version_interactive() { + local choice="" newest branches + + newest="$(latest_stable_tag)" + branches="$(list_release_branches | tr '\n' ' ')" + + echo + echo "Select the OpenWrt version to build:" + echo " 1) Stable release (newest: ${newest:-none found})" + echo " 2) Release branch (${branches:-none found}main)" + echo " 3) Snapshot (the '$SNAPSHOT_BRANCH' branch)" + read -r -p "Enter choice [1-3, default=1]: " choice || choice="" + + case "${choice:-}" in 2) - echo -n "Enter beta version (e.g. 24.10 or openwrt-24.10): " - read -r VERSION - SELECTED_TYPE="beta" + VERSION_MODE="branch" + read -r -p "Enter branch (e.g. 25.12 or openwrt-25.12): " VERSION_INPUT || VERSION_INPUT="" + if [ -z "${VERSION_INPUT:-}" ]; then + err "No branch given." + exit 1 + fi ;; 3) - SELECTED_TYPE="snapshot" - VERSION="master" + VERSION_MODE="snapshot" ;; *) - echo -n "Enter stable version number (e.g. 24.10.4, default: latest): " - read -r VERSION - SELECTED_TYPE="stable" + VERSION_MODE="stable" + read -r -p "Enter version (e.g. 25.12.5, empty for newest): " VERSION_INPUT || VERSION_INPUT="" ;; esac } -# --- Start --- -normalize_structure -choose_version -prepare_openwrt_repo +resolve_target_ref() { + local v candidate -pushd "$WORK_ROOT" >/dev/null + case "$VERSION_MODE" in + snapshot) + TARGET_KIND="branch" + if remote_has_head "$SNAPSHOT_BRANCH"; then + TARGET_REF="$SNAPSHOT_BRANCH" + elif remote_has_head "$SNAPSHOT_FALLBACK"; then + warn "Branch '$SNAPSHOT_BRANCH' not found; using '$SNAPSHOT_FALLBACK'." + TARGET_REF="$SNAPSHOT_FALLBACK" + else + err "Neither '$SNAPSHOT_BRANCH' nor '$SNAPSHOT_FALLBACK' exists upstream." + exit 1 + fi + ;; -# --- Version selection logic --- -if [ "$SELECTED_TYPE" == "snapshot" ]; then - echo "[INFO] Checking out latest master (snapshot)..." - if has_remote_branch "master"; then - checkout_or_update_branch "master" - else - echo "[ERROR] Remote branch 'master' not found." - exit 1 - fi -elif [ "$SELECTED_TYPE" == "beta" ]; then - local_branch="" - local_tag="" + branch) + TARGET_KIND="branch" + v="$VERSION_INPUT" + case "$v" in + main|master|openwrt-*) candidate="$v" ;; + [0-9]*) candidate="openwrt-$v" ;; + *) candidate="$v" ;; + esac - if [[ "$VERSION" == openwrt-* ]]; then - local_branch="$VERSION" - local_tag="v${VERSION#openwrt-}-SNAPSHOT" - elif [[ "$VERSION" == v*-SNAPSHOT ]]; then - local_tag="$VERSION" - local_branch="openwrt-${VERSION#v}" - local_branch="${local_branch%-SNAPSHOT}" - else - local_branch="openwrt-${VERSION}" - local_tag="v${VERSION}-SNAPSHOT" - fi + if remote_has_head "$candidate"; then + TARGET_REF="$candidate" + elif remote_has_head "openwrt-$v"; then + TARGET_REF="openwrt-$v" + else + err "Branch not found upstream for '$v'." + err "Available release branches:" + list_release_branches | sed 's/^/ /' >&2 + err " main" + exit 1 + fi + ;; - if has_remote_branch "$local_branch"; then - echo "[INFO] Checking out beta branch '$local_branch'..." - checkout_or_update_branch "$local_branch" - elif has_tag "$local_tag"; then - echo "[INFO] Checking out beta tag '$local_tag'..." - git checkout "$local_tag" - else - echo "[ERROR] Beta branch or tag not found for version '$VERSION'." - exit 1 - fi -else - # Stable version - if [ -z "$VERSION" ]; then - LATEST_TAG=$(git tag -l "v*" | sort -V | tail -n 1) - if [ -z "$LATEST_TAG" ]; then - echo "[ERROR] No stable tags found in remote repository." + stable) + TARGET_KIND="tag" + if [ -z "${VERSION_INPUT:-}" ]; then + TARGET_REF="$(latest_stable_tag)" + if [ -z "$TARGET_REF" ]; then + err "No stable tag found upstream." + exit 1 + fi + info "No version given; using the newest stable tag: $TARGET_REF" + else + v="$VERSION_INPUT" + case "$v" in v*) ;; *) v="v$v" ;; esac + if ! remote_has_tag "$v"; then + err "Version tag '$v' not found upstream." + err "Recent stable tags:" + printf '%s\n' "$REMOTE_TAGS" | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | + sort -V | tail -n 5 | sed 's/^/ /' >&2 + exit 1 + fi + TARGET_REF="$v" + fi + ;; + + *) + err "Internal error: no version mode selected." exit 1 + ;; + esac +} + +# --------------------------------------------------------------------------- +# Phase C': switch safety (read-only) +# --------------------------------------------------------------------------- +current_ref_of() { + local d="$1" + git -C "$d" describe --tags --exact-match 2>/dev/null || + git -C "$d" symbolic-ref --short HEAD 2>/dev/null || + git -C "$d" rev-parse --short HEAD 2>/dev/null || + true +} + +has_build_state() { + local d="$1" p + for p in .config build_dir staging_dir bin tmp feeds; do + [ -e "$d/$p" ] && return 0 + done + return 1 +} + +check_switch_safety() { + local dirty="" + + is_git_root "$WORK_ROOT" || return 0 + + CURRENT_REF="$(current_ref_of "$WORK_ROOT")" + + # Same ref means no switch, hence no risk. This is what keeps the guard + # from firing on every routine re-run. + [ -n "$CURRENT_REF" ] && [ "$CURRENT_REF" = "$TARGET_REF" ] && return 0 + is_openwrt_tree "$WORK_ROOT" || return 0 + + # Untracked files are excluded on purpose: a built OpenWrt tree always has + # feeds/, build_dir/, staging_dir/, bin/, .config and helper/ untracked. + dirty="$(git -C "$WORK_ROOT" status --porcelain --untracked-files=no 2>/dev/null || true)" + + if [ -n "$dirty" ]; then + warn "Tracked files in '$WORK_ROOT' have local modifications:" + git -C "$WORK_ROOT" status --short --untracked-files=no | sed 's/^/ /' + if [ "$FORCE" = true ]; then + warn "--force given; switching anyway." + elif [ "$ASSUME_YES" = true ]; then + err "Refusing to switch versions with local modifications under -y." + err "Commit or stash them, or re-run with --force." + exit 1 + elif ! confirm "Switching keeps your changes only if they apply cleanly. Continue?" no; then + info "Aborted by user." + exit 0 fi - echo "[INFO] No version specified, using latest stable tag: $LATEST_TAG" - VERSION="$LATEST_TAG" - else - if [[ "$VERSION" != v* ]]; then - VERSION="v${VERSION}" + elif has_build_state "$WORK_ROOT"; then + warn "'$WORK_ROOT' contains build state from '$CURRENT_REF'." + if [ "$ASSUME_YES" != true ] && + ! confirm "Build state is kept and .config is backed up. Continue?" yes; then + info "Aborted by user." + exit 0 fi fi - if has_tag "$VERSION"; then - git checkout "$VERSION" + if [ -f "$WORK_ROOT/.config" ]; then + CONFIG_BACKUP="$WORK_ROOT/.config.${CURRENT_REF//\//-}.bak" + fi +} + +# --------------------------------------------------------------------------- +# Phase D: plan and confirmation +# --------------------------------------------------------------------------- +print_plan() { + local root_note helper_note clone_note + + case "$ROOT_STATE" in + absent) root_note="(create)" ;; + empty) root_note="(use empty directory)" ;; + inplace-helper) root_note="(reuse this directory)" ;; + openwrt-complete) root_note="(update existing)" ;; + openwrt-partial) root_note="(resume interrupted run)" ;; + nonempty) root_note="(forced, contains other files)" ;; + *) root_note="" ;; + esac + + case "$RELOCATE_MODE" in + none) helper_note="(already in place)" ;; + move) helper_note="(move from $SCRIPT_DIR)" ;; + inplace) helper_note="(move contents of $SCRIPT_DIR down one level)" ;; + esac + + if [ "$FULL_CLONE" = true ]; then + clone_note="full history" else - echo "[ERROR] Version tag '$VERSION' not found." + clone_note="partial (--filter=blob:none)" + fi + + echo + echo "========================================" + echo " Planned actions" + echo "========================================" + echo " OpenWrt root : $WORK_ROOT $root_note" + echo " Helper : $HELPER_DIR $helper_note" + echo " Version : $TARGET_REF ($TARGET_KIND)" + echo " Currently at : ${CURRENT_REF:-(new checkout)}" + echo " Clone mode : $clone_note" + [ -n "$CONFIG_BACKUP" ] && echo " .config : back up to $(basename "$CONFIG_BACKUP")" + if [ "$RELOCATE_MODE" = "move" ] && [ "$COMPAT_LINK" = true ]; then + echo " Old path : $SCRIPT_DIR -> symlink to the new helper location" + fi + if [ "$SKIP_FEEDS" = true ]; then + echo " Feeds : skipped (--skip-feeds)" + else + echo " Feeds : update -a, install -a" + fi + if [ "$SKIP_DEFCONFIG" = true ]; then + echo " Post : skipped (--skip-defconfig)" + else + echo " Post : make defconfig" + fi + echo "========================================" +} + +# --------------------------------------------------------------------------- +# Phase E: mutation +# --------------------------------------------------------------------------- +relocate_helper() { + local entry name + + case "$RELOCATE_MODE" in + none) + return 0 + ;; + + move) + # Single rename inside one parent directory: atomic, no crash window. + info "Moving the helper checkout to '$HELPER_DIR'." + run mkdir -p "$WORK_ROOT" + run mv -- "$SCRIPT_DIR" "$HELPER_DIR" + + # Anything still pointing at the old path -- a shell, an editor, an + # agent session -- would otherwise be left on a dangling directory. + if [ "$COMPAT_LINK" = true ]; then + if [ "$DRY_RUN" = true ] || [ ! -e "$SCRIPT_DIR" ]; then + info "Leaving a compatibility symlink at '$SCRIPT_DIR'." + run ln -s -- "$HELPER_DIR" "$SCRIPT_DIR" + COMPAT_LINK_PATH="$SCRIPT_DIR" + else + warn "'$SCRIPT_DIR' still exists; skipping the compatibility symlink." + fi + fi + ;; + + inplace) + # The checkout is itself named 'openwrt'. Move its contents down one + # level instead of staging through a temporary sibling, so that an + # interrupt never strands anything outside the root. + if [ -z "$WORK_ROOT" ] || [ "$WORK_ROOT" = "/" ]; then + err "Refusing to reorganize '$WORK_ROOT'." + exit 1 + fi + info "Moving the checkout contents into '$HELPER_DIR'." + run mkdir -p "$HELPER_DIR" + shopt -s dotglob nullglob + for entry in "$WORK_ROOT"/*; do + name="$(basename -- "$entry")" + [ "$name" = "helper" ] && continue + [ "$name" = ".prepare-openwrt.lock" ] && continue + if [ -e "$HELPER_DIR/$name" ]; then + warn "Skipping '$name': already present in helper/." + continue + fi + run mv -- "$entry" "$HELPER_DIR/" + done + shopt -u dotglob nullglob + ;; + esac +} + +ensure_origin_remote() { + local current="" + if current="$(git -C "$WORK_ROOT" remote get-url origin 2>/dev/null)"; then + if [ "$current" != "$REPO_URL" ]; then + info "Pointing remote 'origin' at '$REPO_URL'." + run git -C "$WORK_ROOT" remote set-url origin "$REPO_URL" + fi + else + run git -C "$WORK_ROOT" remote add origin "$REPO_URL" + fi +} + +supports_partial_clone() { + local v + v="$(git version | awk '{print $3}')" + [ "$(printf '%s\n2.19.0\n' "$v" | sort -V | head -n 1)" = "2.19.0" ] +} + +ensure_repo() { + # Fresh, resumed and complete roots all take this one path, so there is no + # separate resume branch that can rot. + if ! is_git_root "$WORK_ROOT"; then + info "Initializing a Git repository in '$WORK_ROOT'." + run mkdir -p "$WORK_ROOT" + run git -C "$WORK_ROOT" init -q + fi + + ensure_origin_remote + + if [ "$FULL_CLONE" != true ] && supports_partial_clone; then + info "Fetching (partial clone, blobs on demand)..." + run git -C "$WORK_ROOT" config remote.origin.promisor true + run git -C "$WORK_ROOT" config remote.origin.partialclonefilter blob:none + if ! run git -C "$WORK_ROOT" fetch --tags --prune --prune-tags \ + --filter=blob:none origin; then + warn "Partial fetch failed; retrying with a full fetch." + run git -C "$WORK_ROOT" config --unset remote.origin.promisor || true + run git -C "$WORK_ROOT" config --unset remote.origin.partialclonefilter || true + run git -C "$WORK_ROOT" fetch --tags --prune --prune-tags origin + fi + else + if [ "$FULL_CLONE" != true ]; then + warn "git $(git version | awk '{print $3}') has no partial clone; fetching everything." + fi + info "Fetching (full history)..." + run git -C "$WORK_ROOT" fetch --tags --prune --prune-tags origin + fi +} + +ensure_helper_excluded() { + local gitdir ex + + [ "$DRY_RUN" = true ] && { echo "[DRY-RUN] append '/helper/' to .git/info/exclude"; return 0; } + + gitdir="$(git -C "$WORK_ROOT" rev-parse --git-dir)" + case "$gitdir" in + /*) ;; + *) gitdir="$WORK_ROOT/$gitdir" ;; + esac + ex="$gitdir/info/exclude" + + if [ -f "$ex" ] && grep -qxF '/helper/' "$ex"; then + return 0 + fi + + info "Marking /helper/ as ignored in $ex" + mkdir -p "$gitdir/info" + printf '\n# Added by prepare-openwrt.sh: the build-helper checkout lives here.\n/helper/\n' >> "$ex" +} + +checkout_target() { + if [ -n "$CONFIG_BACKUP" ]; then + info "Backing up .config to $(basename "$CONFIG_BACKUP")" + run cp -a "$WORK_ROOT/.config" "$CONFIG_BACKUP" + fi + + # One-line insurance in case upstream ever ships a top-level 'helper' path. + if [ "$DRY_RUN" != true ] && + [ -n "$(git -C "$WORK_ROOT" ls-tree --name-only "$TARGET_REF" helper 2>/dev/null)" ]; then + err "The target ref '$TARGET_REF' contains a tracked 'helper' path." + err "It would collide with the helper checkout. Use --root for a different layout." exit 1 fi + + if [ "$TARGET_KIND" = "tag" ]; then + info "Checking out tag $TARGET_REF ..." + run git -C "$WORK_ROOT" checkout --detach "refs/tags/$TARGET_REF" + else + info "Checking out branch $TARGET_REF ..." + if git -C "$WORK_ROOT" show-ref --verify --quiet "refs/heads/$TARGET_REF"; then + run git -C "$WORK_ROOT" checkout "$TARGET_REF" + run git -C "$WORK_ROOT" merge --ff-only "origin/$TARGET_REF" + else + run git -C "$WORK_ROOT" checkout -b "$TARGET_REF" --track "origin/$TARGET_REF" + fi + fi + + restore_worktree_if_missing +} + +# Checking out the ref that HEAD already points at is a no-op, so it does not +# bring back tracked files that were deleted by an interrupted run. Only ever +# runs when the source tree is actually missing, so it cannot clobber edits. +restore_worktree_if_missing() { + [ "$DRY_RUN" = true ] && return 0 + is_openwrt_tree "$WORK_ROOT" && return 0 + + warn "Source tree is incomplete; restoring tracked files from $TARGET_REF." + run git -C "$WORK_ROOT" checkout --force HEAD -- . + + if ! is_openwrt_tree "$WORK_ROOT"; then + err "'$WORK_ROOT' still has no OpenWrt source tree after restoring." + exit 1 + fi +} + +# --------------------------------------------------------------------------- +# Phase F: feeds and defconfig +# --------------------------------------------------------------------------- +update_feeds() { + [ "$SKIP_FEEDS" = true ] && { info "Skipping feeds (--skip-feeds)."; return 0; } + + if [ "$DRY_RUN" != true ] && ! is_openwrt_tree "$WORK_ROOT"; then + err "'$WORK_ROOT' has no OpenWrt source tree." + exit 1 + fi + + info "Updating and installing all feeds..." + run env -C "$WORK_ROOT" ./scripts/feeds update -a + run env -C "$WORK_ROOT" ./scripts/feeds install -a +} + +run_defconfig() { + [ "$SKIP_DEFCONFIG" = true ] && { info "Skipping defconfig (--skip-defconfig)."; return 0; } + info "Running make defconfig..." + run env -C "$WORK_ROOT" make defconfig +} + +# --------------------------------------------------------------------------- +# Phase G: summary +# --------------------------------------------------------------------------- +print_summary() { + local version="$TARGET_REF" + + if [ "$DRY_RUN" != true ]; then + version="$(git -C "$WORK_ROOT" describe --tags 2>/dev/null || + git -C "$WORK_ROOT" rev-parse --abbrev-ref HEAD 2>/dev/null || echo "$TARGET_REF")" + fi + + echo + echo "========================================" + echo "βœ… OpenWrt source is ready." + echo " Version: $version" + echo " Root: $WORK_ROOT" + echo " Helper: $HELPER_DIR" + [ -n "$COMPAT_LINK_PATH" ] && echo " Old path: $COMPAT_LINK_PATH (symlink)" + echo "========================================" + echo + echo "⚠ helper/ lives inside the OpenWrt Git tree but is not tracked by it." + echo " 'git clean -xdff' in the OpenWrt root WILL DELETE it, .git and all." + echo " Use instead: git clean -xdf -e /helper" + echo + if [ -n "$COMPAT_LINK_PATH" ]; then + echo "β„Ή '$COMPAT_LINK_PATH' is now a symlink to '$HELPER_DIR'," + echo " so shells and tools still sitting in the old directory keep working." + echo " Run 'cd .' there to refresh a stale shell." + echo + fi + echo "πŸ‘‰ Next steps:" + echo " cd $WORK_ROOT" + echo " ./helper/download-config.sh # optional: import a device config" + echo " make menuconfig" + echo " make -j\$(nproc) download world" + echo +} + +# --------------------------------------------------------------------------- +# main +# --------------------------------------------------------------------------- +require_sort_v + +discover_layout +discover_remote_refs + +if [ -z "$VERSION_MODE" ]; then + if [ "$ASSUME_YES" = true ]; then + VERSION_MODE="stable" + else + choose_version_interactive + fi fi -# --- Update feeds --- -echo "[INFO] Updating and installing all feeds..." -./scripts/feeds update -a -./scripts/feeds install -a +resolve_target_ref +check_switch_safety +print_plan -# --- Run defconfig --- -echo "[INFO] Running make defconfig..." -make defconfig - -CURRENT_VERSION=$(git describe --tags 2>/dev/null || git rev-parse --abbrev-ref HEAD) - -popd >/dev/null +if [ "$DRY_RUN" = true ]; then + echo + relocate_helper + ensure_repo + ensure_helper_excluded + echo "[DRY-RUN] checkout $TARGET_REF" + echo + info "Dry run complete; nothing was changed." + exit 0 +fi echo -echo "========================================" -echo "βœ… OpenWRT source is ready." -echo " Version: $CURRENT_VERSION" -echo " Root: $WORK_ROOT" -echo "========================================" -echo -echo "πŸ‘‰ Next steps:" -echo " cd $WORK_ROOT" -echo " make menuconfig" -echo " make -j\$(nproc) download world" +if ! confirm "Proceed?" yes; then + info "Aborted by user." + exit 0 +fi + echo +# Everything below this line mutates the filesystem. +relocate_helper +acquire_lock +ensure_repo +ensure_helper_excluded +checkout_target +update_feeds +run_defconfig +print_summary