Zhe Yuan 2abd8aba00 Rewrite the README for the current script set
The README had been patched incrementally across several changes and no longer
matched the repository: sections about adding a device and the package list
format had ended up nested under Quick start, and the layout it described
predated the per-device packages/ directory.

Restructure it around what someone actually does: layout, requirements, a
five-step quick start, then per-script reference. Document the parts that are
easy to get wrong and were learned the hard way — the buildbot flags and why
stripping them is safe, the Go toolchain options, the WSL PATH problem that
breaks every Go package, and the git clean -xdff hazard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-06 00:28:54 -04:00

OpenWrt Build Helper Scripts

Shell scripts that automate building OpenWrt from source: installing the host dependencies, fetching the source tree, importing a device's official configuration, and keeping a per-device package list.

The design goal is that a new OpenWrt release requires no file changes — everything version specific is fetched from upstream, and only your own package choices are stored in this repository.

Layout

prepare-openwrt.sh normalizes the checkout on first run:

before        <parent>/openwrt-build-helper/       this repository
first run     <parent>/openwrt/                    the OpenWrt source root
              <parent>/openwrt/helper/             this repository, moved in
              <parent>/openwrt-build-helper -> openwrt/helper   (symlink)
later runs    the same <parent>/openwrt/ is updated in place

The symlink means shells, editors and agent sessions still sitting in the old path keep working. Disable it with --no-compat-link.

Inside helper/:

Path In git Purpose
packages/<device-id>.txt yes your packages for one device, version independent
.generated/ no combined package lists and saved buildinfo (build artefacts)

Requirements

  • Linux with one of: apt (Debian/Ubuntu/Mint), dnf/yum (Fedora/RHEL/Rocky/Alma), pacman (Arch/Manjaro), zypper (openSUSE), apk (Alpine)
  • x86_64/amd64 or aarch64/arm64
  • git, curl, wget, bash, python3 (used to read the upstream JSON indexes; jq is not required)
  • Internet access, and roughly 30 GB of disk for a full build

Quick start

# 1. host dependencies
./prepare-openwrt-env.sh                    # -n to preview, --with-go for Go

# 2. OpenWrt source (relocates this checkout to openwrt/helper on first run)
./prepare-openwrt.sh                        # or --stable 25.12.5 -y

# from here on, run everything from the OpenWrt root
cd ../openwrt        # or: cd openwrt

# 3. the device's official configuration
./helper/download-config.sh --device linksys_mx8500 -y

# 4. built-in packages + your own, applied to .config
./helper/gen-package-list.sh --device linksys_mx8500 --apply -y

# 5. build
make -j$(nproc) download world

Firmware lands in bin/targets/<target>/<subtarget>/.

Run ./helper/download-config.sh without --device to search for a device by name. Add --with-packages to fold step 4 into step 3.

Adding a device

./helper/gen-package-list.sh --device <id> --init-extras   # creates packages/<id>.txt
$EDITOR helper/packages/<id>.txt                           # list what you want
./helper/gen-package-list.sh --device <id> --apply

Upgrading to a new OpenWrt release

Nothing to edit — the built-in list is re-fetched and packages/<id>.txt is reused as is:

./helper/prepare-openwrt.sh --stable -y
cd ../openwrt
./helper/download-config.sh --device <id> -y
./helper/gen-package-list.sh --device <id> --apply -y
make -j$(nproc) download world

Scripts

  • prepare-openwrt-env.sh — installs the build dependencies from the official build system guide. Dispatches on the package manager rather than guessing from the distro version, and probes every package against the running release so a renamed or dropped package cannot abort the install. -y, -n/--dry-run, --with-go.

  • prepare-openwrt.sh — clones or updates the OpenWrt source in openwrt/. Resolves the version over git ls-remote and prints a plan before touching anything, so aborting at the prompt leaves the checkout untouched. Uses a partial clone (--filter=blob:none) by default and resumes an interrupted first run automatically. --stable [<ver>], --branch <name>, --snapshot, -y, -n, -f/--force, --full-clone, --allow-rc, --root <dir>, --skip-feeds, --skip-defconfig, --repo-url <url>, --no-compat-link.

  • download-config.sh — installs a device's official config.buildinfo as .config. Looks the device up in the official index, so any supported device can be selected by name. Downloads to a temp file and validates it before touching .config, keeping a .config.download.bak. By default builds only the selected device and strips the buildbot flags (see below). --device <id>, --target <t>, --version <ver>, --snapshot, --with-packages, --all-profiles, --keep-buildbot-flags, --save-buildinfo, --offline, --output <file>, -y, -n.

  • gen-package-list.sh — builds the complete package list: the built-in part from the target's profiles.json (default_packages + device_packages plus the two packages the Firmware Selector adds), combined with your packages/<device-id>.txt. --device <id>, --apply, --init-extras, --stdout, --full-stdout, --out <path>, --no-extras, -y, -n.

  • add-openwrt-packages.sh — applies a package list to .config. Rewrites existing CONFIG_PACKAGE_ lines in place, so repeated runs do not grow the file. After make defconfig it reports any package whose final state differs from what you asked for. -y, -n, --no-defconfig, --no-backup, --strict, and - to read stdin.

  • update-go-path.sh — points the build at an installed Go toolchain. --external clears CONFIG_GOLANG_BUILD_BOOTSTRAP so the installed toolchain builds host Go directly instead of OpenWrt building the whole bootstrap chain from source. --external, --go-root <dir>, -n.

  • add-external-repos.sh — clones extra package repositories into package/. Edit the REPOS array to change the list.

Package list format

Used by both packages/<device-id>.txt and the generated lists:

curl yq luci-app-ttyd      packages, any number per line, set to =y
-luci-app-wol              a leading '-' removes a package (=n)
##module                   packages after this are set to =m
##remove                   packages after this are set to =n
##built-in                 back to =y (the default)
pkg  # note                text after a single '#' is a comment

Notes and gotchas

Buildbot flags

The official config.buildinfo is the buildbot's own configuration: it enables every model in the target and sets ALL_KMODS, ALL_NONSHARED, SDK, IB, MAKE_TOOLCHAIN, COLLECT_KERNEL_DEBUG and AUTOREMOVE. For a Linksys MX8500 that is 1248 extra module packages plus the SDK, ImageBuilder and a toolchain tarball on every build.

download-config.sh turns these off by default. This was verified not to change the firmware: the set of packages built into the image (=y) is identical either way — 193 packages before and after. The trade-off is that bin/ no longer contains prebuilt packages for modules you did not select, so you cannot later install an arbitrary kmod from your own build output. --keep-buildbot-flags and --all-profiles restore upstream behaviour.

Go packages

Packages such as tailscale, adguardhome, easytier and cloudflared need a host Go. By default OpenWrt builds the entire bootstrap chain from source, which works but is slow. To skip it, install a Go matching the version the tree expects and run:

sudo apt install golang-1.24-go
./helper/update-go-path.sh --external

WSL: Windows PATH breaks Go package builds

On WSL, PATH includes Windows directories containing spaces and parentheses (/mnt/c/Program Files (x86)/...). OpenWrt's Go build recipe interpolates PATH unquoted into a shell command, so every Go package fails with:

bash: -c: line 1: syntax error near unexpected token `('

Either build with a sanitized PATH:

CLEAN_PATH=$(echo "$PATH" | tr ':' '\n' | grep -v '^/mnt/' | paste -sd:)
env PATH="$CLEAN_PATH" make -j$(nproc) download world

or disable the interop PATH permanently in /etc/wsl.conf and restart WSL (wsl --shutdown):

[interop]
appendWindowsPath = false

Hazard: git clean -xdff

helper/ lives inside the OpenWrt Git worktree but is not tracked by it. prepare-openwrt.sh adds /helper/ to openwrt/.git/info/exclude, which keeps git status clean and stops git add -A from staging it — but that does not protect it from git clean -xdff: -x deliberately includes ignored files and -ff removes Git's refusal to recurse into a nested repository. Use instead:

git clean -xdf -e /helper

Troubleshooting

  • Version detection in download-config.sh — it uses, in order: an explicit --version/--snapshot; the exact tag at HEAD; the branch (main/master → SNAPSHOT, openwrt-XX.YY → the newest matching tag merged into HEAD); otherwise it asks. If HEAD is ahead of the tag it warns that the downloaded config describes the release rather than your tree. Version queries are anchored to the OpenWrt tree, not to whichever repository the script sits in.
  • The device symbol did not survive make defconfig — your tree does not match the release you downloaded a config for. The script stops and points at the .config.download.bak it kept.
  • A package is reported missing — names are verified against tmp/.config-package.in and tmp/.packageinfo. Enable the relevant feed in feeds.conf.default, then ./scripts/feeds update -a && ./scripts/feeds install -a.
  • A removed package (-pkg) comes back after make defconfig — something else depends on it. Reported explicitly as "wanted =n, got =y".
  • A package is in .config but not in the manifest — it is probably there under a provider name (libgcclibgcc1, nftablesnftables-json).
  • Rebuild one packagemake package/<pkg>/{clean,compile} V=s
  • Full rebuildmake clean; make -j$(nproc) download world

Safety

  • Scripts use set -euo pipefail and umask 022, and need root only to install host packages.
  • .config is backed up before it is replaced or before a version switch.
  • prepare-openwrt.sh refuses to switch versions on a tree with modified tracked files unless you pass --force.
  • Scripts write under package/ and feeds/ inside your OpenWrt tree.

License and authorship

  • Scripts authored by Zhe Yuan.
  • License: MIT (unless you choose a different license; update this line accordingly).
Description
No description provided
Readme 583 KiB
Languages
Shell 100%