Zhe Yuan 40d3b21e71 Add AGENTS.md and fill the gaps in the README
AGENTS.md describes the project for anyone -- human or agent -- changing a
script: what the repository is, where it sits inside the OpenWrt tree, which
scripts run from where, the shell conventions the existing code follows, and
the bash traps that have already caused bugs here.

README: document the options that were missing (--refresh, --skip-defconfig,
--extras-file/--no-extras-file, --extras, --wrap), distinguish --extras from
--extras-file, and replace the Debian-only Go install line with a
distribution-neutral description of how update-go-path.sh finds a toolchain.
Drop the hedged MIT claim, since no LICENSE file exists.

Both documents stay environment-neutral: no absolute home paths and no
assumption of a distribution, with platform-specific advice kept under an
explicit conditional heading.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 10:59:41 -04:00

OpenWrt Build Helper Scripts

Shell scripts that automate building OpenWrt from source: installing the host dependencies, fetching the source tree, importing a device's official configuration, and keeping a per-device package list.

The design goal is that a new OpenWrt release requires no file changes — everything version specific is fetched from upstream, and only your own package choices are stored in this repository.

Layout

prepare-openwrt.sh normalizes the checkout on first run:

before        <parent>/openwrt-build-helper/       this repository
first run     <parent>/openwrt/                    the OpenWrt source root
              <parent>/openwrt/helper/             this repository, moved in
              <parent>/openwrt-build-helper -> openwrt/helper   (symlink)
later runs    the same <parent>/openwrt/ is updated in place

The symlink means shells, editors and agent sessions still sitting in the old path keep working. Disable it with --no-compat-link.

Inside helper/:

Path In git Purpose
packages/<device-id>.txt yes your packages for one device, version independent
.generated/ no combined package lists and saved buildinfo (build artefacts)

Requirements

  • Linux with one of: apt (Debian/Ubuntu/Mint), dnf/yum (Fedora/RHEL/Rocky/Alma), pacman (Arch/Manjaro), zypper (openSUSE), apk (Alpine)
  • x86_64/amd64 or aarch64/arm64
  • git, curl, wget, bash, python3 (used to read the upstream JSON indexes; jq is not required)
  • Internet access, and roughly 30 GB of disk for a full build

Quick start

# 1. host dependencies
./prepare-openwrt-env.sh                    # -n to preview, --with-go for Go

# 2. OpenWrt source (relocates this checkout to openwrt/helper on first run)
./prepare-openwrt.sh                        # or --stable 25.12.5 -y

# from here on, run everything from the OpenWrt root
cd ../openwrt        # or: cd openwrt

# 3. the device's official configuration
./helper/download-config.sh --device linksys_mx8500 -y

# 4. built-in packages + your own, applied to .config
./helper/gen-package-list.sh --device linksys_mx8500 --apply -y

# 5. build
make -j$(nproc) download world

Firmware lands in bin/targets/<target>/<subtarget>/.

Run ./helper/download-config.sh without --device to search for a device by name. Add --with-packages to fold step 4 into step 3.

Adding a device

./helper/gen-package-list.sh --device <id> --init-extras   # creates packages/<id>.txt
$EDITOR helper/packages/<id>.txt                           # list what you want
./helper/gen-package-list.sh --device <id> --apply

Upgrading to a new OpenWrt release

Nothing to edit — the built-in list is re-fetched and packages/<id>.txt is reused as is:

./helper/prepare-openwrt.sh --stable -y
cd ../openwrt
./helper/download-config.sh --device <id> -y
./helper/gen-package-list.sh --device <id> --apply -y
make -j$(nproc) download world

Scripts

Every script takes -h/--help and prints its full option list; the summaries below cover the options worth knowing about.

  • prepare-openwrt-env.sh — installs the build dependencies from the official build system guide. Dispatches on the package manager rather than guessing from the distro version, and probes every package against the running release so a renamed or dropped package cannot abort the install. -y, -n/--dry-run, --with-go.

  • prepare-openwrt.sh — clones or updates the OpenWrt source in openwrt/. Resolves the version over git ls-remote and prints a plan before touching anything, so aborting at the prompt leaves the checkout untouched. Uses a partial clone (--filter=blob:none) by default and resumes an interrupted first run automatically. --stable [<ver>], --branch <name>, --snapshot, -y, -n, -f/--force, --full-clone, --allow-rc, --root <dir>, --skip-feeds, --skip-defconfig, --repo-url <url>, --no-compat-link.

  • download-config.sh — installs a device's official config.buildinfo as .config. Looks the device up in the official index, so any supported device can be selected by name. Downloads to a temp file and validates it before touching .config, keeping a .config.download.bak. By default builds only the selected device and strips the buildbot flags (see below). --device <id>, --target <t>, --version <ver>, --snapshot, --with-packages, --all-profiles, --keep-buildbot-flags, --save-buildinfo, --offline, --output <file>, --refresh, --skip-defconfig, -y, -n.

  • gen-package-list.sh — builds the complete package list: the built-in part from the target's profiles.json (default_packages + device_packages plus the two packages the Firmware Selector adds), combined with your packages/<device-id>.txt. --device <id>, --target <t>, --version <ver>, --snapshot, --apply, --init-extras, --extras-file <f>, --no-extras-file, --stdout, --full-stdout, --out <path>, --extras "<pkgs>", --no-extras, --wrap <cols>, --refresh, -y, -n.

    --extras/--no-extras control the two packages the Firmware Selector adds on top of the target's own set; --extras-file/--no-extras-file control your packages/<device-id>.txt. They are different things.

  • add-openwrt-packages.sh — applies a package list to .config. Rewrites existing CONFIG_PACKAGE_ lines in place, so repeated runs do not grow the file. After make defconfig it reports any package whose final state differs from what you asked for. -y, -n, --no-defconfig, --no-backup, --strict, and - to read stdin.

  • update-go-path.sh — points the build at an installed Go toolchain. --external clears CONFIG_GOLANG_BUILD_BOOTSTRAP so the installed toolchain builds host Go directly instead of OpenWrt building the whole bootstrap chain from source. --external, --go-root <dir>, --skip-defconfig, -n.

  • add-external-repos.sh — clones extra package repositories into package/. Edit the REPOS array to change the list.

Package list format

Used by both packages/<device-id>.txt and the generated lists:

curl yq luci-app-ttyd      packages, any number per line, set to =y
-luci-app-wol              a leading '-' removes a package (=n)
##module                   packages after this are set to =m
##remove                   packages after this are set to =n
##built-in                 back to =y (the default)
pkg  # note                text after a single '#' is a comment

Notes and gotchas

Buildbot flags

The official config.buildinfo is the buildbot's own configuration: it enables every model in the target and sets ALL_KMODS, ALL_NONSHARED, SDK, IB, MAKE_TOOLCHAIN, COLLECT_KERNEL_DEBUG and AUTOREMOVE. For a Linksys MX8500 that is 1248 extra module packages plus the SDK, ImageBuilder and a toolchain tarball on every build.

download-config.sh turns these off by default. This was verified not to change the firmware: the set of packages built into the image (=y) is identical either way — 193 packages before and after. The trade-off is that bin/ no longer contains prebuilt packages for modules you did not select, so you cannot later install an arbitrary kmod from your own build output. --keep-buildbot-flags and --all-profiles restore upstream behaviour.

Go packages

Packages such as tailscale, adguardhome, easytier and cloudflared need a host Go. By default OpenWrt builds the entire bootstrap chain from source, which works but is slow. To skip it, install a Go from your distribution (or from go.dev) at least as new as the version the tree expects, then run:

./helper/update-go-path.sh --external

The script searches /usr/lib/go-*, /usr/local/go and /usr/lib/golang, picks the newest, and warns if it is older than the bootstrap version the tree asks for. Point it elsewhere with --go-root <dir>.

WSL: Windows PATH breaks Go package builds

On WSL, PATH includes Windows directories containing spaces and parentheses (/mnt/c/Program Files (x86)/...). OpenWrt's Go build recipe interpolates PATH unquoted into a shell command, so every Go package fails with:

bash: -c: line 1: syntax error near unexpected token `('

Either build with a sanitized PATH:

CLEAN_PATH=$(echo "$PATH" | tr ':' '\n' | grep -v '^/mnt/' | paste -sd:)
env PATH="$CLEAN_PATH" make -j$(nproc) download world

or disable the interop PATH permanently in /etc/wsl.conf and restart WSL (wsl --shutdown):

[interop]
appendWindowsPath = false

Hazard: git clean -xdff

helper/ lives inside the OpenWrt Git worktree but is not tracked by it. prepare-openwrt.sh adds /helper/ to openwrt/.git/info/exclude, which keeps git status clean and stops git add -A from staging it — but that does not protect it from git clean -xdff: -x deliberately includes ignored files and -ff removes Git's refusal to recurse into a nested repository. Use instead:

git clean -xdf -e /helper

Troubleshooting

  • Version detection in download-config.sh — it uses, in order: an explicit --version/--snapshot; the exact tag at HEAD; the branch (main/master → SNAPSHOT, openwrt-XX.YY → the newest matching tag merged into HEAD); otherwise it asks. If HEAD is ahead of the tag it warns that the downloaded config describes the release rather than your tree. Version queries are anchored to the OpenWrt tree, not to whichever repository the script sits in.
  • The device symbol did not survive make defconfig — your tree does not match the release you downloaded a config for. The script stops and points at the .config.download.bak it kept.
  • A package is reported missing — names are verified against tmp/.config-package.in and tmp/.packageinfo. Enable the relevant feed in feeds.conf.default, then ./scripts/feeds update -a && ./scripts/feeds install -a.
  • A removed package (-pkg) comes back after make defconfig — something else depends on it. Reported explicitly as "wanted =n, got =y".
  • A package is in .config but not in the manifest — it is probably there under a provider name (libgcclibgcc1, nftablesnftables-json).
  • Rebuild one packagemake package/<pkg>/{clean,compile} V=s
  • Full rebuildmake clean; make -j$(nproc) download world

Safety

  • Scripts use set -euo pipefail and umask 022, and need root only to install host packages.
  • .config is backed up before it is replaced or before a version switch.
  • prepare-openwrt.sh refuses to switch versions on a tree with modified tracked files unless you pass --force.
  • Scripts write under package/ and feeds/ inside your OpenWrt tree.

Contributing and authorship

  • Scripts authored by Zhe Yuan.
  • No LICENSE file is present yet; add one before redistributing.
  • AGENTS.md documents the layout, the shell conventions and the invariants to preserve — read it before changing a script.
Description
No description provided
Readme 583 KiB
Languages
Shell 100%