yuanzhe1999/openwrt-build-helper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rewrite the README for the current script set

Browse Source 
The README had been patched incrementally across several changes and no longer
matched the repository: sections about adding a device and the package list
format had ended up nested under Quick start, and the layout it described
predated the per-device packages/ directory.

Restructure it around what someone actually does: layout, requirements, a
five-step quick start, then per-script reference. Document the parts that are
easy to get wrong and were learned the hard way — the buildbot flags and why
stripping them is safe, the Go toolchain options, the WSL PATH problem that
breaks every Go package, and the git clean -xdff hazard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Zhe Yuan
2026-09-06 00:28:54 -04:00
parent a98b1e1c76
commit 2abd8aba00
1 changed files with 200 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files

366
README.md
View File

@@ -1,113 +1,144 @@
# OpenWrt Build Helper Scripts # OpenWrt Build Helper Scripts
## Overview Shell scripts that automate building OpenWrt from source: installing the host
This repository contains shell scripts that streamline preparing, configuring, and building OpenWrt from source on Debian/Ubuntu systems (x86_64 and ARM64). It also includes convenience utilities to manage package selections, fetch a device's official configuration, and add external LuCI packages. dependencies, fetching the source tree, importing a device's official
configuration, and keeping a per-device package list.
## What’s included The design goal is that **a new OpenWrt release requires no file changes** —
- prepare-openwrt-env.sh everything version specific is fetched from upstream, and only your own package
- Installs the build dependencies listed by the official OpenWrt build system guide. choices are stored in this repository.
- Supports apt (Debian/Ubuntu/Mint), dnf/yum (Fedora/RHEL/Rocky/Alma), pacman (Arch/Manjaro), zypper (openSUSE) and apk (Alpine).
- Probes each package against the running release and skips the ones that no longer exist, so a renamed or dropped package cannot abort the whole install.
- Handles x86_64 and ARM64 differences (native vs. cross multilib).
- Options: `-y/--yes` (no prompt), `-n/--dry-run` (only print), `--with-go` (also install the Go toolchain).
- prepare-openwrt.sh
- Clones or updates the OpenWrt source repo in the top-level `openwrt/` root.
- Lets you select a stable tag, a release branch, or the snapshot (`main`) interactively, or non-interactively via flags.
- Uses the script location as the source of truth, so running it from another current working directory is supported.
- On first run from a helper checkout, relocates that checkout to `openwrt/helper/` and leaves a symlink behind at the old path.
- Prints a plan and asks for confirmation; nothing on disk is touched before you confirm.
- Uses a partial clone (`--filter=blob:none`) by default, so the first fetch is far smaller than full history.
- Resumes automatically if a previous run was interrupted.
- Updates feeds and runs make defconfig.
- Options: `--stable [<ver>]`, `--branch <name>`, `--snapshot`, `-y/--yes`, `-n/--dry-run`,
`-f/--force`, `--full-clone`, `--allow-rc`, `--root <dir>`, `--skip-feeds`,
`--skip-defconfig`, `--repo-url <url>`, `--no-compat-link`, `-h/--help`.
- add-external-repos.sh
- Clones or updates additional package repositories into package/.
- Currently includes luci-app-netspeedtest and luci-app-easytier (adjust REPOS as needed).
- download-config.sh
- Downloads config.buildinfo for a device and installs it as .config.
- Looks the device up in the official index, so any supported device can be picked by name; nothing is hardcoded. Use `--device <id>` for non-interactive runs.
- By default builds only the selected device instead of every model in the target, and turns off the buildbot flags (ALL_KMODS, ALL_NONSHARED, SDK, IB, MAKE_TOOLCHAIN, COLLECT_KERNEL_DEBUG, AUTOREMOVE). Neither changes the firmware contents — see "Buildbot flags" below.
- Downloads to a temp file and validates it before touching .config, and keeps a .config.download.bak.
- Prints the official sysupgrade/factory URLs (taken from profiles.json, so they are always correct) and a Firmware Selector link.
- gen-package-list.sh
- Builds a device's complete package list: the built-in part from the official profiles.json (default_packages + device_packages + the packages the Firmware Selector adds), combined with your own packages from `packages/<device-id>.txt`.
- The built-in part never has to be copied by hand for a new release, and your packages are version independent — a new release needs no file changes at all.
- Writes the combined list to `.generated/` (not kept in git); `--apply` applies it straight to .config, `--full-stdout` just prints it.
- add-openwrt-packages.sh
- Applies a package list to .config: `##built-in` → =y, `##module` → =m, `##remove` or a `-pkg` prefix → =n.
- Rewrites existing CONFIG_PACKAGE_ lines in place, so repeated runs do not grow .config.
- Verifies each package against tmp/.config-package.in and tmp/.packageinfo and reports missing ones.
- After make defconfig it reports any package whose final state differs from what you asked for (a dependency pulling a removed package back in, for example).
- update-go-path.sh
- Detects the latest Go installation under /usr/lib/go-* and sets CONFIG_GOLANG_EXTERNAL_BOOTSTRAP_ROOT in .config.
- OS: Debian/Ubuntu, Fedora/RHEL, Arch, openSUSE or Alpine (the other scripts are developed and tested on Debian/Ubuntu) ## Layout
- Architectures: x86_64/amd64 and aarch64/arm64
- Tools: git, wget, curl, bash, python3 (used to read the official JSON indexes; jq is not required) `prepare-openwrt.sh` normalizes the checkout on first run:
- Internet access for feeds and package downloads
- Optional: Go (required by some OpenWrt packages; install it with `./prepare-openwrt-env.sh --with-go`; configurable via update-go-path.sh) ```
- Note: with the default partial clone, switching to another tag or branch later fetches missing blobs on demand and therefore needs network access before <parent>/openwrt-build-helper/ this repository
first run <parent>/openwrt/ the OpenWrt source root
<parent>/openwrt/helper/ this repository, moved in
<parent>/openwrt-build-helper -> openwrt/helper (symlink)
later runs the same <parent>/openwrt/ is updated in place
```
The symlink means shells, editors and agent sessions still sitting in the old
path keep working. Disable it with `--no-compat-link`.
Inside `helper/`:
| Path | In git | Purpose |
|---|---|---|
| `packages/<device-id>.txt` | yes | your packages for one device, version independent |
| `.generated/` | no | combined package lists and saved buildinfo (build artefacts) |
## Requirements
- Linux with one of: apt (Debian/Ubuntu/Mint), dnf/yum (Fedora/RHEL/Rocky/Alma),
pacman (Arch/Manjaro), zypper (openSUSE), apk (Alpine)
- x86_64/amd64 or aarch64/arm64
- git, curl, wget, bash, python3 (used to read the upstream JSON indexes; jq is
not required)
- Internet access, and roughly 30 GB of disk for a full build
## Quick start ## Quick start
1) Prepare the host machine
- Run: ./prepare-openwrt-env.sh
- This installs compilers, headers, Python tooling, and other build prerequisites.
- Preview without touching the system: ./prepare-openwrt-env.sh -n
2) Get OpenWrt sources ```bash
- Run: ./prepare-openwrt.sh # 1. host dependencies
- Choose a stable tag, a release branch, or the snapshot. ./prepare-openwrt-env.sh # -n to preview, --with-go for Go
- Non-interactive: ./prepare-openwrt.sh --stable 25.12.5 -y
- Preview only, changes nothing: ./prepare-openwrt.sh --stable -n
- The script:
- Reuses or creates the top-level OpenWrt root at `./openwrt`.
- Moves this helper checkout to `./openwrt/helper` on first run.
- Updates/install feeds and runs make defconfig.
3) Add optional external packages # 2. OpenWrt source (relocates this checkout to openwrt/helper on first run)
- From the OpenWrt root: ./prepare-openwrt.sh # or --stable 25.12.5 -y
- ./helper/add-external-repos.sh
- This pulls extra LuCI packages into package/.
4) Import a device config (optional but convenient) # from here on, run everything from the OpenWrt root
- From the OpenWrt root: cd ../openwrt # or: cd openwrt
- ./helper/download-config.sh
- Search for your device by name, or run it non-interactively:
- ./helper/download-config.sh --device linksys_mx8500 -y
- Add `--with-packages` to also generate the device's package list in the same run.
- Preview without changing anything: `./helper/download-config.sh --device linksys_mx8500 -n`
5) Enable additional packages (optional) # 3. the device's official configuration
- Your own packages live in `helper/packages/<device-id>.txt`, one file per device, independent of the OpenWrt version. ./helper/download-config.sh --device linksys_mx8500 -y
- Apply the built-in list plus your packages in one step:
- ./helper/gen-package-list.sh --device linksys_mx8500 --apply # 4. built-in packages + your own, applied to .config
- Or write the combined list out first and review it before applying: ./helper/gen-package-list.sh --device linksys_mx8500 --apply -y
- ./helper/gen-package-list.sh --device linksys_mx8500
- ./helper/add-openwrt-packages.sh helper/.generated/packages_linksys_mx8500_<version>.txt # 5. build
make -j$(nproc) download world
```
Firmware lands in `bin/targets/<target>/<subtarget>/`.
Run `./helper/download-config.sh` without `--device` to search for a device by
name. Add `--with-packages` to fold step 4 into step 3.
### Adding a device ### Adding a device
1. `./helper/gen-package-list.sh --device <id> --init-extras` — creates `packages/<id>.txt`
2. Edit `packages/<id>.txt` and list the packages you want on top of the defaults
3. `./helper/gen-package-list.sh --device <id> --apply`
Use `./helper/download-config.sh` without `--device` to search for the id by name. ```bash
./helper/gen-package-list.sh --device <id> --init-extras # creates packages/<id>.txt
$EDITOR helper/packages/<id>.txt # list what you want
./helper/gen-package-list.sh --device <id> --apply
```
### Upgrading to a new OpenWrt release ### Upgrading to a new OpenWrt release
Nothing to edit. Check out the new version, then:
``` Nothing to edit — the built-in list is re-fetched and `packages/<id>.txt` is
reused as is:
```bash
./helper/prepare-openwrt.sh --stable -y
cd ../openwrt
./helper/download-config.sh --device <id> -y ./helper/download-config.sh --device <id> -y
./helper/gen-package-list.sh --device <id> --apply -y ./helper/gen-package-list.sh --device <id> --apply -y
make -j$(nproc) download world
``` ```
The built-in list is re-fetched for the new release and your `packages/<id>.txt` is reused as is.
### Files ## Scripts
- `packages/<device-id>.txt` — your packages, one file per device, kept in git and edited by hand.
- `.generated/` — combined package lists and saved buildinfo. Build artefacts, not kept in git. - **prepare-openwrt-env.sh** — installs the build dependencies from the official
build system guide. Dispatches on the package manager rather than guessing from
the distro version, and probes every package against the running release so a
renamed or dropped package cannot abort the install.
`-y`, `-n/--dry-run`, `--with-go`.
- **prepare-openwrt.sh** — clones or updates the OpenWrt source in `openwrt/`.
Resolves the version over `git ls-remote` and prints a plan before touching
anything, so aborting at the prompt leaves the checkout untouched. Uses a
partial clone (`--filter=blob:none`) by default and resumes an interrupted
first run automatically.
`--stable [<ver>]`, `--branch <name>`, `--snapshot`, `-y`, `-n`, `-f/--force`,
`--full-clone`, `--allow-rc`, `--root <dir>`, `--skip-feeds`,
`--skip-defconfig`, `--repo-url <url>`, `--no-compat-link`.
- **download-config.sh** — installs a device's official `config.buildinfo` as
`.config`. Looks the device up in the official index, so any supported device
can be selected by name. Downloads to a temp file and validates it before
touching `.config`, keeping a `.config.download.bak`. By default builds only
the selected device and strips the buildbot flags (see below).
`--device <id>`, `--target <t>`, `--version <ver>`, `--snapshot`,
`--with-packages`, `--all-profiles`, `--keep-buildbot-flags`,
`--save-buildinfo`, `--offline`, `--output <file>`, `-y`, `-n`.
- **gen-package-list.sh** — builds the complete package list: the built-in part
from the target's `profiles.json` (`default_packages` + `device_packages` plus
the two packages the Firmware Selector adds), combined with your
`packages/<device-id>.txt`.
`--device <id>`, `--apply`, `--init-extras`, `--stdout`, `--full-stdout`,
`--out <path>`, `--no-extras`, `-y`, `-n`.
- **add-openwrt-packages.sh** — applies a package list to `.config`. Rewrites
existing `CONFIG_PACKAGE_` lines in place, so repeated runs do not grow the
file. After `make defconfig` it reports any package whose final state differs
from what you asked for.
`-y`, `-n`, `--no-defconfig`, `--no-backup`, `--strict`, and `-` to read stdin.
- **update-go-path.sh** — points the build at an installed Go toolchain.
`--external` clears `CONFIG_GOLANG_BUILD_BOOTSTRAP` so the installed toolchain
builds host Go directly instead of OpenWrt building the whole bootstrap chain
from source.
`--external`, `--go-root <dir>`, `-n`.
- **add-external-repos.sh** — clones extra package repositories into `package/`.
Edit the `REPOS` array to change the list.
## Package list format
### Package list format
Used by both `packages/<device-id>.txt` and the generated lists: Used by both `packages/<device-id>.txt` and the generated lists:
``` ```
curl yq luci-app-ttyd packages, any number per line, set to =y curl yq luci-app-ttyd packages, any number per line, set to =y
-luci-app-wol a leading '-' removes a package (=n) -luci-app-wol a leading '-' removes a package (=n)
@@ -117,95 +148,98 @@ curl yq luci-app-ttyd packages, any number per line, set to =y
pkg # note text after a single '#' is a comment pkg # note text after a single '#' is a comment
``` ```
## Notes and gotchas
### Buildbot flags ### Buildbot flags
The official config.buildinfo is the buildbot's own configuration: it enables every The official `config.buildinfo` is the buildbot's own configuration: it enables
model in the target and sets ALL_KMODS / ALL_NONSHARED / SDK / IB / MAKE_TOOLCHAIN / every model in the target and sets `ALL_KMODS`, `ALL_NONSHARED`, `SDK`, `IB`,
COLLECT_KERNEL_DEBUG / AUTOREMOVE. For a Linksys MX8500 that means 1248 extra module `MAKE_TOOLCHAIN`, `COLLECT_KERNEL_DEBUG` and `AUTOREMOVE`. For a Linksys MX8500
packages plus the SDK, ImageBuilder and a toolchain tarball on every build. that is 1248 extra module packages plus the SDK, ImageBuilder and a toolchain
download-config.sh turns these off by default. This was verified not to change the tarball on every build.
firmware: the set of packages built into the image (`=y`) is identical either way —
193 packages before and after. The trade-off is that bin/ no longer contains prebuilt
packages for modules you did not select, so you cannot later install an arbitrary kmod
from your own build output. Use `--keep-buildbot-flags` to restore upstream behaviour,
and `--all-profiles` to build every model in the target.
6) Configure Go bootstrap path (if needed) `download-config.sh` turns these off by default. This was verified not to change
- From the OpenWrt root: the firmware: the set of packages built into the image (`=y`) is identical either
- ./helper/update-go-path.sh way — 193 packages before and after. The trade-off is that `bin/` no longer
- This sets CONFIG_GOLANG_EXTERNAL_BOOTSTRAP_ROOT to the latest /usr/lib/go-X.XX/ found. contains prebuilt packages for modules you did not select, so you cannot later
- You can override it manually in .config if necessary. install an arbitrary kmod from your own build output.
`--keep-buildbot-flags` and `--all-profiles` restore upstream behaviour.
### Go packages
Packages such as tailscale, adguardhome, easytier and cloudflared need a host Go.
By default OpenWrt builds the entire bootstrap chain from source, which works but
is slow. To skip it, install a Go matching the version the tree expects and run:
- Ensure telephony feed is installed: ```bash
- ./scripts/feeds update telephony sudo apt install golang-1.24-go
- ./scripts/feeds install -a ./helper/update-go-path.sh --external
- Then run: ```
- Build the package: ### WSL: Windows PATH breaks Go package builds
On WSL, `PATH` includes Windows directories containing spaces and parentheses
(`/mnt/c/Program Files (x86)/...`). OpenWrt's Go build recipe interpolates `PATH`
unquoted into a shell command, so every Go package fails with:
- Common commands: ```
- make menuconfig bash: -c: line 1: syntax error near unexpected token `('
- make -j$(nproc) download world ```
- Artifacts will be in bin/ after the build completes.
## Script usage notes and tips Either build with a sanitized PATH:
- Run locations:
- `prepare-openwrt.sh`: resolves paths from the script location, not your current shell directory. ```bash
- `prepare-openwrt-env.sh`: can be run before the first normalization from the fresh helper checkout. CLEAN_PATH=$(echo "$PATH" | tr ':' '\n' | grep -v '^/mnt/' | paste -sd:)
- All other helper scripts: run them from inside the OpenWrt source root as `./helper/...` unless otherwise indicated. env PATH="$CLEAN_PATH" make -j$(nproc) download world
- Feeds: ```
- If a package can’t be found, ensure feeds are updated/installed:
- ./scripts/feeds update -a && ./scripts/feeds install -a or disable the interop PATH permanently in `/etc/wsl.conf` and restart WSL
- Go toolchain: (`wsl --shutdown`):
- On x86_64, Go is optional unless your selection pulls packages needing Go (e.g., firewall4 in some versions). If needed:
- sudo apt install golang -y ```ini
- Then use update-go-path.sh to set .config automatically. [interop]
- ARM64 multilib: appendWindowsPath = false
- The script installs cross multilib packages for better compatibility, but not all targets need them. If apt errors on specific multilib packages, remove or adjust as appropriate for your environment. ```
### Hazard: `git clean -xdff`
`helper/` lives inside the OpenWrt Git worktree but is not tracked by it.
`prepare-openwrt.sh` adds `/helper/` to `openwrt/.git/info/exclude`, which keeps
`git status` clean and stops `git add -A` from staging it — but that does **not**
protect it from `git clean -xdff`: `-x` deliberately includes ignored files and
`-ff` removes Git's refusal to recurse into a nested repository. Use instead:
```bash
git clean -xdf -e /helper
```
## Troubleshooting ## Troubleshooting
- Version detection in download-config.sh:
- It uses, in order: an explicit `--version`/`--snapshot`; the exact tag at HEAD; the branch (`main`/`master` → SNAPSHOT, `openwrt-XX.YY` → the newest matching tag merged into HEAD); otherwise it asks. If HEAD is ahead of the tag it warns that the downloaded config describes the release rather than your tree.
- If the device symbol does not survive `make defconfig`, the script stops and tells you the tree does not match that release, and points at the .config.download.bak it kept.
- Missing packages in add-openwrt-packages.sh:
- It verifies names against tmp/.config-package.in and tmp/.packageinfo. If a package is reported missing, ensure the corresponding feed is enabled in feeds.conf.default and run feeds update/install.
- A removed package (`-pkg`) comes back after make defconfig:
- Something else depends on it. The script reports this explicitly as "wanted =n, got =y".
- Permission errors:
- Scripts use umask 022 and do not require root except when installing apt packages.
- Clean up and rebuild a package:
- make package/<pkg>/{clean,compile} V=s
- Full rebuild:
- make clean; make -j$(nproc) download world
## Directory behavior - **Version detection in download-config.sh** — it uses, in order: an explicit
- If you start from a freshly cloned helper checkout at `<parent>/<origin_folder>`, `prepare-openwrt.sh` will: `--version`/`--snapshot`; the exact tag at HEAD; the branch (`main`/`master` →
- Reuse or create `<parent>/openwrt/` as the OpenWrt source root. SNAPSHOT, `openwrt-XX.YY` → the newest matching tag merged into HEAD);
- Move the entire helper checkout, including its `.git`, to `<parent>/openwrt/helper/`. otherwise it asks. If HEAD is ahead of the tag it warns that the downloaded
- Initialize or update the OpenWrt source tree directly in `<parent>/openwrt/`. config describes the release rather than your tree. Version queries are
- Leave a symlink at the original checkout path pointing to `<parent>/openwrt/helper/`, so shells, editors and agent sessions whose working directory is still the old path keep working. Disable with `--no-compat-link`. anchored to the OpenWrt tree, not to whichever repository the script sits in.
- If you later rerun `prepare-openwrt.sh` — from `openwrt/helper/` or through the symlink at the old path — it reuses the parent `openwrt/` directory and does not move anything again. - **The device symbol did not survive make defconfig** — your tree does not match
- The already-relocated layout is detected by content (the OpenWrt tree or its `origin` remote), not by directory name, so renaming `openwrt/` to something else does not trigger a second relocation. the release you downloaded a config for. The script stops and points at the
- If a run is interrupted after the repository was created but before the source tree was checked out, the next run detects that state, reports `resuming it`, and completes. No manual cleanup is needed. `.config.download.bak` it kept.
- If the script is already located in a real OpenWrt source root, it treats that directory as the source root and does not relocate it just because the directory name is `openwrt`. - **A package is reported missing** — names are verified against
- Use `--root <dir>` to point at a different OpenWrt root in any ambiguous situation. `tmp/.config-package.in` and `tmp/.packageinfo`. Enable the relevant feed in
`feeds.conf.default`, then `./scripts/feeds update -a && ./scripts/feeds install -a`.
- **A removed package (`-pkg`) comes back after make defconfig** — something else
depends on it. Reported explicitly as "wanted =n, got =y".
- **A package is in .config but not in the manifest** — it is probably there under
a provider name (`libgcc` → `libgcc1`, `nftables` → `nftables-json`).
- **Rebuild one package** — `make package/<pkg>/{clean,compile} V=s`
- **Full rebuild** — `make clean; make -j$(nproc) download world`
### Known hazard: `git clean -xdff` ## Safety
`helper/` lives inside the OpenWrt Git worktree but is not tracked by it. The script
adds `/helper/` to `openwrt/.git/info/exclude`, which keeps `git status` clean and stops
`git add -A` from staging it — but that does **not** protect it from `git clean -xdff`:
`-x` deliberately includes ignored files and `-ff` removes Git's refusal to recurse into
a nested repository. Running `git clean -xdff` in the OpenWrt root will delete the helper
checkout including its `.git`. Use this instead:
git clean -xdf -e /helper - Scripts use `set -euo pipefail` and `umask 022`, and need root only to install
host packages.
## Security and safety - `.config` is backed up before it is replaced or before a version switch.
- Scripts use set -euo pipefail to stop on errors. - `prepare-openwrt.sh` refuses to switch versions on a tree with modified tracked
- prepare-openwrt.sh backs up `.config` to `.config.<previous-ref>.bak` before switching versions, and refuses to switch a tree with modified tracked files unless you pass --force. files unless you pass `--force`.
- They modify .config and write under package/ and feeds/ inside your OpenWrt tree. - Scripts write under `package/` and `feeds/` inside your OpenWrt tree.
- Backup your .config if you need a known baseline.
## License and authorship ## License and authorship
- Scripts authored by Zhe Yuan with help from ChatGPT.
- Scripts authored by Zhe Yuan.
- License: MIT (unless you choose a different license; update this line accordingly). - License: MIT (unless you choose a different license; update this line accordingly).