Improve trainer capture metadata and USB flashing

Update prebuilt firmware flasher and tagged Docker releases

.github/workflows/docker-publish.yml

@@ -0,0 +1,58 @@
+name: Publish Docker Images
+
+on:
+  push:
+    tags:
+      - "v*"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  packages: write
+
+concurrency:
+  group: docker-publish-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: tatertotterson/microwakeword
+
+jobs:
+  docker:
+    name: Docker image
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Docker metadata
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=raw,value=latest
+            type=ref,event=tag
+
+      - name: Build and push image
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: dockerfile
+          platforms: linux/amd64
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha,scope=mww-trainer-nvidia-docker
+          cache-to: type=gha,mode=max,scope=mww-trainer-nvidia-docker

.gitignore

@@ -1,3 +1,4 @@
 personal_samples/*
 data/
+trim_history/
 .DS_Store

README.md

@@ -1,9 +1,13 @@
 <div align="center">
-  <h1>microWakeWord NVIDIA Docker Trainer UI</h1>
+  <a href="https://taterassistant.com">
-  <img width="800" alt="microWakeWord NVIDIA trainer screenshot" src="https://github.com/user-attachments/assets/694f4cb7-e4d8-4e2b-80ec-b40fb41cbfff" />
+    <img src="images/tater-repo-logo.png" alt="microWakeWord Trainer" width="460"/>
+  </a>
 </div>
+<h3 align="center">
+  <a href="https://taterassistant.com">taterassistant.com</a>
+</h3>
 
-Train custom microWakeWord models in Docker with NVIDIA/CUDA acceleration, generated Piper samples, device-captured samples, reviewed false-wake negatives, live training logs, and ESPHome firmware flashing.
+Train custom microWakeWord models in Docker with NVIDIA/CUDA acceleration, generated Piper samples, device-captured samples, reviewed false-wake negatives, live training logs, and prebuilt Tater firmware flashing.
 
 Real samples come from device-captured wake audio, close misses, or manual uploads. Every saved sample is normalized to `16 kHz / mono / 16-bit PCM WAV` before training.
 
@@ -15,6 +19,12 @@ Real samples come from device-captured wake audio, close misses, or manual uploa
 docker pull ghcr.io/tatertotterson/microwakeword:latest
 ```
 
+Tagged releases also publish matching immutable image tags:
+
+```bash
+docker pull ghcr.io/tatertotterson/microwakeword:v5
+```
+
 ---
 
 ## Run The Container
@@ -28,6 +38,8 @@ docker run -d \
   ghcr.io/tatertotterson/microwakeword:latest
 ```
 
+Use a version tag such as `ghcr.io/tatertotterson/microwakeword:v5` when you want to pin a known release instead of tracking `latest`.
+
 The flags:
 
 - `--gpus all` enables GPU acceleration.
@@ -52,14 +64,14 @@ If you change `REC_PORT`, open that port instead and use the same port in the ES
 - `Trainer` starts a wake-word session, shows positive/negative sample counts, and launches training.
 - `Captured Audio` reviews clips sent by ESPHome sats, including wake hits, close misses, and false wakes.
 - `Samples` plays, removes, clears, and manually imports personal or negative samples.
-- `Firmware` builds the latest `microWakeWords` ESPHome YAMLs from GitHub and flashes VoicePE or Satellite1 over OTA.
+- `Firmware` pulls verified prebuilt Tater firmware images from GitHub and flashes supported satellites over OTA.
 - Popup consoles show colorized training and firmware logs while long-running jobs are active.
 
 ---
 
 ## Captured Audio Workflow
 
-To collect samples from a sat, flash it with the Tater firmware from [TaterTotterson/microWakeWords](https://github.com/TaterTotterson/microWakeWords). The `Firmware` tab can build and flash the VoicePE or Satellite1 YAMLs directly from that repo.
+To collect samples from a sat, flash it with the Tater firmware from [TaterTotterson/microWakeWords](https://github.com/TaterTotterson/microWakeWords). The `Firmware` tab can pull verified prebuilt OTA images from that repo for fast firmware updates.
 
 After flashing, the device exposes ESPHome entities for capture setup:
 
@@ -173,18 +185,18 @@ After those assets are prepared, later runs reuse the local copies unless the mo
 
 ## Firmware Flashing
 
-The `Firmware` tab builds and flashes Tater firmware for supported ESPHome sats.
+The `Firmware` tab flashes prebuilt Tater firmware for supported ESPHome satellites.
 
-- Downloads the latest firmware YAML templates from `TaterTotterson/microWakeWords` on GitHub.
+- Downloads the latest prebuilt firmware manifest plus OTA and USB factory images from `TaterTotterson/microWakeWords`.
-- Lets you choose `VoicePE` or `Satellite1`.
+- Verifies downloaded images by size and SHA before upload.
 - Auto-detects ESPHome devices with mDNS when the container is running with host networking.
 - Allows manual IP or hostname entry if discovery does not find the device.
-- Saves firmware form values so you do not re-enter sounds and URLs every run.
+- Saves the selected OTA target for each firmware family.
-- Lists locally trained wake words from `/data/trained_wake_words/` for easy model selection.
+- Flashes the prebuilt factory image over Browser USB for first installs or recovery when opened in Chrome or Edge.
-- Builds with ESPHome and flashes OTA.
+- Lists locally trained wake words from `/data/trained_wake_words/` for live model switching.
-- Streams ESPHome output in a colorized firmware console.
+- Streams download, verification, and OTA upload progress in a colorized firmware console.
 
-Firmware YAMLs are intentionally pulled from GitHub each time. There is no local fallback path in the trainer UI.
+You usually only flash for firmware updates. New satellites, or devices older than Tater firmware `3.0.3`, need one USB flash first before OTA updates and live wake-word switching are available.
 
 ---
 
@@ -221,7 +233,7 @@ That removes:
 - cached datasets
 - training environments
 - trained models
-- firmware build caches
+- downloaded firmware images
 
 ---
 

cli/setup_mit_audio

@@ -25,9 +25,9 @@ fi
 mkdir -p "${DATA_DIR}/training_datasets/downloads" || :
 cd "${DATA_DIR}/training_datasets"
 
-AUDIO_URL="https://mcdermottlab.mit.edu/Reverb/IRMAudio/Audio.zip"
+HF_RIR_REPO_ID="TaterTotterson/MIT_environmental_impulse_responses"
-AUDIO_ZIPFILE="MIT_RIR_Audio.zip"
+HF_RIR_API_URL="https://huggingface.co/api/datasets/${HF_RIR_REPO_ID}"
-AUDIO_ZIP="./downloads/${AUDIO_ZIPFILE}"
+HF_RIR_SOURCE_KEY="hf_mit_environmental_impulse_responses"
 AUDIO_DIR="./mit_rirs"
 mkdir -p "${AUDIO_DIR}" || :
 AUDIO16K_DIR="./mit_rirs_16k"
@@ -35,10 +35,92 @@ mkdir -p "${AUDIO16K_DIR}" || :
 AUDIO_FILECOUNT="./downloads/mit_rir_filecount"
 AUDIO_IN_GLOB="*.wav"
 
-declare -A filecounts=( [${AUDIO_ZIPFILE}]=0 )
+declare -A filecounts=( [${HF_RIR_SOURCE_KEY}]=0 )
 get_filecounts filecounts "${AUDIO_FILECOUNT}"
 
-echo "===== Checking MIT_RIR ====="
+echo "===== Checking MIT environmental RIRs ====="
+
+download_hf_mit_rirs() {
+    source ${DATA_DIR}/.venv/bin/activate
+    python - "${HF_RIR_REPO_ID}" "${HF_RIR_API_URL}" "${AUDIO_DIR}" <<-'EOF'
+import json
+import sys
+import time
+import urllib.parse
+import urllib.request
+from pathlib import Path
+
+repo_id = sys.argv[1]
+api_url = sys.argv[2]
+audio_dir = Path(sys.argv[3])
+audio_dir.mkdir(parents=True, exist_ok=True)
+
+request = urllib.request.Request(api_url, headers={"User-Agent": "WakeWordTrainer/1.0"})
+with urllib.request.urlopen(request, timeout=30) as response:
+    metadata = json.loads(response.read().decode("utf-8"))
+
+files = sorted(
+    sibling.get("rfilename", "")
+    for sibling in metadata.get("siblings", [])
+    if str(sibling.get("rfilename", "")).startswith("16khz/")
+    and str(sibling.get("rfilename", "")).lower().endswith(".wav")
+)
+if not files:
+    raise SystemExit("Hugging Face MIT RIR dataset did not list any 16khz WAV files")
+
+print(f"   Found {len(files)} MIT environmental RIR files on Hugging Face mirror", flush=True)
+downloaded = 0
+skipped = 0
+
+def download_file(url: str, target: Path, rel: str):
+    tmp = target.with_suffix(target.suffix + ".incomplete")
+    for attempt in range(1, 4):
+        try:
+            if tmp.exists():
+                tmp.unlink()
+            with urllib.request.urlopen(url, timeout=30) as response:
+                with tmp.open("wb") as out:
+                    while True:
+                        chunk = response.read(1024 * 64)
+                        if not chunk:
+                            break
+                        out.write(chunk)
+            if not tmp.exists() or tmp.stat().st_size == 0:
+                raise RuntimeError("empty download")
+            tmp.replace(target)
+            return
+        except Exception as exc:
+            if tmp.exists():
+                tmp.unlink()
+            if attempt == 3:
+                raise RuntimeError(f"download failed for {rel}: {exc}") from exc
+            print(f"   Retry {attempt}/2 for {rel}: {exc}", flush=True)
+            time.sleep(2 * attempt)
+
+total = len(files)
+for idx, rel in enumerate(files, start=1):
+    target = audio_dir / rel
+    if target.exists() and target.stat().st_size > 0:
+        skipped += 1
+        if idx == 1 or idx % 25 == 0 or idx == total:
+            print(f"   MIT RIR download progress: {idx}/{total} files ({downloaded} downloaded, {skipped} reused)", flush=True)
+        continue
+    target.parent.mkdir(parents=True, exist_ok=True)
+    encoded = urllib.parse.quote(rel, safe="/")
+    url = f"https://huggingface.co/datasets/{repo_id}/resolve/main/{encoded}"
+    if idx == 1 or idx % 25 == 0 or idx == total:
+        print(f"   Downloading MIT RIR {idx}/{total}: {rel}", flush=True)
+    download_file(url, target, rel)
+    if not target.exists() or target.stat().st_size == 0:
+        raise SystemExit(f"download failed for {rel}")
+    downloaded += 1
+    if idx == 1 or idx % 25 == 0 or idx == total:
+        print(f"   MIT RIR download progress: {idx}/{total} files ({downloaded} downloaded, {skipped} reused)", flush=True)
+
+print(f"   Hugging Face MIT environmental RIR download complete ({downloaded} downloaded, {skipped} reused)", flush=True)
+print(f"   MIT environmental RIR files available: {len(files)}", flush=True)
+EOF
+}
 
 converter() {
     source ${DATA_DIR}/.venv/bin/activate
@@ -58,9 +140,9 @@ rir_out = Path(sys.argv[2])
 
 waves = list(rir_in.rglob("*.wav"))
 try:
-    print("   MIT RIR normalizing to 16k…")
+    print("   MIT environmental RIR normalizing to 16k…")
     # Normalize to 16k mono
-    for p in tqdm(waves, desc="   MIT_RIR (resample 16k mono)"):
+    for p in tqdm(waves, desc="   MIT environmental RIR (resample 16k mono)"):
         outfile = Path(rir_out / p.name)
         if outfile.exists():
             continue
@@ -70,14 +152,14 @@ try:
         if sr != 16000:
             a, _ = librosa.load(p, sr=16000, mono=True)
         write_wav(outfile, a, 16000)
-    print("   MIT RIR normalization complete")
+    print("   MIT environmental RIR normalization complete")
 except Exception as e2:
-    print(f"   MIT RIR fallback failed: {e2}")
+    print(f"   MIT environmental RIR preparation failed: {e2}")
     raise
 EOF
 }
 
-expected_filecount=${filecounts[${AUDIO_ZIPFILE}]}
+expected_filecount=${filecounts[${HF_RIR_SOURCE_KEY}]}
 actual_filecount=$(find "${AUDIO16K_DIR}" -name '*.wav' 2>/dev/null | wc -l) || :
 write_filecount=false
 
@@ -85,24 +167,16 @@ if [ "${actual_filecount}" -ne 0 ] && [ "${actual_filecount}" -eq "${expected_fi
     echo "   Existing ${AUDIO16K_DIR} valid"
 else
     actual_filecount=$(find "${AUDIO_DIR}" -name "${AUDIO_IN_GLOB}" 2>/dev/null | wc -l) || :
-    if [ "${actual_filecount}" -eq 0 ] || [ "${actual_filecount}" -ne "${expected_filecount}" ] ; then
+    if [ "${actual_filecount}" -eq 0 ] || [ "${expected_filecount}" -eq 0 ] || [ "${actual_filecount}" -ne "${expected_filecount}" ] ; then
-        if [ ! -f "${AUDIO_ZIP}" ]  ; then
-            echo "   Downloading ${AUDIO_ZIPFILE}"
-            curl -sfL "${AUDIO_URL}" -o "${AUDIO_ZIP}"
-        fi
-
         rm -rf "${AUDIO_DIR}" || :
-        echo "   Unzipping ${AUDIO_ZIPFILE}"
+        mkdir -p "${AUDIO_DIR}" || :
-        unzip -u -q -d "${AUDIO_DIR}" "${AUDIO_ZIP}"
+        echo "   Downloading MIT environmental impulse responses from Hugging Face mirror"
-    fi
+        download_hf_mit_rirs
-    if "${CLEANUP_ARCHIVES}" && [ -f "${AUDIO_ZIP}" ] ; then
-        echo "   Cleaning up ${AUDIO_ZIPFILE}"
-        rm -rf "${AUDIO_ZIP}"
     fi
 
     converter
     actual_filecount=$(find "${AUDIO16K_DIR}" -name "*.wav" 2>/dev/null | wc -l) || :
-    filecounts[${AUDIO_ZIPFILE}]="${actual_filecount}"
+    filecounts[${HF_RIR_SOURCE_KEY}]="${actual_filecount}"
     write_filecount=true
 fi
 
@@ -110,15 +184,10 @@ if ${write_filecount} ; then
     write_filecounts filecounts "${AUDIO_FILECOUNT}"
 fi
 
-if "${CLEANUP_ARCHIVES}" && [ -f "${AUDIO_ZIP}" ] ; then
-    echo "   Cleaning up ${AUDIO_ZIPFILE}"
-    rm -rf "${AUDIO_ZIP}"
-fi
-
 if "${CLEANUP_INTERMEDIATE_FILES}" && [ -d "${AUDIO_DIR}" ]; then
     echo "   Cleaning up ${AUDIO_DIR}"
     rm -rf "${AUDIO_DIR}"
 fi
 
-echo "   MIT_RIR complete"
+echo "   MIT environmental RIRs complete"
 exit 0

dockerfile

@@ -6,7 +6,7 @@ ENV DEBIAN_FRONTEND=noninteractive
 # System deps
 RUN apt-get update && apt-get install -y --no-install-recommends \
     python3.12 python3.12-venv python3.12-dev python3-pip python-is-python3 \
-    git wget curl unzip patch ca-certificates nano less \
+    git wget curl unzip patch ninja-build ca-certificates nano less \
  && rm -rf /var/lib/apt/lists/* \
  && mkdir -p /data
 

run.sh

@@ -17,7 +17,6 @@ PIN_FILE="${VENV_DIR}/.pinned_installed"
 FASTAPI_VERSION="${REC_FASTAPI_VERSION:-0.115.6}"
 UVICORN_VERSION="${REC_UVICORN_VERSION:-0.30.6}"
 PY_MULTIPART_VERSION="${REC_PY_MULTIPART_VERSION:-0.0.9}"
-ESPHOME_VERSION="${REC_ESPHOME_VERSION:-2026.4.0}"
 
 echo "microWakeWord Trainer UI (Docker)"
 echo "-> ROOTDIR:  ${ROOTDIR}"
@@ -26,6 +25,16 @@ echo "-> URL:      http://localhost:${PORT}/"
 
 mkdir -p "${DATA_DIR}"
 
+install_ui_deps() {
+  ${PIP} install \
+    "fastapi==${FASTAPI_VERSION}" \
+    "uvicorn[standard]==${UVICORN_VERSION}" \
+    "python-multipart==${PY_MULTIPART_VERSION}" \
+    "zeroconf>=0.132.2" \
+    "silero-vad>=5.0.0" \
+    "numpy>=1.24.0"
+}
+
 # -----------------------------
 # Trainer UI venv (separate)
 # -----------------------------
@@ -40,32 +49,54 @@ source "${VENV_DIR}/bin/activate"
 if [[ ! -f "${PIN_FILE}" ]]; then
   echo "Installing pinned trainer UI deps"
   ${PIP} install -U pip setuptools wheel
-  ${PIP} install \
+  install_ui_deps
-    "fastapi==${FASTAPI_VERSION}" \
-    "uvicorn[standard]==${UVICORN_VERSION}" \
-    "python-multipart==${PY_MULTIPART_VERSION}" \
-    "esphome==${ESPHOME_VERSION}"
   touch "${PIN_FILE}"
 else
   echo "Reusing existing trainer UI venv (no upgrades)"
-  if ! "${PY}" - "${ESPHOME_VERSION}" <<'PY' >/dev/null 2>&1
+  if ! "${PY}" - "${FASTAPI_VERSION}" "${UVICORN_VERSION}" "${PY_MULTIPART_VERSION}" <<'PY' >/dev/null 2>&1
-import importlib.metadata
+import importlib.metadata as md
 import sys
 
-expected = sys.argv[1]
+fastapi_version, uvicorn_version, multipart_version = sys.argv[1:4]
-installed = importlib.metadata.version("esphome")
+
-raise SystemExit(0 if installed == expected else 1)
+def version_tuple(value):
+    parts = []
+    for token in str(value).replace("-", ".").split("."):
+        if token.isdigit():
+            parts.append(int(token))
+        else:
+            digits = "".join(ch for ch in token if ch.isdigit())
+            if digits:
+                parts.append(int(digits))
+            break
+    return tuple(parts)
+
+exact = {
+    "fastapi": fastapi_version,
+    "uvicorn": uvicorn_version,
+    "python-multipart": multipart_version,
+}
+minimum = {
+    "silero-vad": "5.0.0",
+    "numpy": "1.24.0",
+    "zeroconf": "0.132.2",
+}
+present = ("torch",)
+
+for package, expected in exact.items():
+    if md.version(package) != expected:
+        raise SystemExit(1)
+for package, minimum_version in minimum.items():
+    if version_tuple(md.version(package)) < version_tuple(minimum_version):
+        raise SystemExit(1)
+for package in present:
+    md.version(package)
 PY
   then
-    echo "Firmware tab dependencies missing or stale; installing ESPHome firmware dependencies"
+    echo "UI dependencies missing or stale; installing recorder dependencies"
-    ${PIP} install \
+    install_ui_deps
-      "fastapi==${FASTAPI_VERSION}" \
-      "uvicorn[standard]==${UVICORN_VERSION}" \
-      "python-multipart==${PY_MULTIPART_VERSION}" \
-      "esphome==${ESPHOME_VERSION}"
   fi
 fi
-
 # -----------------------------
 # Trainer server env
 # -----------------------------