Release NVIDIA WakeWord Trainer v14

README.md

@@ -22,7 +22,7 @@ docker pull ghcr.io/tatertotterson/microwakeword:latest
 Tagged releases also publish matching immutable image tags:
 
 ```bash
-docker pull ghcr.io/tatertotterson/microwakeword:v13
+docker pull ghcr.io/tatertotterson/microwakeword:v14
 ```
 
 The release tag must match `VERSION`. Update `WHATS_NEW.md` before tagging; the Docker workflow prepends it to GitHub's automatically generated release notes.
@@ -32,7 +32,7 @@ Python 3.13 TensorFlow build for `sm_120`:
 
 ```bash
 docker pull ghcr.io/tatertotterson/microwakeword:blackwell
-docker pull ghcr.io/tatertotterson/microwakeword:v13-blackwell
+docker pull ghcr.io/tatertotterson/microwakeword:v14-blackwell
 ```
 
 Use the Blackwell image only for RTX 50-series cards. It includes the
@@ -53,9 +53,9 @@ docker run -d \
   ghcr.io/tatertotterson/microwakeword:latest
 ```
 
-Use a version tag such as `ghcr.io/tatertotterson/microwakeword:v13` when you want to pin a known release instead of tracking `latest`.
+Use a version tag such as `ghcr.io/tatertotterson/microwakeword:v14` when you want to pin a known release instead of tracking `latest`.
 For RTX 50-series cards, use `ghcr.io/tatertotterson/microwakeword:blackwell`
-or a pinned tag such as `ghcr.io/tatertotterson/microwakeword:v13-blackwell`
+or a pinned tag such as `ghcr.io/tatertotterson/microwakeword:v14-blackwell`
 in the same `docker run` command.
 
 The flags:
@@ -185,7 +185,7 @@ A close miss with an empty transcript or without the configured phrase stays in
 
 The default `small.en` model uses CUDA with `float16` when CTranslate2 can see an NVIDIA GPU, and falls back to CPU with `int8`. Choose a multilingual Faster Whisper model such as `small` when the wake phrase is not English. Downloaded STT models are cached in `/data/auto_train_models/`.
 
-Scheduled training runs only after the configured number of new automatic negatives has accumulated. A successful run publishes the replacement model at the same wake-word URL and can call Tater's native satellite settings API to make connected satellites pull it again. This refresh uses the existing Tater Native update path, so no satellite firmware change is required.
+Scheduled training runs only after the configured number of new automatic negatives has accumulated. A successful run publishes the replacement model at the same wake-word URL, asks Tater for its connected native satellites, and re-saves each satellite's current wake profile so its JSON tuning and model are fetched again. This refresh uses the existing Tater Native update path, so no satellite firmware change is required.
 
 The `Trainer public URL` must be reachable from the satellites. With the documented `--network host` command, the trainer can normally use the LAN address from the browser request or host network. If you open the UI as `http://localhost:8789`, enter a value such as `http://192.168.1.50:8789`, or start the container with `REC_PUBLIC_BASE_URL` set to that value. When using Docker bridge networking, always set this URL to the published host address; a container bridge address is not satellite-reachable.
 

VERSION

@@ -1 +1 @@
-13
+14

WHATS_NEW.md

@@ -1,6 +1,2 @@
-- Added opt-in Auto Training for false-positive wake triggers, using Faster Whisper with automatic CUDA/float16 selection and CPU/int8 fallback.
-- Wake triggers whose transcripts do not contain the configured phrase can now become hard negatives automatically; empty transcripts and phrase matches remain available for manual review unless optional cleanup is enabled.
-- Added optional confirmed-good-wake cleanup and conservative close-miss recovery that promotes only VAD-approved, STT-confirmed phrase matches to positive samples.
-- Added scheduled retraining with a minimum-new-negatives threshold and automatic Tater Native satellite refresh after a successful model build.
-- Wake-word download links now advertise a LAN-reachable trainer URL instead of `127.0.0.1`.
-- Tightened detector calibration defaults to favor fewer ambient false accepts while preserving candidates within 0.5 percentage points of the best recall.
+- Fixed automatic and manual satellite refresh so every connected satellite re-fetches its current custom wake JSON profile before reloading the model.
+- The large WHAM augmentation dataset download now reports visible progress in the training log.

cli/setup_wham

@@ -103,7 +103,8 @@ else
     if [ "${actual_filecount}" -eq 0 ] || [ "${actual_filecount}" -ne "${expected_filecount}" ] ; then
         if [ ! -f "${AUDIO_ZIP}" ] ; then
             echo "   Downloading ${AUDIO_ZIPFILE}"
-            curl -sfL "${AUDIO_URL}" -o "${AUDIO_ZIP}"
+            curl -fL --progress-bar "${AUDIO_URL}" -o "${AUDIO_ZIP}" \
+                2> >(tr '\r' '\n' >&2)
         fi
 
         rm -rf "${AUDIO_DIR}" || :

tests/test_auto_train.py

@@ -280,6 +280,63 @@ class AutoTrainTests(unittest.TestCase):
         self.assertEqual(request.get_header("X-tater-token"), "secret-token")
         self.assertEqual(json.loads(request.data), {"selector": "kitchen-sat", "settings": {}})
 
+    def test_tater_refresh_updates_each_connected_satellite_profile(self):
+        trainer.AUTO_TRAIN_CONFIG.update(
+            {
+                "notify_satellites": True,
+                "tater_url": "http://127.0.0.1:8501",
+                "tater_selector": "",
+            }
+        )
+
+        class Response:
+            def __init__(self, payload):
+                self.payload = payload
+
+            def __enter__(self):
+                return self
+
+            def __exit__(self, *_args):
+                return False
+
+            def read(self):
+                return json.dumps(self.payload).encode("utf-8")
+
+        responses = [
+            Response(
+                {
+                    "clients": {
+                        "native:office": {"selector": "native:office", "connected": True},
+                        "native:kitchen": {"connected": True},
+                        "native:garage": {"selector": "native:garage", "connected": False},
+                    }
+                }
+            ),
+            Response({"push": {"count": 1}}),
+            Response({"push": {"count": 1}}),
+        ]
+        with patch.object(trainer, "urlopen", side_effect=responses) as open_url:
+            result = trainer._notify_tater_satellites()
+
+        self.assertTrue(result["ok"])
+        self.assertEqual(result["count"], 2)
+        self.assertEqual(result["selectors"], ["native:office", "native:kitchen"])
+        self.assertEqual(open_url.call_count, 3)
+
+        status_request = open_url.call_args_list[0].args[0]
+        self.assertEqual(status_request.get_method(), "GET")
+        self.assertEqual(status_request.full_url, "http://127.0.0.1:8501/api/tater/satellite/v1/status")
+
+        refresh_requests = [call.args[0] for call in open_url.call_args_list[1:]]
+        self.assertEqual(
+            [json.loads(request.data) for request in refresh_requests],
+            [
+                {"selector": "native:office", "settings": {}},
+                {"selector": "native:kitchen", "settings": {}},
+            ],
+        )
+        self.assertTrue(all(request.get_method() == "POST" for request in refresh_requests))
+
     def test_advertised_url_uses_non_loopback_browser_host(self):
         request = SimpleNamespace(
             base_url="http://192.168.1.50:8789/",

trainer_server.py

@@ -932,36 +932,86 @@ def _auto_review_capture(file_name: str) -> None:
             AUTO_TRAIN_RUNTIME["review_file"] = ""
 
 
+def _connected_tater_satellite_selectors(payload: Any) -> List[str]:
+    if not isinstance(payload, dict):
+        return []
+
+    clients: Any = None
+    for key in ("clients", "satellites", "devices"):
+        if isinstance(payload.get(key), (dict, list)):
+            clients = payload.get(key)
+            break
+    if isinstance(clients, dict):
+        rows = [
+            (str(key or "").strip(), value)
+            for key, value in clients.items()
+        ]
+    elif isinstance(clients, list):
+        rows = [("", value) for value in clients]
+    else:
+        rows = []
+
+    selectors: List[str] = []
+    seen: set[str] = set()
+    for fallback_selector, row in rows:
+        if not isinstance(row, dict) or not _config_bool(row.get("connected"), False):
+            continue
+        selector = str(row.get("selector") or fallback_selector).strip()
+        if not selector or selector in seen:
+            continue
+        seen.add(selector)
+        selectors.append(selector)
+    return selectors
+
+
 def _notify_tater_satellites() -> Dict[str, Any]:
     with AUTO_TRAIN_LOCK:
         config = dict(AUTO_TRAIN_CONFIG)
     if not config.get("notify_satellites"):
         return {"ok": True, "skipped": True, "message": "Satellite notification is disabled."}
 
-    endpoint = f"{str(config.get('tater_url') or '').rstrip('/')}/api/tater/satellite/v1/settings"
-    body = json.dumps(
-        {
-            "selector": str(config.get("tater_selector") or ""),
-            "settings": {},
-        }
-    ).encode("utf-8")
+    base_url = str(config.get("tater_url") or "").rstrip("/")
+    settings_endpoint = f"{base_url}/api/tater/satellite/v1/settings"
+    status_endpoint = f"{base_url}/api/tater/satellite/v1/status"
     headers = {"Content-Type": "application/json", "User-Agent": "microWakeWord-Trainer/auto-train"}
     token = str(config.get("tater_api_token") or "").strip()
     if token:
         headers["X-Tater-Token"] = token
 
     try:
-        req = URLRequest(endpoint, data=body, headers=headers, method="POST")
-        with urlopen(req, timeout=15) as response:
-            payload = json.loads(response.read().decode("utf-8"))
-        push = payload.get("push") if isinstance(payload, dict) and isinstance(payload.get("push"), dict) else {}
-        count = push.get("count")
+        configured_selector = str(config.get("tater_selector") or "").strip()
+        if configured_selector:
+            selectors = [configured_selector]
+        else:
+            status_request = URLRequest(status_endpoint, headers=headers, method="GET")
+            with urlopen(status_request, timeout=15) as response:
+                status_payload = json.loads(response.read().decode("utf-8"))
+            selectors = _connected_tater_satellite_selectors(status_payload)
+
+        count = 0
+        refreshes: List[Dict[str, Any]] = []
+        for selector in selectors:
+            body = json.dumps({"selector": selector, "settings": {}}).encode("utf-8")
+            request = URLRequest(settings_endpoint, data=body, headers=headers, method="POST")
+            with urlopen(request, timeout=15) as response:
+                payload = json.loads(response.read().decode("utf-8"))
+            push = payload.get("push") if isinstance(payload, dict) and isinstance(payload.get("push"), dict) else {}
+            pushed_count = push.get("count")
+            if isinstance(pushed_count, (int, float)):
+                count += max(0, int(pushed_count))
+            refreshes.append({"selector": selector, "count": pushed_count})
+
         with AUTO_TRAIN_LOCK:
             AUTO_TRAIN_STATE["last_notify_at"] = _iso_now()
             AUTO_TRAIN_STATE["last_notify_count"] = count
             AUTO_TRAIN_STATE["last_notify_error"] = ""
             _save_auto_train_state_locked()
-        return {"ok": True, "count": count, "response": payload}
+        return {
+            "ok": True,
+            "count": count,
+            "selectors": selectors,
+            "refreshes": refreshes,
+        }
     except Exception as exc:
         with AUTO_TRAIN_LOCK:
             AUTO_TRAIN_STATE["last_notify_at"] = _iso_now()