glados-ladosp-tts/README-GLaDOS-TYS-Wyoming-and-ROCM.md

GLaDOS TTS - ROCm Deployment (AMD GPU)

This guide covers running the Wyoming TTS server with AMD GPU acceleration via ROCm.

Prerequisites

• Linux host with an AMD GPU supported by ROCm
• ROCm kernel drivers installed (amdgpu module loaded)
• Docker with amdgpu device support

Verify ROCm is available on the host:

ls /dev/kfd /dev/dri
rocm-smi  # if ROCm tools are installed

Running with docker-compose

docker compose up glados-tts-rocm

This uses the Dockerfile.rocm image with:

• PyTorch ROCm backend (device mapped to cuda internally)
• Half-precision (--half) enabled for ~2x speedup
• Model pre-loaded at startup (--preload)
• MIOpen convolution solver caching for faster repeated inference

Environment Variables

Variable	Default	Description
DEVICE	rocm	Set to rocm for AMD GPU
HALF	true	Use float16 inference
PRELOAD	true	Load model at startup
MODEL_DIR	/data	Model storage directory
URI	tcp://0.0.0.0:10200	Wyoming server URI

ROCm-specific optimizations are applied automatically:

• MIOPEN_FIND_MODE=1 - cache convolution solver solutions
• MIOPEN_DEBUG_CONV_GEMM=0 - default GEMM mode

Device Mapping

The server maps rocm to cuda internally since PyTorch ROCm uses the CUDA API compatibility layer. No code changes are needed.

Troubleshooting

No GPU detected in container logs:

# Verify host has ROCm devices
ls -la /dev/kfd /dev/dri

# Check if amdgpu module is loaded
lsmod | grep amdgpu

Out of memory: Reduce HALF=false or ensure sufficient GPU VRAM (model requires ~4GB).