60 lines
1.6 KiB
Markdown
60 lines
1.6 KiB
Markdown
# GLaDOS TTS - ROCm Deployment (AMD GPU)
|
|
|
|
This guide covers running the Wyoming TTS server with AMD GPU acceleration via ROCm.
|
|
|
|
## Prerequisites
|
|
|
|
- Linux host with an AMD GPU supported by ROCm
|
|
- ROCm kernel drivers installed (`amdgpu` module loaded)
|
|
- Docker with `amdgpu` device support
|
|
|
|
Verify ROCm is available on the host:
|
|
|
|
```bash
|
|
ls /dev/kfd /dev/dri
|
|
rocm-smi # if ROCm tools are installed
|
|
```
|
|
|
|
## Running with docker-compose
|
|
|
|
```bash
|
|
docker compose up glados-tts-rocm
|
|
```
|
|
|
|
This uses the `Dockerfile.rocm` image with:
|
|
- PyTorch ROCm backend (device mapped to `cuda` internally)
|
|
- Half-precision (`--half`) enabled for ~2x speedup
|
|
- Model pre-loaded at startup (`--preload`)
|
|
- MIOpen convolution solver caching for faster repeated inference
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `DEVICE` | `rocm` | Set to `rocm` for AMD GPU |
|
|
| `HALF` | `true` | Use float16 inference |
|
|
| `PRELOAD` | `true` | Load model at startup |
|
|
| `MODEL_DIR` | `/data` | Model storage directory |
|
|
| `URI` | `tcp://0.0.0.0:10200` | Wyoming server URI |
|
|
|
|
ROCm-specific optimizations are applied automatically:
|
|
- `MIOPEN_FIND_MODE=1` - cache convolution solver solutions
|
|
- `MIOPEN_DEBUG_CONV_GEMM=0` - default GEMM mode
|
|
|
|
## Device Mapping
|
|
|
|
The server maps `rocm` to `cuda` internally since PyTorch ROCm uses the CUDA API compatibility layer. No code changes are needed.
|
|
|
|
## Troubleshooting
|
|
|
|
**No GPU detected in container logs**:
|
|
```bash
|
|
# Verify host has ROCm devices
|
|
ls -la /dev/kfd /dev/dri
|
|
|
|
# Check if amdgpu module is loaded
|
|
lsmod | grep amdgpu
|
|
```
|
|
|
|
**Out of memory**: Reduce `HALF=false` or ensure sufficient GPU VRAM (model requires ~4GB).
|