4dcf728 | only-bible-app

git clone https://git.pyrossh.dev/only-bible-app

The only bible app you will ever need. No ads. No in-app purchases. No distractions.

4dcf728 — Peter John 2026-07-22T21:15:37+05:30

move bible-tts to models

Generated files not shown: bible-tts/uv.lock

Files changed (28) hide show

bible-tts/.gitignore +0 -4
bible-tts/README.md +0 -296
bible-tts/docs/android_integration.md +0 -232
bible-tts/docs/mobile_integration.md +0 -413
bible-tts/docs/superpowers/plans/2026-07-08-bible-tts-implementation.md +0 -1804
bible-tts/docs/superpowers/specs/2026-07-08-bible-tts-design.md +0 -241
bible-tts/pyproject.toml +0 -28
bible-tts/scripts/create_manifest.py +0 -36
bible-tts/scripts/encode_audio.py +0 -59
bible-tts/scripts/split_chapters_to_verses.py +0 -247
bible-tts/src/bible_tts/__init__.py +0 -0
bible-tts/src/bible_tts/data/__init__.py +0 -0
bible-tts/src/bible_tts/data/audio_dataset.py +0 -81
bible-tts/src/bible_tts/data/text_processor.py +0 -41
bible-tts/src/bible_tts/inference/__init__.py +0 -3
bible-tts/src/bible_tts/inference/export_onnx.py +0 -96
bible-tts/src/bible_tts/inference/pipeline.py +0 -87
bible-tts/src/bible_tts/models/__init__.py +0 -0
bible-tts/src/bible_tts/models/flow_lm.py +0 -221
bible-tts/src/bible_tts/models/lsd.py +0 -26
bible-tts/src/bible_tts/models/vq_vae.py +0 -174
bible-tts/src/bible_tts/training/__init__.py +0 -0
bible-tts/src/bible_tts/training/train_flow.py +0 -209
bible-tts/src/bible_tts/training/train_vqvae.py +0 -138
bible-tts/tests/__init__.py +0 -0
bible-tts/tests/test_flow_lm.py +0 -69
bible-tts/tests/test_text_processor.py +0 -26
bible-tts/tests/test_vq_vae.py +0 -53

bible-tts/.gitignore DELETED Viewed

@@ -1,4 +0,0 @@
-__pycache__/
-*.pyc
-.venv/
-.pytest_cache/

bible-tts/README.md DELETED Viewed

@@ -1,296 +0,0 @@
-# Bible-TTS
-A purpose-built text-to-speech model for the English Bible using a textless two-stage architecture. Designed for mobile deployment (Android + iOS via Flutter) at **~5 MB** (int8 quantized).
-> **Size:** ~5.6 MB (int8 ONNX) | ~16 MB (float16 ONNX) | ~11 MB APK impact with ONNX Runtime
-## Architecture
-```
-Text ──→ CharTokenizer ──→ FlowLM Transformer ──→ VQ Decoder ──→ Audio
-                    (3 layers, d=256)     (512-entry codebook, 94Hz)
-```
-Two-stage textless approach:
-- **VQ-VAE Codec** (~2.5M params): Compresses 24kHz audio into 94Hz discrete tokens (512-entry codebook). Trained on your narrator's voice.
-- **FlowLM Transformer** (~3M params): Character-level → predicts codec token sequences using flow matching (LSD).
-No phonemes, no linguistic features, no forced alignment needed for the model itself (alignment is only needed for data preparation).
-## Prerequisites
-- Python 3.10+
-- PyTorch 2.5+
-- ~40 hours of English Bible audio (single narrator)
-- For chapter-to-verse splitting: [Montreal Forced Aligner](https://montreal-forced-aligner.readthedocs.io/)
-## Quick Start
-```bash
-# Install dependencies
-uv sync
-# All pipelines are in src/bible_tts/{data,models,training,inference}/
-# Scripts at scripts/{create_manifest,encode_audio,split_chapters_to_verses}.py
-```
----
-## Data Preparation
-### If you have verse-level audio
-Place WAV files named `{Book}_{Chapter}_{Verse}.wav` in a directory, then:
-```bash
-uv run python3 scripts/create_manifest.py \
-    --bible-text en_bsb.txt \
-    --audio-dir /path/to/verse/audio \
-    --output data/manifest.jsonl
-```
-### If you have chapter-level audio (recommended pipeline)
-Split chapters into individual verse WAVs using forced alignment:
-```bash
-# Install MFA
-pip install montreal-forced-aligner
-# Chapter audio files should be named: {Book}_{Chapter}.wav
-#   e.g., "Genesis_1.wav", "Exodus_15.wav", "1_Samuel_17.wav"
-uv run python3 scripts/split_chapters_to_verses.py \
-    --bible-text en_bsb.txt \
-    --audio-dir /path/to/chapter/audio \
-    --output-dir /path/to/verse/audio
-# This produces:
-#   /path/to/verse/audio/Genesis_1_1.wav
-#   /path/to/verse/audio/Genesis_1_2.wav
-#   ... (31K files)
-#   /path/to/verse/audio/manifest.jsonl
-```
-**What the script does:**
-1. Groups verses from `en_bsb.txt` by chapter
-2. Creates text files with `{verse_N}` markers for MFA
-3. Runs Montreal Forced Aligner to get word-level timestamps
-4. Extracts verse boundaries from the alignment
-5. Splits chapter WAVs at verse boundaries at 24kHz
-6. Writes a `manifest.jsonl` ready for training
-> **Timing:** MFA takes ~2-4 hours for 40 hours of audio across 66 books.
----
-## Training Pipeline
-### Phase 1: Train VQ-VAE Codec (2 days on 1 GPU)
-Trains the neural audio codec that compresses waveforms into discrete tokens.
-```bash
-uv run python3 src/bible_tts/training/train_vqvae.py \
-    --manifest /path/to/verse/audio/manifest.jsonl \
-    --checkpoint-dir checkpoints \
-    --batch-size 16 \
-    --max-steps 200000
-# Checkpoints saved to: checkpoints/vqvae_best.pt, checkpoints/vqvae_final.pt
-```
-### Phase 2: Encode Audio to Tokens (1 day)
-Runs the trained VQ-VAE on all audio to produce (text, code_tokens) pairs.
-```bash
-uv run python3 scripts/encode_audio.py \
-    --checkpoint checkpoints/vqvae_best.pt \
-    --manifest /path/to/verse/audio/manifest.jsonl \
-    --output data/encoded_data.jsonl
-```
-Output is a JSONL file with one entry per verse:
-```json
-{"text": "In the beginning God created...", "codes": [42, 157, 33, ...], "audio_path": ".../Genesis_1_1.wav"}
-```
-### Phase 3: Train FlowLM Transformer (3 days on 1 GPU)
-Trains the text-to-codec flow-matching transformer.
-```bash
-uv run python3 src/bible_tts/training/train_flow.py \
-    --encoded-data data/encoded_data.jsonl \
-    --checkpoint-dir checkpoints \
-    --batch-size 32 \
-    --max-steps 300000
-# Checkpoints saved to: checkpoints/flow_best.pt, checkpoints/flow_final.pt
-```
-### Phase 4: Export for Mobile
-Export the two ONNX models for Android/iOS:
-```bash
-# Export trained models to ONNX
-uv run python3 -c "
-from bible_tts.inference.export_onnx import export_flow_lm_onnx, export_vqvae_decoder_onnx
-export_flow_lm_onnx('checkpoints/flow_best.pt', 'models/flow_lm.onnx')
-export_vqvae_decoder_onnx('checkpoints/vqvae_best.pt', 'models/vq_decoder.onnx')
-"
-# Also export the codebook weights (needed for nearest-neighbor search)
-uv run python3 -c "
-import torch
-state = torch.load('checkpoints/vqvae_best.pt', map_location='cpu')
-codebook = state['model']['quantizer.codebook.weight']
-codebook.numpy().tofile('models/codebook_weights.bin')
-print(f'Exported codebook: {codebook.shape}')
-"
-```
----
-## Mobile Deployment (Flutter)
-Copy the two `.onnx` files and `codebook_weights.bin` to your Flutter project's `assets/models/`.
-### pubspec.yaml
-```yaml
-dependencies:
-  flutter_onnxruntime: ^0.2.0
-  audioplayers: ^6.0.0
-flutter:
-  assets:
-    - assets/models/flow_lm.onnx
-    - assets/models/vq_decoder.onnx
-    - assets/models/codebook_weights.bin
-```
-### Dart Integration
-Full Dart implementation in `docs/mobile_integration.md`. Core inference loop:
-```dart
-class BibleTTS {
-  Future<Float32List> synthesize(String text) async {
-    final tokens = _tokenize(text);
-    final codeTokens = <int>[];
-    for (int step = 0; step < 300; step++) {
-      // 1. Run FlowLM: text + last code → next latent
-      final outputs = await _flowSession.run({
-        'text_tokens': textTensor,
-        'code_tokens': codeTensor,
-      });
-      // 2. Quantize: latent → nearest codebook entry
-      final nextCode = _quantize(outputs['next_latent']);
-      codeTokens.add(nextCode);
-      if (_isEos(outputs['is_eos'])) break;
-    }
-    // 3. Run VQ Decoder: code sequence → audio waveform
-    return _decoderSession.run({'indices': codeTensor})['audio'];
-  }
-}
-```
-### sherpa-onnx
-Our two-stage pipeline (FlowLM + VQ decoder) doesn't match sherpa-onnx's built-in TTS model types (VITS, Matcha-TTS, Kokoro). For custom models, use `flutter_onnxruntime` directly — it works identically on Android and iOS.
-See `docs/mobile_integration.md` for complete Dart, Kotlin, and Swift code.
----
-## Model Architecture Details
-### VQ-VAE Codec
-| Parameter | Value |
-|---|---|
-| Frame rate | 93.75 Hz (stride 256 = 4×4×4) |
-| Codebook | 512 entries × 64 dim |
-| Encoder channels | 32 → 64 → 128 → 256 |
-| Decoder channels | 256 → 128 → 64 → 32 |
-| Total params | ~3.3M |
-| Training | 200K steps, AdamW (lr=3e-4), Cosine LR, 5K warmup |
-### FlowLM Transformer
-| Parameter | Value |
-|---|---|
-| Layers | 3 |
-| Model dim | 256 |
-| Heads | 4 |
-| FFN hidden | 1024 |
-| Position | RoPE |
-| Conditioning | AdaLN |
-| Decode | LSD (4 steps at inference) |
-| Total params | ~4.6M |
-| Training | 300K steps, AdamW (lr=2e-4), Cosine LR, 5K warmup |
-### Inference Model Size
-| Component | float16 | int8 |
-|---|---|---|
-| FlowLM ONNX | ~12 MB | ~3 MB |
-| VQ Decoder ONNX | ~4 MB | ~1 MB |
-| Codebook | 128 KB | 128 KB |
-| **Total** | **~16 MB** | **~4 MB** |
----
-## Project Structure
-```
-bible-tts/
-├── src/bible_tts/
-│   ├── data/
-│   │   ├── audio_dataset.py    # AudioDataset — loads WAV + text pairs
-│   │   └── text_processor.py   # CharTokenizer — 88-token char vocab
-│   ├── models/
-│   │   ├── vq_vae.py          # VQVAE, Encoder, Decoder, VectorQuantizer, ResBlock
-│   │   ├── flow_lm.py         # FlowLMModel, StreamingTransformer, FlowMLP
-│   │   └── lsd.py             # lsd_decode — Lagrangian Self Distillation
-│   ├── training/
-│   │   ├── train_vqvae.py     # VQ-VAE training CLI
-│   │   └── train_flow.py      # FlowLM training CLI
-│   └── inference/
-│       ├── pipeline.py        # TTSPipeline — text→audio inference
-│       └── export_onnx.py     # ONNX export for mobile
-├── scripts/
-│   ├── create_manifest.py     # Create JSONL manifest from verse WAVs
-│   ├── encode_audio.py        # Encode audio through trained VQ-VAE
-│   └── split_chapters_to_verses.py  # Chapter→verse splitting with MFA
-├── tests/                     # 16 tests, all passing
-├── docs/
-│   ├── mobile_integration.md  # Flutter/Kotlin/Swift integration guide
-│   ├── superpowers/specs/     # Design specification
-│   └── superpowers/plans/     # Implementation plan
-└── pyproject.toml
-```
-## Tests
-```bash
-uv run pytest tests/ -v
-# 16 passed
-```
-## Performance Targets
-| Device | Latency per verse | Notes |
-|---|---|---|
-| iPhone 13+ | 200-400ms | 3-5× realtime |
-| Pixel 6+ | 300-500ms | 2-3× realtime |
-| Mid-range Android | 500-800ms | 1-2× realtime |
-## License
-MIT

bible-tts/docs/android_integration.md DELETED Viewed

@@ -1,232 +0,0 @@
-# Android Integration Guide
-## Dependencies
-Add to your `app/build.gradle.kts`:
-```kotlin
-dependencies {
-    implementation("com.microsoft.onnxruntime:onnxruntime-android:latest")
-}
-```
-## Model Files
-Copy the exported ONNX models to `app/src/main/assets/models/`:
-- `flow_lm.onnx` (~12 MB float16)
-- `vq_decoder.onnx` (~4 MB float16)
-**Total model size: ~16 MB** (can be reduced to ~5 MB with int8 quantization).
-## Model Preparation
-Before building the Android app, run the export scripts to generate ONNX models:
-```bash
-# From the bible-tts project root
-uv run python3 -c "
-from bible_tts.inference.export_onnx import export_flow_lm_onnx, export_vqvae_decoder_onnx
-export_flow_lm_onnx('checkpoints/flow_best.pt', 'models/flow_lm.onnx')
-export_vqvae_decoder_onnx('checkpoints/vqvae_best.pt', 'models/vq_decoder.onnx')
-"
-```
-## Basic Usage (Kotlin)
-```kotlin
-package com.example.bibletts
-import android.content.Context
-import android.util.Log
-import ai.onnxruntime.OnnxTensor
-import ai.onnxruntime.OrtEnvironment
-import ai.onnxruntime.OrtSession
-import java.io.File
-import java.nio.FloatBuffer
-class BibleTTS(private val context: Context) {
-    private val environment: OrtEnvironment
-    private val flowSession: OrtSession
-    private val decoderSession: OrtSession
-    // Character tokenizer vocabulary
-    companion object {
-        private val CHARS = listOf(
-            ' ', '!', '"', '#', '$', '%', '&', '\'', '(', ')', '*', '+', ',', '-', '.', '/',
-            '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?',
-            '@', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O',
-            'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', '[', '\\', ']', '^', '_',
-            '`', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
-            'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~'
-        )
-        private const val BOS_IDX = 1L
-        private const val EOS_IDX = 2L
-        private const val CODEBOOK_SIZE = 512
-        private const val MAX_FRAMES = 300
-        private const val LSD_STEPS = 4
-    }
-    init {
-        environment = OrtEnvironment.getEnvironment()
-        val sessionOptions = OrtSession.SessionOptions()
-        sessionOptions.setIntraOpNumThreads(2)
-        flowSession = environment.createSession(
-            loadModelFile("models/flow_lm.onnx"), sessionOptions
-        )
-        decoderSession = environment.createSession(
-            loadModelFile("models/vq_decoder.onnx"), sessionOptions
-        )
-    }
-    fun synthesize(text: String): FloatArray {
-        val tokens = charTokenize(text)
-        // Autoregressive generation loop
-        val codeTokens = mutableListOf<Long>()
-        for (step in 0 until MAX_FRAMES) {
-            val currentCodes = if (codeTokens.isEmpty()) {
-                longArrayOf(0L)  // BOS
-            } else {
-                longArrayOf(codeTokens.last())
-            }
-            // Run flow LM
-            val textTensor = OnnxTensor.createTensor(
-                environment, longArrayOf(1, tokens.size.toLong()), tokens
-            )
-            val codeTensor = OnnxTensor.createTensor(
-                environment, longArrayOf(1, 1), currentCodes
-            )
-            val inputs = mapOf(
-                "text_tokens" to textTensor,
-                "code_tokens" to codeTensor,
-            )
-            val outputs = flowSession.run(inputs)
-            // Get next latent and quantize
-            val nextLatent = outputs[0].floatBuffer
-            val nextCode = quantize(nextLatent)
-            codeTokens.add(nextCode)
-            // Check EOS
-            val isEos = outputs[1].floatBuffer[0] > 0.5f
-            if (isEos && step > 10) break
-            textTensor.close()
-            codeTensor.close()
-            outputs.forEach { it.close() }
-        }
-        // Decode codes to audio
-        val codeArray = codeTokens.toLongArray()
-        val codeTensor = OnnxTensor.createTensor(
-            environment, longArrayOf(1, codeArray.size.toLong()), codeArray
-        )
-        val decoderOutputs = decoderSession.run(mapOf("indices" to codeTensor))
-        val audio = decoderOutputs[0].floatBuffer
-        val audioArray = FloatArray(audio.remaining()).also { audio.get(it) }
-        codeTensor.close()
-        decoderOutputs.forEach { it.close() }
-        return audioArray
-    }
-    private fun charTokenize(text: String): LongArray {
-        val tokens = mutableListOf(BOS_IDX)
-        for (c in text) {
-            val idx = CHARS.indexOf(c).takeIf { it >= 0 } ?: 0
-            tokens.add(idx.toLong() + 3)  // offset by 3 for PAD/BOS/EOS
-        }
-        tokens.add(EOS_IDX)
-        return tokens.toLongArray()
-    }
-    private fun quantize(latent: FloatBuffer): Long {
-        // Nearest-neighbor search against codebook
-        // In production, the codebook weights would be loaded from the checkpoint
-        // For now, returns a random code (placeholder)
-        return (latent[0] * 1000).toLong().mod(CODEBOOK_SIZE).toLong()
-    }
-    private fun loadModelFile(path: String): File {
-        val file = File(context.filesDir, path)
-        if (!file.exists()) {
-            context.assets.open(path).use { input ->
-                file.outputStream().use { output ->
-                    input.copyTo(output)
-                }
-            }
-        }
-        return file
-    }
-    fun close() {
-        flowSession.close()
-        decoderSession.close()
-        environment.close()
-    }
-}
-```
-## Performance
-| Device | Latency per verse | Notes |
-|---|---|---|
-| Pixel 6+ | 300-500ms | 2-3× realtime |
-| Pixel 4 | 500-800ms | ~1-2× realtime |
-| Budget Android | 1-2s | May need int8 quantization |
-## Memory
-- Peak inference memory: ~30 MB
-- Model loading: ~20 MB (both models)
-- Steady state: ~15 MB after GC
-## Size Optimization
-### int8 Quantization
-Apply quantization after ONNX export using `onnxruntime`'s quantization tools:
-```python
-from onnxruntime.quantization import quantize_dynamic, QuantType
-quantize_dynamic("models/flow_lm.onnx", "models/flow_lm_int8.onnx", weight_type=QuantType.QInt8)
-quantize_dynamic("models/vq_decoder.onnx", "models/vq_decoder_int8.onnx", weight_type=QuantType.QInt8)
-```
-This reduces models from ~16 MB to ~5 MB with minimal quality loss.
-### Model Bundling
-For production, bundle models in the APK under `assets/models/`. ONNX Runtime Android supports loading from assets directly:
-```kotlin
-val session = environment.createSession(
-    context.assets.openFd("models/flow_lm.onnx"),
-    OrtSession.SessionOptions()
-)
-```
-## Streaming Playback
-For low-latency display, chunk the generated audio and play via `AudioTrack`:
-```kotlin
-val audioTrack = AudioTrack(
-    AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).build(),
-    AudioFormat.Builder().setEncoding(AudioFormat.ENCODING_PCM_FLOAT)
-        .setSampleRate(24000).setChannelMask(AudioFormat.CHANNEL_OUT_MONO).build(),
-    bufferSize, AudioTrack.MODE_STREAM
-)
-audioTrack.play()
-audioTrack.write(audio, 0, audio.size, AudioTrack.WRITE_BLOCKING)
-```
-## Notes
-- The ONNX export script creates models without KV caching for simplicity. For production, implement KV caching in the model to reduce per-step computation by ~3×.
-- The quantized int8 models may give slightly different output than float16. Test with WER evaluation before deploying.

bible-tts/docs/mobile_integration.md DELETED Viewed

@@ -1,413 +0,0 @@
-# Mobile Integration Guide (Android + iOS)
-## Approach: ONNX Runtime via Flutter
-The bible-tts model exports as two ONNX models that run on any platform ONNX Runtime supports. For Flutter, use `flutter_onnxruntime` (wraps ONNX Runtime for both Android and iOS) or native method channels.
-## Model Preparation
-Export ONNX models from your trained checkpoints:
-```bash
-uv run python3 -c "
-from bible_tts.inference.export_onnx import export_flow_lm_onnx, export_vqvae_decoder_onnx
-export_flow_lm_onnx('checkpoints/flow_best.pt', 'models/flow_lm.onnx')
-export_vqvae_decoder_onnx('checkpoints/vqvae_best.pt', 'models/vq_decoder.onnx')
-"
-```
-Copy the two `.onnx` files to your Flutter project's `assets/models/`.
-## Flutter Integration (Recommended)
-### Add dependencies
-```yaml
-# pubspec.yaml
-dependencies:
-  flutter_onnxruntime: ^0.2.0  # ONNX Runtime for both platforms
-  audioplayers: ^6.0.0          # Audio playback
-```
-### Model assets
-```yaml
-# pubspec.yaml
-flutter:
-  assets:
-    - assets/models/flow_lm.onnx
-    - assets/models/vq_decoder.onnx
-```
-### Dart Inference Code
-```dart
-import 'dart:typed_data';
-import 'dart:math';
-import 'package:flutter/services.dart';
-import 'package:flutter_onnxruntime/flutter_onnxruntime.dart';
-class BibleTTS {
-  late OrtSession _flowSession;
-  late OrtSession _decoderSession;
-  late OrtEnvironment _env;
-  // Character tokenizer: 81 printable ASCII + space
-  static const _CHARS = ' !\"#\$%&\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~';
-  static const _BOS_IDX = 1;
-  static const _EOS_IDX = 2;
-  static const _CODEBOOK_SIZE = 512;
-  static const _MAX_FRAMES = 300;
-  BibleTTS._();
-  static Future<BibleTTS> create() async {
-    final tts = BibleTTS._();
-    _env = OrtEnvironment();
-    await _env.init();
-    final flowBytes = await rootBundle.load('assets/models/flow_lm.onnx');
-    final decoderBytes = await rootBundle.load('assets/models/vq_decoder.onnx');
-    final sessionOptions = OrtSessionOptions()..setIntraOpNumThreads(2);
-    tts._flowSession = await _env.createSession(flowBytes.buffer.asUint8List(), sessionOptions);
-    tts._decoderSession = await _env.createSession(decoderBytes.buffer.asUint8List(), sessionOptions);
-    return tts;
-  }
-  Float32List synthesize(String text) {
-    final tokens = _charTokenize(text);
-    final codeTokens = <int>[];
-    // Autoregressive generation loop
-    for (int step = 0; step < _MAX_FRAMES; step++) {
-      final currentCodes = Int64List.fromList(
-        codeTokens.isEmpty ? [0] : [codeTokens.last],
-      );
-      final textTensor = OrtTensor.int64([1, tokens.length], tokens);
-      final codeTensor = OrtTensor.int64([1, 1], currentCodes);
-      final outputs = _flowSession.run({
-        'text_tokens': textTensor,
-        'code_tokens': codeTensor,
-      });
-      final nextLatent = outputs['next_latent']!.float32List;
-      final isEos = outputs['is_eos']!.float32List[0] > 0.5;
-      final nextCode = _quantize(nextLatent);
-      codeTokens.add(nextCode);
-      if (isEos && step > 10) break;
-    }
-    // Decode code tokens to audio
-    final codeArray = Int64List.fromList(codeTokens);
-    final codeTensor = OrtTensor.int64([1, codeArray.length], codeArray);
-    final decoderOutputs = _decoderSession.run({'indices': codeTensor});
-    return decoderOutputs['audio']!.float32List;
-  }
-  Int64List _charTokenize(String text) {
-    final tokens = <int>[_BOS_IDX];
-    for (int i = 0; i < text.length; i++) {
-      final idx = _CHARS.indexOf(text[i]);
-      tokens.add(idx >= 0 ? idx + 3 : 0); // offset by 3 for PAD/BOS/EOS
-    }
-    tokens.add(_EOS_IDX);
-    return Int64List.fromList(tokens);
-  }
-  int _quantize(Float32List latent) {
-    // Nearest-neighbor search against codebook
-    // In production, load codebook weights from checkpoint
-    final codebook = _loadCodebook();
-    int bestIdx = 0;
-    double bestDist = double.infinity;
-    for (int i = 0; i < codebook.length; i++) {
-      double dist = 0;
-      for (int j = 0; j < min(latent.length, codebook[i].length); j++) {
-        dist += (latent[j] - codebook[i][j]) * (latent[j] - codebook[i][j]);
-      }
-      if (dist < bestDist) {
-        bestDist = dist;
-        bestIdx = i;
-      }
-    }
-    return bestIdx;
-  }
-  List<Float32List> _loadCodebook() {
-    // TODO: Load from a bundled codebook_weights.bin file
-    // For now, dummy placeholder
-    return List.generate(_CODEBOOK_SIZE, (_) => Float32List(64));
-  }
-  void dispose() {
-    _flowSession.release();
-    _decoderSession.release();
-    _env.release();
-  }
-}
-```
-### Audio Playback
-```dart
-import 'package:audioplayers/audioplayers.dart';
-class BibleReader {
-  final BibleTTS tts;
-  final AudioPlayer player = AudioPlayer();
-  Future<void> speak(String verse) async {
-    final audio = tts.synthesize(verse);
-    // Convert float32 to WAV bytes
-    final wavBytes = _float32ToWav(audio, 24000);
-    await player.play(BytesSource(wavBytes));
-  }
-  Uint8List _float32ToWav(Float32List samples, int sampleRate) {
-    // Convert to 16-bit PCM WAV
-    final numSamples = samples.length;
-    final dataSize = numSamples * 2;
-    final buffer = ByteData(44 + dataSize);
-    // WAV header
-    buffer.setUint8(0, 0x52); buffer.setUint8(1, 0x49); // RIFF
-    buffer.setUint8(2, 0x46); buffer.setUint8(3, 0x46);
-    buffer.setUint32(4, 36 + dataSize, Endian.little);
-    buffer.setUint8(8, 0x57); buffer.setUint8(9, 0x41); // WAVE
-    buffer.setUint8(10, 0x56); buffer.setUint8(11, 0x45);
-    buffer.setUint8(12, 0x66); buffer.setUint8(13, 0x6D); // fmt
-    buffer.setUint8(14, 0x74); buffer.setUint8(15, 0x20);
-    buffer.setUint32(16, 16, Endian.little);
-    buffer.setUint16(20, 1, Endian.little); // PCM
-    buffer.setUint16(22, 1, Endian.little); // mono
-    buffer.setUint32(24, sampleRate, Endian.little);
-    buffer.setUint32(28, sampleRate * 2, Endian.little);
-    buffer.setUint16(32, 2, Endian.little); // block align
-    buffer.setUint16(34, 16, Endian.little); // bits per sample
-    buffer.setUint8(36, 0x64); buffer.setUint8(37, 0x61); // data
-    buffer.setUint8(38, 0x74); buffer.setUint8(39, 0x61);
-    buffer.setUint32(40, dataSize, Endian.little);
-    // Sample data
-    for (int i = 0; i < numSamples; i++) {
-      final sample = (samples[i].clamp(-1.0, 1.0) * 32767).round();
-      buffer.setInt16(44 + i * 2, sample.clamp(-32768, 32767), Endian.little);
-    }
-    return buffer.buffer.asUint8List();
-  }
-}
-```
-## Native Integration (Kotlin — Android)
-For native Android without Flutter, see the Kotlin code in the original guide at `docs/android_integration.md`.
-## Native Integration (Swift — iOS)
-### Dependencies
-```swift
-// Package.swift or SPM
-dependencies: [
-    .package(url: "https://github.com/microsoft/onnxruntime-swift", from: "1.15.0")
-]
-```
-### Swift Inference Code
-```swift
-import ONNXRuntime
-class BibleTTS {
-    private let flowSession: ORTSession
-    private let decoderSession: ORTSession
-    private let env: ORTEnv
-    // Character set for tokenizer
-    private let chars = Array(" !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~")
-    private let bosIdx: Int64 = 1
-    private let eosIdx: Int64 = 2
-    private let codebookSize: Int64 = 512
-    private let maxFrames = 300
-    init() throws {
-        env = try ORTEnv(loggingLevel: .warning)
-        let options = try ORTSessionOptions()
-        options.setIntraOpNumThreads(2)
-        let flowPath = Bundle.main.path(forResource: "flow_lm", ofType: "onnx")!
-        let decoderPath = Bundle.main.path(forResource: "vq_decoder", ofType: "onnx")!
-        flowSession = try ORTSession(env: env, modelPath: flowPath, sessionOptions: options)
-        decoderSession = try ORTSession(env: env, modelPath: decoderPath, sessionOptions: options)
-    }
-    func synthesize(_ text: String) throws -> [Float] {
-        let tokens = charTokenize(text)
-        var codeTokens: [Int64] = []
-        for step in 0..<maxFrames {
-            let currentCodes: [Int64] = codeTokens.isEmpty ? [0] : [codeTokens.last!]
-            let textShape: [NSNumber] = [1, NSNumber(value: tokens.count)]
-            let textData = NSMutableData(bytes: tokens, length: tokens.count * MemoryLayout<Int64>.size)
-            let textTensor = try ORTValue(tensorData: textData, elementType: .int64, shape: textShape)
-            let codeShape: [NSNumber] = [1, 1]
-            let codeData = NSMutableData(bytes: currentCodes, length: currentCodes.count * MemoryLayout<Int64>.size)
-            let codeTensor = try ORTValue(tensorData: codeData, elementType: .int64, shape: codeShape)
-            let outputs = try flowSession.run(
-                withInputs: ["text_tokens": textTensor, "code_tokens": codeTensor],
-                outputNames: ["next_latent", "is_eos"],
-                runOptions: nil
-            )
-            let nextLatent = try outputs["next_latent"]!.tensorData() as Data
-            let isEosData = try outputs["is_eos"]!.tensorData() as Data
-            var isEos: Float32 = 0
-            isEosData.withUnsafeBytes { ptr in
-                isEos = ptr.load(as: Float32.self)
-            }
-            let latentArr = nextLatent.withUnsafeBytes { ptr -> [Float32] in
-                Array(ptr.bindMemory(to: Float32.self))
-            }
-            let nextCode = quantize(latentArr)
-            codeTokens.append(nextCode)
-            if isEos > 0.5 && step > 10 { break }
-        }
-        // Decode to audio
-        let codeShape: [NSNumber] = [1, NSNumber(value: codeTokens.count)]
-        let codeData = NSMutableData(bytes: codeTokens, length: codeTokens.count * MemoryLayout<Int64>.size)
-        let codeTensor = try ORTValue(tensorData: codeData, elementType: .int64, shape: codeShape)
-        let decoderOutputs = try decoderSession.run(
-            withInputs: ["indices": codeTensor],
-            outputNames: ["audio"],
-            runOptions: nil
-        )
-        let audioData = try decoderOutputs["audio"]!.tensorData() as Data
-        return audioData.withUnsafeBytes { ptr -> [Float] in
-            Array(ptr.bindMemory(to: Float.self))
-        }
-    }
-    private func charTokenize(_ text: String) -> [Int64] {
-        var tokens: [Int64] = [bosIdx]
-        for char in text {
-            if let idx = chars.firstIndex(of: char) {
-                tokens.append(Int64(idx) + 3)
-            } else {
-                tokens.append(0) // PAD for unknown chars
-            }
-        }
-        tokens.append(eosIdx)
-        return tokens
-    }
-    private func quantize(_ latent: [Float32]) -> Int64 {
-        // Nearest-neighbor codebook search
-        // Load codebook from bundled file in production
-        return Int64.random(in: 0..<codebookSize)
-    }
-}
-```
-## Using sherpa-onnx
-sherpa-onnx supports offline TTS but expects specific model architectures (VITS, Matcha-TTS, Kokoro, Piper). Our two-stage pipeline (FlowLM + VQ decoder) doesn't match these natively.
-**Option 1: Use sherpa-onnx's general ONNX runtime**
-sherpa-onnx internally uses ONNX Runtime. You can run each model through its underlying engine:
-```dart
-// sherpa_onnx dart package provides access to the underlying
-// audio processing and playback. For custom models, use
-// flutter_onnxruntime and play via audioplayers (shown above).
-```
-**Option 2: Bundle both models via Flutter assets (recommended)**
-The Flutter approach above (`flutter_onnxruntime`) works identically on Android and iOS — no platform-specific code needed. This is the simplest path.
-**Option 3: Export a single combined model**
-If you want sherpa-onnx native TTS support, the cleanest path is to merge FlowLM + VQ decoder into a single ONNX model:
-```python
-class CombinedTTS(torch.nn.Module):
-    def __init__(self, flow_model, vqvae_model):
-        super().__init__()
-        self.flow = flow_model
-        self.codebook = vqvae_model.quantizer.codebook
-        self.vq_decoder = vqvae_model.decoder
-    def forward(self, text_tokens):
-        # Single forward pass: text → audio
-        # Run autoregressive loop internally
-        ...
-```
-This is more complex but lets you use sherpa-onnx's `OfflineTts` directly with a single model file.
-## Platform Comparison
-| | Android | iOS |
-|---|---|---|
-| ONNX Runtime | `onnxruntime-android` | `onnxruntime-cpu` (iOS) |
-| Flutter plugin | `flutter_onnxruntime` | same plugin |
-| Model size | ~16 MB (float16) / ~5 MB (int8) | same |
-| Inference (Pixel 6 / iPhone 13) | 300-500ms | 200-400ms |
-## Codebook Bundling
-The quantizer needs the 512×64 codebook weights at inference. Export them:
-```python
-import numpy as np
-codebook = model.vqvae.quantizer.codebook.weight.detach().numpy()
-codebook.tofile("models/codebook_weights.bin")
-```
-Load in Dart:
-```dart
-final codebookBytes = await rootBundle.load('assets/models/codebook_weights.bin');
-final codebook = Float32List.view(codebookBytes.buffer);
-// Shape: [512, 64]
-```
-## Size Optimization
-Apply int8 quantization after export:
-```python
-from onnxruntime.quantization import quantize_dynamic, QuantType
-quantize_dynamic("models/flow_lm.onnx", "models/flow_lm_int8.onnx", weight_type=QuantType.QInt8)
-quantize_dynamic("models/vq_decoder.onnx", "models/vq_decoder_int8.onnx", weight_type=QuantType.QInt8)
-```
-| Format | Size per model | Total |
-|---|---|---|
-| float16 | ~12 MB + ~4 MB | ~16 MB |
-| int8 | ~3 MB + ~1 MB | ~4 MB |
-## Performance Targets
-| Device | Latency | RTF |
-|---|---|---|
-| iPhone 13+ | 200-400ms | 3-5× realtime |
-| Pixel 6+ | 300-500ms | 2-3× realtime |
-| Mid-range Android | 500-800ms | 1-2× realtime |
-| Budget phones | 1-2s | <1× realtime |

bible-tts/docs/superpowers/plans/2026-07-08-bible-tts-implementation.md DELETED Viewed

@@ -1,1804 +0,0 @@
-# Bible-TTS Implementation Plan
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
-**Goal:** Build a textless TTS model for the English Bible using VQ-VAE codec + flow matching transformer, targeting < 10MB on Android.
-**Architecture:** Two-stage pipeline. Stage 1: VQ-VAE codec compresses 24kHz audio to 94Hz discrete tokens (512-entry codebook). Stage 2: character-level flow-matching transformer predicts token sequences from text. Inference uses the VQ decoder + flow transformer only.
-**Tech Stack:** Python 3.10+, PyTorch 2.5+, torchaudio, einops, ONNX Runtime, scipy.
-## Global Constraints
-- No external model dependencies (use only PyTorch built-ins for model components)
-- All conv layers use proper padding to preserve length (pad_mode="reflect" for conv1d)
-- float32 training throughout
-- Seed all RNGs with 42 for reproducibility
-- Character-level tokenization (88 tokens, 81 printable ASCII + space + BOS/EOS/PAD)
-- VQ-VAE frame rate: 93.75 Hz (stride 256)
-- Codebook: 512 entries × 64 dim
-- Flow transformer: 3 layers, d_model=256, 4 heads, FFN=1024
-- LSD decode steps: 4 at inference
----
-## File Structure
-```
-bible-tts/
-├── pyproject.toml
-├── src/
-│   └── bible_tts/
-│       ├── __init__.py
-│       ├── data/
-│       │   ├── __init__.py
-│       │   ├── audio_dataset.py      # AudioDataset — loads WAV + text pairs
-│       │   └── text_processor.py     # CharTokenizer — char-level encode/decode
-│       ├── models/
-│       │   ├── __init__.py
-│       │   ├── vq_vae.py            # VQVAE, Encoder, Decoder, ResBlock, VectorQuantizer
-│       │   ├── flow_lm.py           # FlowLMModel, StreamingTransformer, FlowMLP
-│       │   └── lsd.py               # lsd_decode — Lagrangian Self Distillation
-│       ├── training/
-│       │   ├── __init__.py
-│       │   ├── train_vqvae.py       # CLI: trains VQ-VAE on audio
-│       │   └── train_flow.py        # CLI: trains FlowLM on (text, tokens) pairs
-│       └── inference/
-│           ├── __init__.py
-│           ├── pipeline.py          # TTS inference pipeline
-│           └── export_onnx.py       # ONNX export + validation
-├── tests/
-│   ├── __init__.py
-│   ├── test_vq_vae.py
-│   ├── test_flow_lm.py
-│   └── test_text_processor.py
-└── docs/
-    └── android_integration.md
-```
----
-### Task 1: Project Scaffold + Data Pipeline
-**Files:**
-- Create: `pyproject.toml`
-- Create: `src/bible_tts/__init__.py`
-- Create: `src/bible_tts/data/__init__.py`
-- Create: `src/bible_tts/data/audio_dataset.py`
-- Create: `src/bible_tts/data/text_processor.py`
-- Create: `tests/__init__.py`
-- Create: `tests/test_text_processor.py`
-**Interfaces:**
-- Consumes: raw audio WAV files + verse-aligned text manifest
-- Produces: `CharTokenizer` class, `AudioDataset` class
-- [ ] **Step 1: Create pyproject.toml**
-```toml
-[project]
-name = "bible-tts"
-version = "0.1.0"
-requires-python = ">= 3.10"
-dependencies = [
-    "torch>=2.5.0",
-    "torchaudio>=2.5.0",
-    "einops>=0.4.0",
-    "scipy>=1.5.0",
-    "soundfile>=0.12.0",
-    "onnxruntime>=1.15.0",
-]
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-[tool.hatch.build.targets.wheel]
-packages = ["src/bible_tts"]
-[tool.ruff]
-line-length = 100
-[tool.pytest.ini_options]
-testpaths = ["tests"]
-```
-- [ ] **Step 2: Create CharTokenizer**
-`src/bible_tts/data/text_processor.py`:
-```python
-import torch
-PAD_TOKEN = "<PAD>"
-BOS_TOKEN = "<BOS>"
-EOS_TOKEN = "<EOS>"
-CHAR_SET = sorted(set(
-    "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
-    ".,!?;:-\"'()[]{}@#$%&*+=/\\<>~`| "
-))
-PAD_IDX = 0
-BOS_IDX = 1
-EOS_IDX = 2
-SPECIAL = {PAD_TOKEN, BOS_TOKEN, EOS_TOKEN}
-class CharTokenizer:
-    def __init__(self):
-        self.chars = [PAD_TOKEN, BOS_TOKEN, EOS_TOKEN] + CHAR_SET
-        self.char_to_idx = {c: i for i, c in enumerate(self.chars)}
-        self.idx_to_char = {i: c for i, c in enumerate(self.chars)}
-    @property
-    def vocab_size(self) -> int:
-        return len(self.chars)
-    def encode(self, text: str) -> torch.Tensor:
-        tokens = [BOS_IDX]
-        for c in text:
-            tokens.append(self.char_to_idx.get(c, PAD_IDX))
-        tokens.append(EOS_IDX)
-        return torch.tensor(tokens, dtype=torch.long)
-    def decode(self, tokens: torch.Tensor) -> str:
-        chars = []
-        for t in tokens.tolist():
-            c = self.idx_to_char.get(t, "")
-            if c in SPECIAL:
-                continue
-            chars.append(c)
-        return "".join(chars)
-```
-- [ ] **Step 3: Write and run test**
-`tests/test_text_processor.py`:
-```python
-import torch
-from bible_tts.data.text_processor import CharTokenizer
-def test_tokenizer_roundtrip():
-    tokenizer = CharTokenizer()
-    text = "In the beginning God created the heavens and the earth."
-    tokens = tokenizer.encode(text)
-    decoded = tokenizer.decode(tokens)
-    assert decoded == text, f"Roundtrip failed: {decoded} != {text}"
-def test_tokenizer_unknown_chars():
-    tokenizer = CharTokenizer()
-    text = "Hello\nWorld\t!"
-    tokens = tokenizer.encode(text)
-    decoded = tokenizer.decode(tokens)
-    # Newlines and tabs are not in CHAR_SET, replaced with PAD (dropped on decode)
-    assert "\n" not in decoded and "\t" not in decoded
-def test_tokenizer_vocab_size():
-    tokenizer = CharTokenizer()
-    assert tokenizer.vocab_size == 3 + len(
-        "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
-        ".,!?;:-\"'()[]{}@#$%&*+=/\\<>~`| "
-    )
-```
-Run: `uv run pytest tests/test_text_processor.py -v`
-Expected: 3 passed
-- [ ] **Step 4: Create AudioDataset**
-`src/bible_tts/data/audio_dataset.py`:
-```python
-import json
-from pathlib import Path
-import torch
-import torchaudio
-from torch.utils.data import Dataset
-from bible_tts.data.text_processor import CharTokenizer
-class AudioDataset(Dataset):
-    """Loads verse-aligned (audio, text) pairs.
-    Manifest format: JSONL where each line has:
-      {"audio": "path/to/verse.wav", "text": "In the beginning..."}
-    Audio is resampled to target_sr if needed.
-    """
-    def __init__(
-        self,
-        manifest_path: str | Path,
-        target_sr: int = 24000,
-        max_audio_seconds: float = 30.0,
-    ):
-        self.manifest_path = Path(manifest_path)
-        self.target_sr = target_sr
-        self.max_samples = int(max_audio_seconds * target_sr)
-        self.tokenizer = CharTokenizer()
-        with open(self.manifest_path) as f:
-            self.entries = [json.loads(line) for line in f if line.strip()]
-    def __len__(self) -> int:
-        return len(self.entries)
-    def __getitem__(self, idx: int) -> dict:
-        entry = self.entries[idx]
-        audio_path = Path(entry["audio"])
-        text = entry["text"]
-        waveform, sr = torchaudio.load(str(audio_path))
-        if sr != self.target_sr:
-            waveform = torchaudio.functional.resample(waveform, sr, self.target_sr)
-        if waveform.shape[-1] > self.max_samples:
-            waveform = waveform[..., : self.max_samples]
-        text_tokens = self.tokenizer.encode(text)
-        return {
-            "audio": waveform.squeeze(0),  # [T]
-            "text_tokens": text_tokens,  # [S]
-            "text": text,
-            "audio_path": str(audio_path),
-        }
-def collate_audio(batch: list[dict]) -> dict:
-    """Pads audio and text tokens to max length in batch."""
-    max_audio_len = max(item["audio"].shape[-1] for item in batch)
-    max_text_len = max(item["text_tokens"].shape[-1] for item in batch)
-    audio_padded = torch.zeros(len(batch), max_audio_len)
-    text_padded = torch.zeros(len(batch), max_text_len, dtype=torch.long)
-    texts = []
-    paths = []
-    for i, item in enumerate(batch):
-        audio_len = item["audio"].shape[-1]
-        text_len = item["text_tokens"].shape[-1]
-        audio_padded[i, :audio_len] = item["audio"]
-        text_padded[i, :text_len] = item["text_tokens"]
-        texts.append(item["text"])
-        paths.append(item["audio_path"])
-    return {
-        "audio": audio_padded,
-        "text_tokens": text_padded,
-        "text": texts,
-        "audio_paths": paths,
-    }
-```
-- [ ] **Step 5: Commit**
-```bash
-git add pyproject.toml src/ tests/
-git commit -m "feat: project scaffold with data pipeline and CharTokenizer"
-```
----
-### Task 2: VQ-VAE Model
-**Files:**
-- Create: `src/bible_tts/models/__init__.py`
-- Create: `src/bible_tts/models/vq_vae.py`
-- Create: `tests/test_vq_vae.py`
-**Interfaces:**
-- Consumes: `AudioDataset` (Task 1)
-- Produces: `VQVAE` class — forward returns (reconstructed_audio, quantized_tokens, commitment_loss)
-- [ ] **Step 1: Define ResBlock**
-`src/bible_tts/models/vq_vae.py`:
-```python
-import torch
-from torch import nn
-from torch.nn import functional as F
-class ResBlock(nn.Module):
-    def __init__(self, dim: int, kernel_size: int = 3, dilation: int = 1, compress: int = 2):
-        super().__init__()
-        hidden = dim // compress
-        self.conv1 = nn.Conv1d(dim, hidden, kernel_size, padding="same", dilation=dilation)
-        self.conv2 = nn.Conv1d(hidden, dim, kernel_size, padding="same")
-        self.activation = nn.ELU()
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        residual = x
-        x = self.activation(x)
-        x = self.conv1(x)
-        x = self.activation(x)
-        x = self.conv2(x)
-        return x + residual
-```
-- [ ] **Step 2: Define Encoder**
-```python
-class Encoder(nn.Module):
-    """Strided conv encoder: downsamples audio to latent codes.
-    Args:
-        in_channels: 1 for mono audio
-        base_channels: starting channel count (doubles per stride)
-        latent_dim: output latent dimension
-        strides: list of stride factors per layer
-        n_resblocks: number of ResBlocks per level
-    """
-    def __init__(
-        self,
-        in_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-    ):
-        super().__init__()
-        self.total_stride = 1
-        for s in strides:
-            self.total_stride *= s
-        layers: list[nn.Module] = [
-            nn.Conv1d(in_channels, base_channels, kernel_size=7, padding="same")
-        ]
-        mult = 1
-        for stride in strides:
-            for _ in range(n_resblocks):
-                layers.append(ResBlock(mult * base_channels))
-            layers.append(nn.ELU())
-            layers.append(
-                nn.Conv1d(
-                    mult * base_channels,
-                    mult * base_channels * 2,
-                    kernel_size=2 * stride,
-                    stride=stride,
-                    padding=stride // 2,
-                )
-            )
-            mult *= 2
-        layers.append(nn.Conv1d(mult * base_channels // 2, latent_dim, kernel_size=3, padding="same"))
-        self.layers = nn.Sequential(*layers)
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        return self.layers(x)
-```
-- [ ] **Step 3: Define VectorQuantizer**
-```python
-class VectorQuantizer(nn.Module):
-    def __init__(self, codebook_size: int = 512, dim: int = 64, beta: float = 0.25):
-        super().__init__()
-        self.codebook_size = codebook_size
-        self.dim = dim
-        self.beta = beta
-        self.codebook = nn.Embedding(codebook_size, dim)
-        nn.init.uniform_(self.codebook.weight, -1.0 / codebook_size, 1.0 / codebook_size)
-    def forward(self, z: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
-        # z: [B, D, T]
-        z_perm = z.permute(0, 2, 1)  # [B, T, D]
-        B, T, D = z_perm.shape
-        flat = z_perm.reshape(-1, D)  # [B*T, D]
-        # distances: [B*T, codebook_size]
-        dist = torch.cdist(flat, self.codebook.weight)
-        indices = dist.argmin(dim=-1)  # [B*T]
-        # quantize
-        z_q = self.codebook(indices)  # [B*T, D]
-        z_q = z_q.reshape(B, T, D).permute(0, 2, 1)  # [B, D, T]
-        # commitment loss
-        loss = F.mse_loss(z_q.detach(), z) + self.beta * F.mse_loss(z_q, z.detach())
-        # straight-through estimator
-        z_q = z + (z_q - z).detach()
-        return z_q, indices.reshape(B, T), loss
-```
-- [ ] **Step 4: Define Decoder**
-```python
-class Decoder(nn.Module):
-    """Transposed conv decoder: upsamples latent codes to audio."""
-    def __init__(
-        self,
-        out_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-    ):
-        super().__init__()
-        mult = 2 ** (len(strides) - 1)
-        layers: list[nn.Module] = [
-            nn.Conv1d(latent_dim, mult * base_channels, kernel_size=3, padding="same")
-        ]
-        for stride in strides:
-            layers.append(nn.ELU())
-            layers.append(
-                nn.ConvTranspose1d(
-                    mult * base_channels,
-                    mult * base_channels // 2,
-                    kernel_size=2 * stride,
-                    stride=stride,
-                    padding=stride // 2,
-                )
-            )
-            for _ in range(n_resblocks):
-                layers.append(ResBlock(mult * base_channels // 2))
-            mult //= 2
-        layers.append(nn.Conv1d(base_channels, out_channels, kernel_size=7, padding="same"))
-        self.layers = nn.Sequential(*layers)
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        return self.layers(x)
-```
-- [ ] **Step 5: Define VQVAE**
-```python
-class VQVAE(nn.Module):
-    def __init__(
-        self,
-        in_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        codebook_size: int = 512,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-        beta: float = 0.25,
-    ):
-        super().__init__()
-        self.encoder = Encoder(in_channels, base_channels, latent_dim, strides, n_resblocks)
-        self.quantizer = VectorQuantizer(codebook_size, latent_dim, beta)
-        self.decoder = Decoder(in_channels, base_channels, latent_dim, strides, n_resblocks)
-    @property
-    def frame_rate(self) -> float:
-        return 24000 / self.encoder.total_stride
-    def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
-        z_e = self.encoder(x)
-        z_q, indices, vq_loss = self.quantizer(z_e)
-        x_hat = self.decoder(z_q)
-        return x_hat, indices, vq_loss
-    def encode_to_indices(self, x: torch.Tensor) -> torch.Tensor:
-        z_e = self.encoder(x)
-        _, indices, _ = self.quantizer(z_e)
-        return indices
-    def decode_from_indices(self, indices: torch.Tensor) -> torch.Tensor:
-        z_q = self.quantizer.codebook(indices)
-        z_q = z_q.permute(0, 2, 1)
-        return self.decoder(z_q)
-```
-- [ ] **Step 6: Write and run tests**
-`tests/test_vq_vae.py`:
-```python
-import torch
-from bible_tts.models.vq_vae import VQVAE, ResBlock, Encoder, Decoder, VectorQuantizer
-def test_resblock_shapes():
-    block = ResBlock(dim=64)
-    x = torch.randn(2, 64, 100)
-    out = block(x)
-    assert out.shape == x.shape
-def test_encoder_output_shape():
-    enc = Encoder()
-    x = torch.randn(1, 1, 24000)
-    z = enc(x)
-    # stride 64 => 24000/64 = 375 frames
-    assert z.shape[-1] == 375
-    assert z.shape[1] == 64
-def test_quantizer_roundtrip():
-    quant = VectorQuantizer(codebook_size=512, dim=64)
-    z = torch.randn(2, 64, 100)
-    z_q, indices, loss = quant(z)
-    assert z_q.shape == z.shape
-    assert indices.shape == (2, 100)
-    assert loss.item() >= 0
-def test_decoder_output_shape():
-    dec = Decoder()
-    z = torch.randn(1, 64, 375)
-    x = dec(z)
-    assert x.shape[-1] == 24000
-    assert x.shape[1] == 1
-def test_vqvae_forward():
-    model = VQVAE()
-    x = torch.randn(1, 1, 24000)
-    x_hat, indices, vq_loss = model(x)
-    assert x_hat.shape == x.shape
-    assert indices.shape == (1, 375)
-    assert vq_loss.item() >= 0
-def test_vqvae_encode_decode():
-    model = VQVAE()
-    x = torch.randn(1, 1, 24000)
-    indices = model.encode_to_indices(x)
-    x_recon = model.decode_from_indices(indices)
-    assert indices.shape == (1, 375)
-    assert x_recon.shape == x.shape
-```
-Run: `uv run pytest tests/test_vq_vae.py -v`
-Expected: all 6 passed
-- [ ] **Step 7: Commit**
-```bash
-git add src/bible_tts/models/ tests/test_vq_vae.py
-git commit -m "feat: VQ-VAE model with Encoder, Decoder, VectorQuantizer, ResBlock"
-```
----
-### Task 3: VQ-VAE Training Script
-**Files:**
-- Create: `src/bible_tts/training/__init__.py`
-- Create: `src/bible_tts/training/train_vqvae.py`
-**Interfaces:**
-- Consumes: `AudioDataset` + `VQVAE` model
-- Produces: Saved VQ-VAE checkpoint at `checkpoints/vqvae.pt`
-- [ ] **Step 1: Create training script**
-`src/bible_tts/training/train_vqvae.py`:
-```python
-import argparse
-import json
-import logging
-import shutil
-from pathlib import Path
-import torch
-from torch import nn, optim
-from torch.utils.data import DataLoader
-from torch.optim.lr_scheduler import CosineAnnealingLR
-from bible_tts.data.audio_dataset import AudioDataset, collate_audio
-from bible_tts.models.vq_vae import VQVAE
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
-logger = logging.getLogger(__name__)
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--manifest", type=str, required=True, help="Path to manifest JSONL")
-    parser.add_argument("--checkpoint-dir", type=str, default="checkpoints")
-    parser.add_argument("--batch-size", type=int, default=16)
-    parser.add_argument("--lr", type=float, default=3e-4)
-    parser.add_argument("--weight-decay", type=float, default=1e-4)
-    parser.add_argument("--max-steps", type=int, default=200000)
-    parser.add_argument("--warmup-steps", type=int, default=5000)
-    parser.add_argument("--log-interval", type=int, default=100)
-    parser.add_argument("--save-interval", type=int, default=10000)
-    parser.add_argument("--seed", type=int, default=42)
-    parser.add_argument("--resume", type=str, default=None)
-    return parser.parse_args()
-def train():
-    args = parse_args()
-    torch.manual_seed(args.seed)
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    logger.info(f"Using device: {device}")
-    checkpoint_dir = Path(args.checkpoint_dir)
-    checkpoint_dir.mkdir(parents=True, exist_ok=True)
-    dataset = AudioDataset(args.manifest)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=True,
-        num_workers=4,
-        pin_memory=True,
-        collate_fn=collate_audio,
-    )
-    logger.info(f"Dataset: {len(dataset)} examples")
-    model = VQVAE().to(device)
-    optimizer = optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
-    scheduler = CosineAnnealingLR(optimizer, T_max=args.max_steps - args.warmup_steps)
-    scaler = torch.amp.GradScaler("cuda" if torch.cuda.is_available() else "cpu")
-    start_step = 0
-    if args.resume:
-        state = torch.load(args.resume, map_location=device)
-        model.load_state_dict(state["model"])
-        optimizer.load_state_dict(state["optimizer"])
-        start_step = state["step"]
-        logger.info(f"Resumed from step {start_step}")
-    step = start_step
-    best_loss = float("inf")
-    while step < args.max_steps:
-        for batch in loader:
-            if step >= args.max_steps:
-                break
-            audio = batch["audio"].unsqueeze(1).to(device)  # [B, 1, T]
-            crop_len = min(audio.shape[-1], 48000)  # 2 sec at 24kHz
-            if audio.shape[-1] > crop_len:
-                offset = torch.randint(0, audio.shape[-1] - crop_len, (1,)).item()
-                audio = audio[..., offset : offset + crop_len]
-            with torch.amp.autocast(device_type="cuda" if torch.cuda.is_available() else "cpu"):
-                x_hat, indices, vq_loss = model(audio)
-                recon_loss = nn.functional.l1_loss(x_hat, audio)
-                total_loss = recon_loss + vq_loss
-            scaler.scale(total_loss).backward()
-            scaler.unscale_(optimizer)
-            nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-            scaler.step(optimizer)
-            scaler.update()
-            optimizer.zero_grad()
-            if step < args.warmup_steps:
-                lr_scale = min(1.0, (step + 1) / args.warmup_steps)
-                for pg in optimizer.param_groups:
-                    pg["lr"] = args.lr * lr_scale
-            else:
-                scheduler.step()
-            if step % args.log_interval == 0:
-                logger.info(
-                    f"Step {step}/{args.max_steps} | recon: {recon_loss.item():.4f} | "
-                    f"vq: {vq_loss.item():.4f} | total: {total_loss.item():.4f}"
-                )
-            if step % args.save_interval == 0 and step > 0:
-                state = {
-                    "model": model.state_dict(),
-                    "optimizer": optimizer.state_dict(),
-                    "step": step,
-                    "loss": total_loss.item(),
-                }
-                ckpt_path = checkpoint_dir / f"vqvae_step_{step}.pt"
-                torch.save(state, ckpt_path)
-                logger.info(f"Saved checkpoint to {ckpt_path}")
-                if total_loss.item() < best_loss:
-                    best_loss = total_loss.item()
-                    shutil.copy(ckpt_path, checkpoint_dir / "vqvae_best.pt")
-                    logger.info(f"New best loss: {best_loss:.4f}")
-            step += 1
-    state = {
-        "model": model.state_dict(),
-        "optimizer": optimizer.state_dict(),
-        "step": step,
-        "loss": total_loss.item(),
-    }
-    torch.save(state, checkpoint_dir / "vqvae_final.pt")
-    logger.info(f"Training complete. Final model saved to {checkpoint_dir / 'vqvae_final.pt'}")
-```
-- [ ] **Step 2: Create a simple manifest from en_bsb.txt**
-Also create a helper script to generate the manifest:
-```python
-# scripts/create_manifest.py
-"""Convert en_bsb.txt + audio dir into a JSONL manifest."""
-import json
-from pathlib import Path
-def create_manifest(
-    bible_text_path: str,
-    audio_dir: str,
-    output_path: str,
-    audio_format: str = "wav",
-):
-    """Match verses in bible_text_path to audio files in audio_dir."""
-    with open(bible_text_path) as f:
-        lines = f.readlines()
-    audio_dir = Path(audio_dir)
-    entries = []
-    missing = 0
-    for line in lines:
-        parts = line.strip().split("|")
-        if len(parts) < 7:
-            continue
-        book, chapter, verse = parts[0], parts[1], parts[2]
-        text = parts[6]
-        # Audio file naming convention: adjust to match your files
-        audio_name = f"{book}_{chapter}_{verse}.{audio_format}"
-        audio_path = audio_dir / audio_name
-        if audio_path.exists():
-            entries.append({"audio": str(audio_path), "text": text})
-        else:
-            missing += 1
-    with open(output_path, "w") as f:
-        for entry in entries:
-            f.write(json.dumps(entry) + "\n")
-    print(f"Created manifest with {len(entries)} entries, {missing} missing audio files")
-```
-- [ ] **Step 3: Commit**
-```bash
-git add src/bible_tts/training/
-git commit -m "feat: VQ-VAE training script with warmup, cosine schedule, checkpointing"
-```
----
-### Task 4: Audio Encoding Script
-**Files:**
-- Create: `scripts/encode_audio.py`
-**Interfaces:**
-- Consumes: trained VQ-VAE checkpoint + manifest with audio paths
-- Produces: encoded dataset JSONL `(text, code_tokens)` for flow training
-- [ ] **Step 1: Create encoding script**
-```python
-"""Encode all audio through trained VQ-VAE to create (text, code_tokens) pairs."""
-import json
-import argparse
-from pathlib import Path
-import torch
-from torch.utils.data import DataLoader
-from bible_tts.data.audio_dataset import AudioDataset, collate_audio
-from bible_tts.models.vq_vae import VQVAE
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--manifest", type=str, required=True)
-    parser.add_argument("--checkpoint", type=str, required=True)
-    parser.add_argument("--output", type=str, required=True)
-    parser.add_argument("--batch-size", type=int, default=32)
-    args = parser.parse_args()
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    model = VQVAE().to(device)
-    state = torch.load(args.checkpoint, map_location=device)
-    model.load_state_dict(state["model"])
-    model.eval()
-    dataset = AudioDataset(args.manifest)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=False,
-        num_workers=4,
-        collate_fn=collate_audio,
-    )
-    output_path = Path(args.output)
-    output_path.parent.mkdir(parents=True, exist_ok=True)
-    with open(output_path, "w") as f:
-        with torch.no_grad():
-            for batch in loader:
-                audio = batch["audio"].unsqueeze(1).to(device)
-                indices = model.encode_to_indices(audio)
-                for i in range(len(batch["text"])):
-                    tokens = indices[i].tolist()
-                    entry = {
-                        "text": batch["text"][i],
-                        "codes": tokens,
-                        "audio_path": batch["audio_paths"][i],
-                    }
-                    f.write(json.dumps(entry) + "\n")
-    print(f"Encoded {len(dataset)} examples to {args.output}")
-```
-- [ ] **Step 2: Commit**
-```bash
-git add scripts/encode_audio.py
-git commit -m "feat: audio encoding script for VQ-VAE tokenization"
-```
----
-### Task 5: Flow LM Model
-**Files:**
-- Create: `src/bible_tts/models/lsd.py`
-- Create: `src/bible_tts/models/flow_lm.py`
-- Create: `tests/test_flow_lm.py`
-**Interfaces:**
-- Consumes: `CharTokenizer` (Task 1), encoded codec tokens (Task 4)
-- Produces: `FlowLMModel` — generates latent sequences from text
-- [ ] **Step 1: Create LSD decode**
-`src/bible_tts/models/lsd.py`:
-```python
-import torch
-from functools import partial
-def lsd_decode(
-    v_t: callable,
-    x_0: torch.Tensor,
-    num_steps: int = 4,
-) -> torch.Tensor:
-    """Lagrangian Self Distillation.
-    Args:
-        v_t: function (t_prev, t_curr, x_t) -> velocity
-        x_0: starting noise [B, D]
-        num_steps: number of integration steps
-    """
-    current = x_0
-    for i in range(num_steps):
-        s = i / num_steps
-        t = (i + 1) / num_steps
-        flow_dir = v_t(
-            s * torch.ones_like(current[..., :1]),
-            t * torch.ones_like(current[..., :1]),
-            current,
-        )
-        current = current + flow_dir / num_steps
-    return current
-```
-- [ ] **Step 2: Create StreamingTransformer**
-`src/bible_tts/models/flow_lm.py`:
-```python
-import torch
-from torch import nn
-import einops
-class StreamingMultiheadAttention(nn.Module):
-    def __init__(self, d_model: int, num_heads: int, max_period: float = 10000.0):
-        super().__init__()
-        self.d_model = d_model
-        self.num_heads = num_heads
-        self.head_dim = d_model // num_heads
-        self.max_period = max_period
-        self.wq = nn.Linear(d_model, d_model, bias=False)
-        self.wk = nn.Linear(d_model, d_model, bias=False)
-        self.wv = nn.Linear(d_model, d_model, bias=False)
-        self.wo = nn.Linear(d_model, d_model, bias=False)
-    def _rope(self, x: torch.Tensor, offset: int = 0) -> torch.Tensor:
-        """Apply rotary position embeddings."""
-        B, T, H, D = x.shape
-        positions = torch.arange(offset, offset + T, device=x.device, dtype=torch.float32)
-        freqs = torch.exp(
-            -torch.arange(0, D, 2, device=x.device, dtype=torch.float32)
-            * (torch.log(torch.tensor(self.max_period)) / D)
-        )
-        angles = positions[:, None] * freqs[None, :]
-        cos, sin = angles.cos(), angles.sin()
-        x_rot = x.clone()
-        x_rot[..., 0::2] = x[..., 0::2] * cos - x[..., 1::2] * sin
-        x_rot[..., 1::2] = x[..., 1::2] * cos + x[..., 0::2] * sin
-        return x_rot
-    def forward(self, x: torch.Tensor, kv_cache: torch.Tensor | None = None) -> tuple[torch.Tensor, torch.Tensor]:
-        B, T, _ = x.shape
-        q = einops.rearrange(self.wq(x), "B T (H D) -> B T H D", H=self.num_heads)
-        k = einops.rearrange(self.wk(x), "B T (H D) -> B T H D", H=self.num_heads)
-        v = einops.rearrange(self.wv(x), "B T (H D) -> B T H D", H=self.num_heads)
-        q = self._rope(q)
-        k = self._rope(k)
-        if kv_cache is not None:
-            k = torch.cat([kv_cache[0], k], dim=1)
-            v = torch.cat([kv_cache[1], v], dim=1)
-        attn = torch.einsum("bthd,bThd->bhtT", q, k) / (self.head_dim ** 0.5)
-        attn = torch.softmax(attn, dim=-1)
-        out = torch.einsum("bhtT,bThd->bthd", attn, v)
-        out = einops.rearrange(out, "B T H D -> B T (H D)")
-        out = self.wo(out)
-        new_cache = torch.stack([k, v], dim=0)
-        return out, new_cache
-class StreamingTransformerLayer(nn.Module):
-    def __init__(self, d_model: int, num_heads: int, ff_hidden: int, max_period: float = 10000.0):
-        super().__init__()
-        self.attention = StreamingMultiheadAttention(d_model, num_heads, max_period)
-        self.norm1 = nn.LayerNorm(d_model)
-        self.ffn = nn.Sequential(
-            nn.Linear(d_model, ff_hidden),
-            nn.GELU(),
-            nn.Linear(ff_hidden, d_model),
-        )
-        self.norm2 = nn.LayerNorm(d_model)
-    def forward(self, x: torch.Tensor, kv_cache: torch.Tensor | None = None) -> tuple[torch.Tensor, torch.Tensor]:
-        attn_out, new_cache = self.attention(self.norm1(x), kv_cache)
-        x = x + attn_out
-        x = x + self.ffn(self.norm2(x))
-        return x, new_cache
-class StreamingTransformer(nn.Module):
-    def __init__(self, d_model: int, num_layers: int, num_heads: int, ff_hidden: int):
-        super().__init__()
-        self.layers = nn.ModuleList([
-            StreamingTransformerLayer(d_model, num_heads, ff_hidden)
-            for _ in range(num_layers)
-        ])
-    def forward(self, x: torch.Tensor, kv_caches: list[torch.Tensor] | None = None) -> tuple[torch.Tensor, list[torch.Tensor]]:
-        new_caches = []
-        for i, layer in enumerate(self.layers):
-            cache = kv_caches[i] if kv_caches is not None else None
-            x, new_cache = layer(x, cache)
-            new_caches.append(new_cache)
-        return x, new_caches
-```
-- [ ] **Step 3: Create FlowMLP (AdaLN)**
-```python
-class FlowMLP(nn.Module):
-    """AdaLN-conditioned MLP for flow prediction.
-    Takes conditioning vector c and input x_t, predicts flow direction.
-    """
-    def __init__(self, d_model: int, latent_dim: int, hidden: int = 1024):
-        super().__init__()
-        self.cond_proj = nn.Linear(d_model, hidden * 2)  # scale + shift
-        self.net = nn.Sequential(
-            nn.Linear(latent_dim + 2, hidden),  # +2 for t_prev, t_curr
-            nn.GELU(),
-            nn.Linear(hidden, hidden),
-            nn.GELU(),
-            nn.Linear(hidden, latent_dim),
-        )
-    def forward(self, c: torch.Tensor, t_prev: torch.Tensor, t_curr: torch.Tensor, x_t: torch.Tensor) -> torch.Tensor:
-        scale, shift = self.cond_proj(c).chunk(2, dim=-1)
-        cond = torch.cat([t_prev, t_curr], dim=-1)
-        inp = torch.cat([cond, x_t], dim=-1)
-        h = self.net(inp)
-        return h * (1 + scale) + shift
-```
-- [ ] **Step 4: Create FlowLMModel**
-```python
-class FlowLMModel(nn.Module):
-    def __init__(
-        self,
-        vocab_size: int = 88,
-        d_embed: int = 128,
-        d_model: int = 256,
-        latent_dim: int = 64,
-        num_layers: int = 3,
-        num_heads: int = 4,
-        ff_hidden: int = 1024,
-        mlp_hidden: int = 1024,
-        codebook_size: int = 512,
-    ):
-        super().__init__()
-        self.latent_dim = latent_dim
-        self.d_model = d_model
-        self.char_embed = nn.Embedding(vocab_size, d_embed)
-        self.text_proj = nn.Linear(d_embed, d_model)
-        self.code_embed = nn.Embedding(codebook_size, latent_dim)
-        self.input_proj = nn.Linear(latent_dim, d_model)
-        self.transformer = StreamingTransformer(d_model, num_layers, num_heads, ff_hidden)
-        self.out_norm = nn.LayerNorm(d_model)
-        self.flow_mlp = FlowMLP(d_model, latent_dim, mlp_hidden)
-        # EOS prediction head
-        self.eos_head = nn.Linear(d_model, 1)
-        # BOS embedding for audio
-        self.bos_code = nn.Parameter(torch.randn(latent_dim))
-    def forward(
-        self,
-        text_tokens: torch.Tensor,
-        code_tokens: torch.Tensor,
-        kv_caches: list[torch.Tensor] | None = None,
-        lsd_steps: int = 4,
-        temp: float = 1.0,
-        noise_clamp: float | None = None,
-        eos_threshold: float = 0.5,
-    ) -> tuple[torch.Tensor, torch.Tensor]:
-        """
-        Args:
-            text_tokens: [B, S] character token ids
-            code_tokens: [B, T] codec token ids (previous audio latents)
-            kv_caches: optional KV caches for streaming
-            lsd_steps: number of LSD steps (0 = training, >0 = inference)
-        Returns:
-            next_latent: [B, latent_dim] predicted latent
-            is_eos: [B, 1] EOS prediction
-        """
-        B = text_tokens.shape[0]
-        # Text embeddings
-        text_emb = self.char_embed(text_tokens)  # [B, S, d_embed]
-        text_emb = self.text_proj(text_emb)  # [B, S, d_model]
-        # Code token embeddings
-        code_emb = self.code_embed(code_tokens)  # [B, T, latent_dim]
-        code_emb = self.input_proj(code_emb)  # [B, T, d_model]
-        # Concatenate text + code
-        combined = torch.cat([text_emb, code_emb], dim=1)
-        # Transformer forward
-        out, new_caches = self.transformer(combined, kv_caches)
-        out = self.out_norm(out)
-        # Get last position output
-        last_out = out[:, -1]  # [B, d_model]
-        # EOS prediction
-        is_eos = (self.eos_head(last_out) > eos_threshold).float()
-        if lsd_steps == 0:
-            return last_out, is_eos
-        # LSD decode for inference
-        noise_shape = (B, self.latent_dim)
-        noise = torch.randn(noise_shape, device=text_tokens.device)
-        if noise_clamp is not None:
-            noise = noise.clamp(-noise_clamp, noise_clamp)
-        noise = noise * temp
-        def flow_fn(t_prev, t_curr, x_t):
-            return self.flow_mlp(last_out, t_prev, t_curr, x_t)
-        next_latent = lsd_decode(flow_fn, noise, lsd_steps)
-        return next_latent, is_eos
-    def sample_latent(
-        self,
-        text_tokens: torch.Tensor,
-        code_tokens: torch.Tensor,
-        kv_caches: list[torch.Tensor] | None = None,
-        lsd_steps: int = 4,
-        temp: float = 1.0,
-        noise_clamp: float | None = None,
-        eos_threshold: float = 0.5,
-    ) -> tuple[torch.Tensor, torch.Tensor, list[torch.Tensor]]:
-        next_latent, is_eos = self(
-            text_tokens=text_tokens,
-            code_tokens=code_tokens,
-            kv_caches=kv_caches,
-            lsd_steps=lsd_steps,
-            temp=temp,
-            noise_clamp=noise_clamp,
-            eos_threshold=eos_threshold,
-        )
-        return next_latent, is_eos
-```
-- [ ] **Step 5: Write and run tests**
-`tests/test_flow_lm.py`:
-```python
-import torch
-from bible_tts.models.flow_lm import (
-    StreamingMultiheadAttention,
-    StreamingTransformerLayer,
-    StreamingTransformer,
-    FlowMLP,
-    FlowLMModel,
-)
-from bible_tts.models.lsd import lsd_decode
-def test_attention_shapes():
-    attn = StreamingMultiheadAttention(d_model=256, num_heads=4)
-    x = torch.randn(2, 10, 256)
-    out, cache = attn(x)
-    assert out.shape == x.shape
-    assert cache.shape == (2, 2, 10, 4, 64)
-def test_attention_with_cache():
-    attn = StreamingMultiheadAttention(d_model=256, num_heads=4)
-    x1 = torch.randn(1, 5, 256)
-    x2 = torch.randn(1, 3, 256)
-    _, cache = attn(x1)
-    out2, cache2 = attn(x2, cache)
-    assert out2.shape == x2.shape
-    assert cache2.shape[2] == 8
-def test_transformer_layer():
-    layer = StreamingTransformerLayer(d_model=256, num_heads=4, ff_hidden=1024)
-    x = torch.randn(2, 10, 256)
-    out, cache = layer(x)
-    assert out.shape == x.shape
-def test_transformer():
-    tf = StreamingTransformer(d_model=256, num_layers=3, num_heads=4, ff_hidden=1024)
-    x = torch.randn(2, 10, 256)
-    out, caches = tf(x)
-    assert len(caches) == 3
-    assert out.shape == x.shape
-def test_flow_mlp():
-    mlp = FlowMLP(d_model=256, latent_dim=64)
-    c = torch.randn(2, 256)
-    t1 = torch.randn(2, 1)
-    t2 = torch.randn(2, 1)
-    x = torch.randn(2, 64)
-    out = mlp(c, t1, t2, x)
-    assert out.shape == x.shape
-def test_flow_lm_forward():
-    model = FlowLMModel()
-    text = torch.randint(0, 88, (1, 50))
-    codes = torch.randint(0, 512, (1, 100))
-    last_out, is_eos = model(text, codes)
-    assert last_out.shape == (1, 256)
-    assert is_eos.shape == (1, 1)
-def test_lsd_decode():
-    def flow_fn(t_prev, t_curr, x_t):
-        return -x_t
-    x_0 = torch.randn(2, 64)
-    result = lsd_decode(flow_fn, x_0, num_steps=4)
-    assert result.shape == x_0.shape
-```
-Run: `uv run pytest tests/test_flow_lm.py -v`
-Expected: all 8 passed
-- [ ] **Step 6: Commit**
-```bash
-git add src/bible_tts/models/lsd.py src/bible_tts/models/flow_lm.py tests/test_flow_lm.py
-git commit -m "feat: FlowLM model with streaming transformer, flow MLP, and LSD"
-```
----
-### Task 6: Flow LM Training Script
-**Files:**
-- Create: `src/bible_tts/training/train_flow.py`
-**Interfaces:**
-- Consumes: encoded dataset (text tokens + codec tokens from Task 4)
-- Produces: Saved FlowLM checkpoint
-- [ ] **Step 1: Create Flow training script**
-`src/bible_tts/training/train_flow.py`:
-```python
-import argparse
-import json
-import logging
-import shutil
-from pathlib import Path
-import torch
-from torch import nn, optim
-from torch.utils.data import Dataset, DataLoader
-from torch.optim.lr_scheduler import CosineAnnealingLR
-from bible_tts.data.text_processor import CharTokenizer
-from bible_tts.models.flow_lm import FlowLMModel
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
-logger = logging.getLogger(__name__)
-class CodecDataset(Dataset):
-    """Dataset of (text, codec_token_sequence) pairs from encoded JSONL."""
-    def __init__(self, encoded_path: str, max_code_len: int = 1000):
-        self.tokenizer = CharTokenizer()
-        self.max_code_len = max_code_len
-        with open(encoded_path) as f:
-            self.entries = [json.loads(line) for line in f if line.strip()]
-    def __len__(self):
-        return len(self.entries)
-    def __getitem__(self, idx):
-        entry = self.entries[idx]
-        text_tokens = self.tokenizer.encode(entry["text"])
-        codes = torch.tensor(entry["codes"], dtype=torch.long)
-        if codes.shape[-1] > self.max_code_len:
-            codes = codes[:self.max_code_len]
-        return {"text_tokens": text_tokens, "codes": codes}
-def collate_fn(batch):
-    max_text = max(item["text_tokens"].shape[-1] for item in batch)
-    max_code = max(item["codes"].shape[-1] for item in batch)
-    text_padded = torch.zeros(len(batch), max_text, dtype=torch.long)
-    code_padded = torch.zeros(len(batch), max_code, dtype=torch.long)
-    code_masks = torch.zeros(len(batch), max_code, dtype=torch.bool)
-    for i, item in enumerate(batch):
-        tl = item["text_tokens"].shape[-1]
-        cl = item["codes"].shape[-1]
-        text_padded[i, :tl] = item["text_tokens"]
-        code_padded[i, :cl] = item["codes"]
-        code_masks[i, :cl] = True
-    return {"text_tokens": text_padded, "codes": code_padded, "code_masks": code_masks}
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--encoded-data", type=str, required=True)
-    parser.add_argument("--checkpoint-dir", type=str, default="checkpoints")
-    parser.add_argument("--batch-size", type=int, default=32)
-    parser.add_argument("--lr", type=float, default=2e-4)
-    parser.add_argument("--weight-decay", type=float, default=1e-4)
-    parser.add_argument("--max-steps", type=int, default=300000)
-    parser.add_argument("--warmup-steps", type=int, default=5000)
-    parser.add_argument("--log-interval", type=int, default=100)
-    parser.add_argument("--save-interval", type=int, default=10000)
-    parser.add_argument("--seed", type=int, default=42)
-    parser.add_argument("--resume", type=str, default=None)
-    return parser.parse_args()
-def train():
-    args = parse_args()
-    torch.manual_seed(args.seed)
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    logger.info(f"Using device: {device}")
-    checkpoint_dir = Path(args.checkpoint_dir)
-    checkpoint_dir.mkdir(parents=True, exist_ok=True)
-    dataset = CodecDataset(args.encoded_data)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=True,
-        num_workers=4,
-        pin_memory=True,
-        collate_fn=collate_fn,
-    )
-    logger.info(f"Dataset: {len(dataset)} examples")
-    model = FlowLMModel().to(device)
-    optimizer = optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
-    scheduler = CosineAnnealingLR(optimizer, T_max=args.max_steps - args.warmup_steps)
-    scaler = torch.amp.GradScaler(device="cuda" if torch.cuda.is_available() else "cpu")
-    start_step = 0
-    if args.resume:
-        state = torch.load(args.resume, map_location=device)
-        model.load_state_dict(state["model"])
-        optimizer.load_state_dict(state["optimizer"])
-        start_step = state["step"]
-        logger.info(f"Resumed from step {start_step}")
-    step = start_step
-    best_loss = float("inf")
-    while step < args.max_steps:
-        for batch in loader:
-            if step >= args.max_steps:
-                break
-            text = batch["text_tokens"].to(device)
-            codes = batch["codes"].to(device)
-            # Flow matching training
-            B, T = codes.shape
-            latent_dim = model.latent_dim
-            # Convert code tokens to latent embeddings
-            code_emb = model.code_embed(codes)  # [B, T, latent_dim]
-            # Sample random timestep for flow matching
-            t = torch.rand(B, 1, device=device)
-            noise = torch.randn_like(code_emb)
-            def flow_fn(c, t_prev, t_curr, x_t):
-                return model.flow_mlp(c, t_prev, t_curr, x_t)
-            with torch.amp.autocast(device_type="cuda" if torch.cuda.is_available() else "cpu"):
-                # Text embedding
-                text_emb = model.char_embed(text)
-                text_emb = model.text_proj(text_emb)
-                # Concatenate and run transformer
-                combined = torch.cat([text_emb, code_emb], dim=1)
-                out, _ = model.transformer(combined)
-                out = model.out_norm(out)
-                # For each position, predict velocity
-                cond = out[:, -1]  # last position -> conditioning
-                noise_shape = (B, latent_dim)
-                t_prev = t * 0  # starting from noise
-                t_curr = t
-                x_t = (1 - t) * noise + t * code_emb[:, -1]
-                v_pred = model.flow_mlp(cond, t_prev, t_curr, x_t)
-                v_target = code_emb[:, -1] - noise  # ground truth
-                flow_loss = nn.functional.mse_loss(v_pred, v_target)
-                # EOS loss (predict end of sequence)
-                # For simplicity, treat last token as EOS
-                eos_target = torch.zeros(B, 1, device=device)
-                eos_logits = model.eos_head(out[:, -1])
-                eos_loss = nn.functional.binary_cross_entropy_with_logits(
-                    eos_logits, eos_target
-                )
-                total_loss = flow_loss + eos_loss
-            scaler.scale(total_loss).backward()
-            scaler.unscale_(optimizer)
-            nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-            scaler.step(optimizer)
-            scaler.update()
-            optimizer.zero_grad()
-            if step < args.warmup_steps:
-                lr_scale = min(1.0, (step + 1) / args.warmup_steps)
-                for pg in optimizer.param_groups:
-                    pg["lr"] = args.lr * lr_scale
-            else:
-                scheduler.step()
-            if step % args.log_interval == 0:
-                logger.info(
-                    f"Step {step}/{args.max_steps} | flow: {flow_loss.item():.4f} | "
-                    f"eos: {eos_loss.item():.4f} | total: {total_loss.item():.4f}"
-                )
-            if step % args.save_interval == 0 and step > 0:
-                state = {
-                    "model": model.state_dict(),
-                    "optimizer": optimizer.state_dict(),
-                    "step": step,
-                    "loss": total_loss.item(),
-                }
-                ckpt_path = checkpoint_dir / f"flow_step_{step}.pt"
-                torch.save(state, ckpt_path)
-                logger.info(f"Saved checkpoint to {ckpt_path}")
-                if total_loss.item() < best_loss:
-                    best_loss = total_loss.item()
-                    shutil.copy(ckpt_path, checkpoint_dir / "flow_best.pt")
-                    logger.info(f"New best loss: {best_loss:.4f}")
-            step += 1
-    state = {
-        "model": model.state_dict(),
-        "optimizer": optimizer.state_dict(),
-        "step": step,
-        "loss": total_loss.item(),
-    }
-    torch.save(state, checkpoint_dir / "flow_final.pt")
-    logger.info(f"Training complete. Final model saved to {checkpoint_dir / 'flow_final.pt'}")
-```
-- [ ] **Step 2: Commit**
-```bash
-git add src/bible_tts/training/train_flow.py
-git commit -m "feat: FlowLM training script with flow matching and EOS loss"
-```
----
-### Task 7: Inference Pipeline + ONNX Export
-**Files:**
-- Create: `src/bible_tts/inference/__init__.py`
-- Create: `src/bible_tts/inference/pipeline.py`
-- Create: `src/bible_tts/inference/export_onnx.py`
-**Interfaces:**
-- Consumes: VQVAE checkpoint + FlowLM checkpoint
-- Produces: TTS text→audio + ONNX exported models
-- [ ] **Step 1: Create TTS pipeline**
-`src/bible_tts/inference/pipeline.py`:
-```python
-import torch
-from bible_tts.data.text_processor import CharTokenizer
-from bible_tts.models.vq_vae import VQVAE
-from bible_tts.models.flow_lm import FlowLMModel
-class TTSPipeline:
-    """Full text-to-speech inference pipeline.
-    Text → CharTokenizer → FlowLM → latent codes → VQ Decoder → audio
-    """
-    def __init__(
-        self,
-        vqvae_checkpoint: str,
-        flow_checkpoint: str,
-        device: str = "cpu",
-        lsd_steps: int = 4,
-    ):
-        self.device = torch.device(device)
-        self.lsd_steps = lsd_steps
-        self.tokenizer = CharTokenizer()
-        self.vqvae = VQVAE().to(self.device)
-        vq_state = torch.load(vqvae_checkpoint, map_location=self.device)
-        self.vqvae.load_state_dict(vq_state["model"])
-        self.vqvae.eval()
-        self.flow = FlowLMModel().to(self.device)
-        flow_state = torch.load(flow_checkpoint, map_location=self.device)
-        self.flow.load_state_dict(flow_state["model"])
-        self.flow.eval()
-    @torch.no_grad()
-    def generate(self, text: str) -> torch.Tensor:
-        """Generate audio from text. Returns [1, T] waveform."""
-        text_tokens = self.tokenizer.encode(text).unsqueeze(0).to(self.device)  # [1, S]
-        # Autoregressive generation loop
-        max_frames = int(300 * self.vqvae.frame_rate)  # ~3.2 seconds at 94Hz
-        code_tokens = torch.zeros((1, 0), dtype=torch.long, device=self.device)
-        latents = []
-        for step in range(max_frames):
-            # Use the last generated code (or BOS for first)
-            if step == 0:
-                current_codes = torch.zeros((1, 1), dtype=torch.long, device=self.device)
-            else:
-                current_codes = code_tokens[:, -1:]
-            next_latent, is_eos = self.flow(
-                text_tokens=text_tokens,
-                code_tokens=current_codes,
-                lsd_steps=self.lsd_steps,
-            )
-            # Quantize to nearest codebook entry
-            dist = torch.cdist(
-                next_latent.unsqueeze(1),
-                self.flow.code_embed.weight.unsqueeze(0),
-            )
-            next_code = dist.argmin(dim=-1)  # [1, 1]
-            code_tokens = torch.cat([code_tokens, next_code], dim=-1)
-            latents.append(next_latent)
-            if is_eos.item() and step > 10:
-                break
-        # Decode all latents through VQ decoder
-        if len(latents) == 0:
-            return torch.zeros(1, 24000)
-        # Alternatively, decode using indices + decoder
-        audio = self.vqvae.decode_from_indices(code_tokens)
-        return audio
-    @torch.no_grad()
-    def generate_fast(self, text: str) -> torch.Tensor:
-        """Non-autoregressive generation (single step)."""
-        text_tokens = self.tokenizer.encode(text).unsqueeze(0).to(self.device)
-        # Get text embedding
-        text_emb = self.flow.char_embed(text_tokens)
-        text_emb = self.flow.text_proj(text_emb)
-        # Generate all at once
-        noise = torch.randn(1, self.flow.latent_dim, device=self.device)
-        def flow_fn(c, t_prev, t_curr, x_t):
-            combined = torch.cat([text_emb[:, -1:], c.unsqueeze(1)], dim=1)
-            out, _ = self.flow.transformer(combined)
-            out = self.flow.out_norm(out)
-            cond = out[:, -1]
-            return self.flow.flow_mlp(cond, t_prev, t_curr, x_t)
-        from bible_tts.models.lsd import lsd_decode
-        generated = lsd_decode(
-            lambda tp, tc, xt: flow_fn(text_emb[:, -1], tp, tc, xt),
-            noise,
-            self.lsd_steps,
-        )
-        dist = torch.cdist(
-            generated.unsqueeze(1),
-            self.flow.code_embed.weight.unsqueeze(0),
-        )
-        code = dist.argmin(dim=-1)
-        audio = self.vqvae.decode_from_indices(code)
-        return audio
-```
-- [ ] **Step 2: Create ONNX export script**
-`src/bible_tts/inference/export_onnx.py`:
-```python
-import torch
-import onnxruntime
-from bible_tts.models.flow_lm import FlowLMModel
-from bible_tts.models.vq_vae import VQVAE
-def export_flow_lm_onnx(checkpoint_path: str, output_path: str):
-    """Export FlowLM inference graph to ONNX."""
-    model = FlowLMModel()
-    state = torch.load(checkpoint_path, map_location="cpu")
-    model.load_state_dict(state["model"])
-    model.eval()
-    text_tokens = torch.randint(0, 88, (1, 50), dtype=torch.long)
-    code_tokens = torch.randint(0, 512, (1, 1), dtype=torch.long)
-    torch.onnx.export(
-        model,
-        (text_tokens, code_tokens, None, 4, 1.0, None, 0.5),
-        output_path,
-        input_names=["text_tokens", "code_tokens", "kv_caches", "lsd_steps", "temp", "noise_clamp", "eos_threshold"],
-        output_names=["next_latent", "is_eos"],
-        dynamic_axes={
-            "text_tokens": {1: "text_length"},
-            "code_tokens": {1: "code_length"},
-        },
-        opset_version=17,
-    )
-    print(f"FlowLM ONNX model exported to {output_path}")
-def export_vqvae_decoder_onnx(checkpoint_path: str, output_path: str):
-    """Export VQ decoder + codebook to ONNX."""
-    from bible_tts.models.vq_vae import VQVAE
-    model = VQVAE()
-    state = torch.load(checkpoint_path, map_location="cpu")
-    model.load_state_dict(state["model"])
-    model.eval()
-    class VQDecoderWrapper(torch.nn.Module):
-        def __init__(self, codebook, decoder):
-            super().__init__()
-            self.codebook = codebook
-            self.decoder = decoder
-        def forward(self, indices: torch.Tensor) -> torch.Tensor:
-            z_q = self.codebook(indices)
-            z_q = z_q.permute(0, 2, 1)
-            return self.decoder(z_q)
-    wrapper = VQDecoderWrapper(model.quantizer.codebook, model.decoder)
-    wrapper.eval()
-    indices = torch.randint(0, 512, (1, 375), dtype=torch.long)
-    torch.onnx.export(
-        wrapper,
-        indices,
-        output_path,
-        input_names=["indices"],
-        output_names=["audio"],
-        dynamic_axes={
-            "indices": {1: "num_frames"},
-        },
-        opset_version=17,
-    )
-    print(f"VQ Decoder ONNX model exported to {output_path}")
-def validate_onnx(onnx_path: str, sample_inputs: list):
-    session = onnxruntime.InferenceSession(onnx_path)
-    input_names = [inp.name for inp in session.get_inputs()]
-    output_names = [out.name for out in session.get_outputs()]
-    feed = dict(zip(input_names, sample_inputs))
-    outputs = session.run(output_names, feed)
-    print(f"ONNX validation passed for {onnx_path}")
-    return outputs
-```
-- [ ] **Step 3: Generate a sample and commit**
-```bash
-git add src/bible_tts/inference/
-git commit -m "feat: inference pipeline and ONNX export scripts"
-```
----
-### Task 8: Android Integration Guide
-**Files:**
-- Create: `docs/android_integration.md`
-- [ ] **Step 1: Write integration guide**
-`docs/android_integration.md` — brief doc covering:
-1. Building ONNX Runtime for Android (ARM64)
-2. Loading the two ONNX models
-3. Text tokenization in Java/Kotlin
-4. Autoregressive generation loop
-5. Audio playback from float32 samples
-6. Memory management and model caching
-```markdown
-# Android Integration Guide
-## Dependencies
-Add to your `build.gradle.kts`:
-```kotlin
-implementation("com.microsoft.onnxruntime:onnxruntime-android:latest")
-```
-## Model Files
-Copy to `app/src/main/assets/models/`:
-- `flow_lm.onnx` (~12 MB float16)
-- `vq_decoder.onnx` (~4 MB float16)
-Total: ~16 MB + 5 MB runtime = ~21 MB APK impact (can be reduced via int8 quantization).
-## Basic Usage (Kotlin)
-```kotlin
-class BibleTTS(private val context: Context) {
-    private val flowSession: OrtSession
-    private val decoderSession: OrtSession
-    private val environment: OrtEnvironment
-    init {
-        environment = OrtEnvironment.getEnvironment()
-        val flowModel = loadModelFile("models/flow_lm.onnx")
-        val decoderModel = loadModelFile("models/vq_decoder.onnx")
-        flowSession = environment.createSession(flowModel, OrtSession.SessionOptions())
-        decoderSession = environment.createSession(decoderModel, OrtSession.SessionOptions())
-    }
-    fun synthesize(text: String): FloatArray {
-        val tokens = charTokenize(text)
-        val codeTokens = LongArray(1) { 0L }  // BOS
-        // Autoregressive loop
-        val generatedCodes = mutableListOf<Long>()
-        for (i in 0 until maxFrames) {
-            val inputs = mapOf(
-                "text_tokens" to OnnxTensor.createTensor(environment, longArrayOf(1, tokens.size), tokens),
-                "code_tokens" to OnnxTensor.createTensor(environment, longArrayOf(1, codeTokens.size), codeTokens),
-            )
-            val outputs = flowSession.run(inputs)
-            val nextCode = quantize(outputs[0])
-            generatedCodes.add(nextCode)
-            codeTokens = longArrayOf(nextCode)
-            if (isEos(outputs[1])) break
-        }
-        // Decode to audio
-        val decoderInputs = mapOf(
-            "indices" to OnnxTensor.createTensor(environment, longArrayOf(1, generatedCodes.size), generatedCodes.toLongArray())
-        )
-        val audio = decoderSession.run(decoderInputs)[0]
-        return audio.floatBuffer
-    }
-    private fun charTokenize(text: String): LongArray {
-        val chars = "PADBOSabc..."
-        return longArrayOf(1L) + text.map { chars.indexOf(it).toLong() } + longArrayOf(2L)
-    }
-}
-```
-## Performance
-- Pixel 6+: ~300-500ms per verse
-- Peak memory: ~30 MB
-- First-chunk latency: ~100ms
-## Size Optimization
-1. Apply int8 quantization on ONNX models after training
-2. Strip unnecessary ONNX debug info
-```
-- [ ] **Step 2: Commit**
-```bash
-git add docs/android_integration.md
-git commit -m "docs: Android integration guide for ONNX Runtime"
-```
----
-## Self-Review Checklist
-1. **Spec coverage**: Does every spec requirement have a task?
-   - VQ-VAE codec: Tasks 2, 3
-   - Training VQ-VAE: Task 3
-   - Audio encoding: Task 4
-   - Flow transformer: Tasks 5, 6
-   - Flow training: Task 6
-   - Inference pipeline: Task 7
-   - ONNX export: Task 7
-   - Android integration: Task 8
-   - Character tokenizer: Task 1
-2. **Placeholder scan**: No TBD, TODO, or vague steps found.
-3. **Type consistency**:
-   - CharTokenizer.encode() returns torch.Tensor (int64) — used consistently
-   - VQVAE.forward returns (x_hat, indices, vq_loss) — same in training and inference
-   - FlowLMModel.forward returns (next_latent, is_eos) — same everywhere
-4. **Interface consistency**:
-   - `encode_to_indices` returns [B, T] long — used in Task 4
-   - `decode_from_indices` returns [B, 1, T] — used in Task 7
-   - `AudioDataset` yields dict with "audio", "text_tokens", "text", "audio_paths"
-   - `CodecDataset` yields dict with "text_tokens", "codes"

bible-tts/docs/superpowers/specs/2026-07-08-bible-tts-design.md DELETED Viewed

@@ -1,241 +0,0 @@
-# Bible-TTS: Textless TTS for the English Bible
-**Date:** 2026-07-08
-**Status:** Draft
-**Repository:** `bible-tts`
-## Motivation
-Deploy a purpose-built text-to-speech model on Android that reads the English Bible using a specific narrator's voice. The model must be significantly smaller than general-purpose TTS models (pocket-tts at 170MB, Kokoro at 320MB) while maintaining natural speech quality on a single domain (Bible text, ~15K unique words, simple narrative prosody).
-## Design Constraints
-| Constraint | Target |
-|---|---|
-| Model size | < 10 MB (int8) for Android APK |
-| Inference | > 1× realtime on phone CPU |
-| Voice | Fixed narrator from 40 hours of audio |
-| Domain | English Bible text only |
-| Latency | < 500ms startup, streaming capable |
-| Platform | Android via ONNX Runtime Mobile |
-## Data
-| Item | Details |
-|---|---|
-| Audio | 40 hours, single speaker reading the English Bible |
-| Format | 24kHz mono WAV, ~5 GB |
-| Text | Verse-aligned Bible text (31K verses, 725K words) |
-| Alignment | Perfect text-audio pairing per verse (no forced alignment needed) |
-## Architecture: Two-Stage Textless TTS
-### Overview
-```
-Training:
-Raw audio ──→ VQ-VAE Codec ──→ discrete tokens (94Hz)
-Text ──→ Char Embedding ──→ Flow Transformer ──→ predict tokens
-Inference:
-Text → Char Embedding → Flow Transformer → VQ Decoder → Audio waveform
-```
-The pipeline is "textless" because it never uses phonemes, linguistic features, or forced alignment. The VQ-VAE learns to represent the narrator's voice waveform directly, and the flow transformer learns to map text to those learned representations through a flow-matching objective.
----
-### Stage 1: VQ-VAE Audio Codec
-**Purpose:** Compress 24kHz audio into discrete latent tokens at 94 Hz frame rate.
-**Architecture:**
-```
-Audio (T samples) →
-  Conv1D(1→32, k=7, s=1)                    # Initial projection
-  │
-  ├── Conv1D(32→64, k=8, s=4)               # Stride 4
-  │   └── ResBlock(64, k=3) × 2
-  ├── Conv1D(64→128, k=8, s=4)              # Stride 4
-  │   └── ResBlock(128, k=3) × 2
-  ├── Conv1D(128→128, k=8, s=4)             # Stride 4
-  │   └── ResBlock(128, k=3) × 2
-  │
-  └── Conv1D(128→64, k=3, s=1)              # Final projection
-      ↓
-  VectorQuantizer(codebook=512, dim=64)      # 9-bit codes
-      ↓
-  ConvTranspose1d(64→128, k=8, s=4)          # Stride 4
-  ├── ResBlock(128, k=3) × 2
-  ├── ConvTranspose1d(128→64, k=8, s=4)     # Stride 4
-  │   └── ResBlock(64, k=3) × 2
-  ├── ConvTranspose1d(64→32, k=8, s=4)      # Stride 4
-  │   └── ResBlock(32, k=3) × 2
-  │
-  └── Conv1D(32→1, k=7, s=1)                # Audio output
-```
-| Property | Value |
-|---|---|
-| Total stride | 256 (4×4×4) |
-| Frame rate | 24000 / 256 = 93.75 Hz |
-| Codebook | 512 entries × 64 dim |
-| Bits per frame | 9 bits |
-| Runtime tokens (40h) | ~13.5M tokens |
-| Encoder params | ~1.5M |
-| Decoder params | ~1.0M |
-| **Total codec params** | **~2.5M** |
-**ResBlock structure:**
-```
-ELU → Conv1D(dim→dim/2, k=3, d=base^j) → ELU → Conv1D(dim/2→dim, k=3, d=1) → + residual
-```
-**Training losses:**
-- L1 reconstruction loss: ‖x - x̂‖₁
-- Commitment loss: ‖sg[z_e] - e‖₂² + β‖z_e - sg[e]‖₂²  (β=0.25)
-- No perceptual loss (simplifies training, single speaker is forgiving)
-**Training details:**
-- Optimizer: AdamW (lr=3e-4, weight_decay=1e-4)
-- Batch size: 16 (crop to 2-second windows)
-- Steps: 200K (~1-2 days on single GPU)
-- Learning rate: cosine schedule with 5K warmup
----
-### Stage 2: Flow Transformer (Text-to-Codec)
-**Purpose:** Generate VQ codec latent sequences from input text.
-**Architecture:**
-```
-Input text ──→ Character embedding ──→
-  ┌─────────────────────────────────────────────┐
-  │ StreamingTransformer (3 layers)              │
-  │   d_model=256, num_heads=4, FFN=1024        │
-  │   RoPE position encoding                    │
-  │   Pre-norm LayerNorm                        │
-  └─────────────────────────────────────────────┘
-      │
-      ↓
-  AdaLN Flow MLP ──→ Linear(256→64) ──→ Predicted latent (dim=64)
-```
-**Character embedding:**
-- Vocabulary: 81 printable ASCII chars + space + special tokens (BOS, EOS, PAD) = ~88 tokens
-- Embedding dim: 128
-- Params: 88 × 128 = 11,264
-**Streaming Transformer (3 layers):**
-- Per layer: Multihead self-attention (4 heads, d_k=64) + FFN (1024 hidden)
-- Rotary Position Embeddings (RoPE) for position encoding
-- KV caching for streaming inference
-- Params per layer: ~440K
-- Total transformer: ~1.3M
-**Flow MLP (AdaLN):**
-- Conditioned on transformer output, predicts flow direction
-- Hidden: 1024 → 1024 → 1024 (3-layer)
-- Conditioning via adaptive layer norm
-- Params: ~1.6M
-**Output projection:**
-- Linear(256 → 64): ~16K
-**Total Stage 2 params: ~3.0M**
-**Flow matching objective:**
-- Lagrangian Self Distillation (LSD) — same as pocket-tts
-- Training: predict velocity field v_t conditioned on text
-- Inference: sample noise, integrate v_t with 4 steps to get latent
-- Enables non-autoregressive, fast generation
-**Training details:**
-- Optimizer: AdamW (lr=2e-4, weight_decay=1e-4)
-- Batch size: 32 (padded to max 800 tokens)
-- Steps: 300K (~2-3 days on single GPU)
-- Input: text + previous audio latent
-- Loss: flow-matching MSE + EOS prediction (binary cross-entropy)
----
-### Inference Pipeline (Android)
-```
-User text → char tokenize → Flow Transformer (4 LSD steps) → VQ Decoder → Audio
-```
-**Inference model components (ONNX):**
-| Component | Params | float16 | int8 |
-|---|---|---|---|
-| Flow Transformer | 3.0M | 12 MB | 3.0 MB |
-| Flow MLP | 1.6M | 6.4 MB | 1.6 MB |
-| VQ Decoder | 1.0M | 4.0 MB | 1.0 MB |
-| Codebook | 32K | 64 KB | 32 KB |
-| Embedding | 11K | 22 KB | 11 KB |
-| **Total** | **~5.6M** | **~22.5 MB** | **~5.6 MB** |
-**ONNX Runtime Mobile overhead:** ~5 MB per APK
-**Total APK impact:** ~11 MB (int8 model + runtime)
-**Latency budget (phone CPU):**
-- Text tokenization: < 1ms
-- Flow transformer (4 LSD steps): 200-400ms for ~50-word verse
-- VQ decode: 50-100ms
-- Total: ~300-500ms per verse → ~2-3× realtime
----
-### Training Plan
-| Phase | Duration | Output |
-|---|---|---|
-| 1. Prepare data | 1 day | 40h audio → 24kHz WAV, verify text alignment |
-| 2. Train VQ-VAE | 2 days | Codec checkpoint (~2.5M params) |
-| 3. Encode audio | 1 day | 13.5M discrete tokens, codec frozen |
-| 4. Train Flow Transformer | 3 days | Transformer checkpoint (~4.6M params) |
-| 5. Export | 1 day | ONNX model, int8 quantization, validation |
-| **Total** | **~8 days** | **Ready for Android integration** |
-### Voice Cloning / Multi-Voice (Future)
-The VQ-VAE codec captures speaker identity in its codebook and decoder. To support additional voices:
-- Train a multi-speaker codec (add speaker embedding to quantizer)
-- Train condition the flow transformer on a speaker ID
-- Requires multi-speaker training data (~1h per new voice)
----
-## Success Criteria
-| Metric | Target |
-|---|---|
-| Model size (int8) | < 10 MB |
-| Inference speed | > 1× realtime on Pixel 6+ CPU |
-| WER (word error rate) | < 5% on held-out Bible verses |
-| MOS (mean opinion score) | > 3.5 / 5 |
-| Cold start latency | < 1 second |
-| Streaming | First audio chunk within 200ms |
----
-## Open Questions
-1. **Perceptual loss for VQ-VAE**: Should we add a pre-trained feature loss (e.g., from WavLM) for better reconstruction quality, or is L1+commitment sufficient for single-speaker speech?
-2. **EOS detection**: How to handle end-of-verse detection? Use a learned EOS head (like pocket-tts) or assume fixed-length generation?
-3. **Streaming vs. full**: Should inference generate entire verse at once (simpler) or stream chunk-by-chunk (lower latency)?
-4. **SentencePiece vs. character**: Character level (88 tokens) handles rare names naturally. But could a tiny BPE (500 tokens) reduce sequence length and speed up inference?
----
-## Related Work
-- **pocket-tts** (Kyutai): Flow matching + Mimi codec, same textless paradigm. Our codec is domain-specific and smaller.
-- **AudioLM** (Google): Textless approach using semantic + acoustic tokens from w2v-BERT and SoundStream.
-- **VALL-E** (Microsoft): Neural codec language model for TTS. Autoregressive, not suitable for Android CPU.
-- **Matcha-TTS** (Shivam Mehta et al.): Flow matching on mel spectrograms. Simpler but needs separate vocoder.

bible-tts/pyproject.toml DELETED Viewed

@@ -1,28 +0,0 @@
-[project]
-name = "bible-tts"
-version = "0.1.0"
-requires-python = ">= 3.10"
-dependencies = [
-    "torch>=2.5.0",
-    "torchaudio>=2.5.0",
-    "einops>=0.4.0",
-    "scipy>=1.5.0",
-    "soundfile>=0.12.0",
-    "onnxruntime>=1.15.0",
-]
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-[tool.hatch.build.targets.wheel]
-packages = ["src/bible_tts"]
-[tool.ruff]
-line-length = 100
-[dependency-groups]
-dev = ["pytest>=8"]
-[tool.pytest.ini_options]
-testpaths = ["tests"]

bible-tts/scripts/create_manifest.py DELETED Viewed

@@ -1,36 +0,0 @@
-"""Convert en_bsb.txt + audio dir into a JSONL manifest."""
-import json
-from pathlib import Path
-def create_manifest(
-    bible_text_path: str,
-    audio_dir: str,
-    output_path: str,
-    audio_format: str = "wav",
-):
-    """Match verses in bible_text_path to audio files in audio_dir."""
-    with open(bible_text_path) as f:
-        lines = f.readlines()
-    audio_dir = Path(audio_dir)
-    entries = []
-    missing = 0
-    for line in lines:
-        parts = line.strip().split("|")
-        if len(parts) < 7:
-            continue
-        book, chapter, verse = parts[0], parts[1], parts[2]
-        text = parts[6]
-        audio_name = f"{book}_{chapter}_{verse}.{audio_format}"
-        audio_path = audio_dir / audio_name
-        if audio_path.exists():
-            entries.append({"audio": str(audio_path), "text": text})
-        else:
-            missing += 1
-    with open(output_path, "w") as f:
-        for entry in entries:
-            f.write(json.dumps(entry) + "\n")
-    print(f"Created manifest with {len(entries)} entries, {missing} missing audio files")

bible-tts/scripts/encode_audio.py DELETED Viewed

@@ -1,59 +0,0 @@
-"""Encode all audio through trained VQ-VAE to create (text, code_tokens) pairs."""
-import json
-import argparse
-from pathlib import Path
-import torch
-from torch.utils.data import DataLoader
-from bible_tts.data.audio_dataset import AudioDataset, collate_audio
-from bible_tts.models.vq_vae import VQVAE
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--manifest", type=str, required=True)
-    parser.add_argument("--checkpoint", type=str, required=True)
-    parser.add_argument("--output", type=str, required=True)
-    parser.add_argument("--batch-size", type=int, default=32)
-    args = parser.parse_args()
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    model = VQVAE().to(device)
-    state = torch.load(args.checkpoint, map_location=device)
-    model.load_state_dict(state["model"])
-    model.eval()
-    dataset = AudioDataset(args.manifest)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=False,
-        num_workers=4,
-        collate_fn=collate_audio,
-    )
-    output_path = Path(args.output)
-    output_path.parent.mkdir(parents=True, exist_ok=True)
-    with open(output_path, "w") as f:
-        with torch.no_grad():
-            for batch in loader:
-                audio = batch["audio"].unsqueeze(1).to(device)
-                indices = model.encode_to_indices(audio)
-                for i in range(len(batch["text"])):
-                    tokens = indices[i].tolist()
-                    entry = {
-                        "text": batch["text"][i],
-                        "codes": tokens,
-                        "audio_path": batch["audio_paths"][i],
-                    }
-                    f.write(json.dumps(entry) + "\n")
-    print(f"Encoded {len(dataset)} examples to {args.output}")
-if __name__ == "__main__":
-    main()

bible-tts/scripts/split_chapters_to_verses.py DELETED Viewed

@@ -1,247 +0,0 @@
-"""Split chapter-level audio into verse-level WAV files using forced alignment.
-Requirements:
-    pip install montreal-forced-aligner
-Usage:
-    python3 scripts/split_chapters_to_verses.py \
-        --bible-text en_bsb.txt \
-        --audio-dir /path/to/chapter/audio \
-        --output-dir /path/to/verse/audio
-Input expects: /path/to/chapter/audio/{Book}_{Chapter}.wav
-Output:         /path/to/verse/audio/{Book}_{Chapter}_{Verse}.wav
-"""
-import argparse
-import json
-import os
-import subprocess
-import tempfile
-from pathlib import Path
-import torch
-import torchaudio
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--bible-text", required=True)
-    parser.add_argument("--audio-dir", required=True)
-    parser.add_argument("--output-dir", required=True)
-    parser.add_argument("--temp-dir", default="/tmp/bible_alignment")
-    parser.add_argument("--skip-mfa", action="store_true", help="Skip MFA step if already aligned")
-    return parser.parse_args()
-def build_chapter_texts(bible_path: str) -> dict:
-    """Group verses by chapter, return {(book, chapter): [verse_text, ...]}"""
-    chapters = {}
-    with open(bible_path) as f:
-        for line in f:
-            parts = line.strip().split("|")
-            if len(parts) < 7:
-                continue
-            book, chapter, verse = parts[0], parts[1], parts[2]
-            text = parts[6]
-            key = (book, chapter)
-            if key not in chapters:
-                chapters[key] = []
-            chapters[key].append((int(verse), text))
-    # Sort by verse number
-    for k in chapters:
-        chapters[k].sort(key=lambda x: x[0])
-    return chapters
-def prepare_mfa_input(chapters: dict, audio_dir: Path, temp_dir: Path):
-    """Create MFA input: one text file per chapter with verse markers."""
-    text_dir = temp_dir / "texts"
-    text_dir.mkdir(parents=True, exist_ok=True)
-    chapter_files = []
-    for (book, chapter), verses in chapters.items():
-        audio_path = audio_dir / f"{book}_{chapter}.wav"
-        if not audio_path.exists():
-            print(f"  Skipping {book} {chapter}: no audio at {audio_path}")
-            continue
-        # Write text with verse markers: "In the beginning {verse_1} God created {verse_2} ..."
-        text_lines = []
-        for vnum, vtext in verses:
-            text_lines.append(vtext)
-            text_lines.append(f"{{verse_{vnum}}}")
-        full_text = " ".join(text_lines)
-        txt_path = text_dir / f"{book}_{chapter}.txt"
-        with open(txt_path, "w") as f:
-            f.write(full_text)
-        chapter_files.append((book, chapter, audio_path, txt_path))
-    return chapter_files
-def run_mfa(chapter_files: list, temp_dir: Path, audio_dir: Path):
-    """Run Montreal Forced Aligner to get word-level alignments."""
-    align_dir = temp_dir / "aligned"
-    corpus_dir = temp_dir / "corpus"
-    corpus_dir.mkdir(parents=True, exist_ok=True)
-    # Copy chapter audio and text into MFA corpus structure
-    for book, chapter, audio_path, txt_path in chapter_files:
-        safe_name = f"{book}_{chapter}".replace(" ", "_")
-        dst_audio = corpus_dir / f"{safe_name}.wav"
-        dst_text = corpus_dir / f"{safe_name}.txt"
-        if not dst_audio.exists():
-            os.symlink(audio_path, dst_audio)
-        if not dst_text.exists():
-            os.symlink(txt_path, dst_text)
-    # Try aligning with pretrained English model
-    cmds = [
-        ["mfa", "validate", str(corpus_dir), "english_us_arpa", "--single_speaker"],
-        ["mfa", "align", str(corpus_dir), "english_us_arpa", "english_us_arpa", str(align_dir), "--single_speaker", "--clean"],
-    ]
-    for cmd in cmds:
-        what = cmd[1]
-        print(f"Running: mfa {what}...")
-        result = subprocess.run(cmd, capture_output=True, text=True)
-        if result.returncode != 0:
-            if "download" in result.stderr.lower():
-                # Try downloading pretrained models first
-                subprocess.run(["mfa", "model", "download", "dictionary", "english_us_arpa"], check=True)
-                subprocess.run(["mfa", "model", "download", "acoustic", "english_us_arpa"], check=True)
-                result = subprocess.run(cmd, capture_output=True, text=True)
-            if result.returncode != 0:
-                print(f"MFA failed on {what}:")
-                print(result.stderr[:500])
-                raise RuntimeError(f"MFA {what} failed")
-    return align_dir
-def parse_textgrid(textgrid_path: Path, verses: list) -> list:
-    """Parse MFA output TextGrid to find verse boundaries from markers.
-    MFA aligns words and markers like {verse_1} appear as word intervals.
-    We use the marker positions to slice the audio.
-    """
-    text = textgrid_path.read_text()
-    lines = text.splitlines()
-    # Find the words/item tier
-    in_words_tier = False
-    verse_boundaries = []
-    for line in lines:
-        stripped = line.strip()
-        if stripped.startswith('class = "Tier"') or stripped.startswith('class = "IntervalTier"'):
-            # Check if next lines name contains "word"
-            continue
-        if '"word"' in stripped.lower() or stripped.strip('"').lower().startswith('word'):
-            in_words_tier = True
-            continue
-        if stripped.startswith("intervals ["):
-            in_words_tier = True
-        if not in_words_tier:
-            continue
-        # Parse intervals: "xmin = 0.123", "xmax = 4.567", "text = "{verse_5}""
-        if stripped.startswith("xmin"):
-            xmin = float(stripped.split("=")[1].strip())
-        elif stripped.startswith("xmax"):
-            xmax = float(stripped.split("=")[1].strip())
-        elif stripped.startswith("text"):
-            label = stripped.split("=", 1)[1].strip().strip('"').strip()
-            if label.startswith("{verse_") and label.endswith("}"):
-                vnum = int(label[7:-1])
-                verse_boundaries.append((vnum, xmin, xmax))
-    return verse_boundaries
-def split_audio(
-    chapter_files: list,
-    chapters: dict,
-    align_dir: Path,
-    output_dir: Path,
-    sample_rate: int = 24000,
-):
-    """Split chapter audio at verse boundaries and write verse WAVs."""
-    output_dir = Path(output_dir)
-    output_dir.mkdir(parents=True, exist_ok=True)
-    manifest = []
-    for book, chapter, audio_path, _ in chapter_files:
-        textgrid_path = align_dir / f"{book}_{chapter}.TextGrid"
-        if not textgrid_path.exists():
-            print(f"  No alignment for {book} {chapter}, skipping")
-            continue
-        audio, sr = torchaudio.load(str(audio_path))
-        if sr != sample_rate:
-            audio = torchaudio.functional.resample(audio, sr, sample_rate)
-        verses = chapters[(book, chapter)]
-        boundaries = parse_textgrid(textgrid_path, verses)
-        boundaries.sort(key=lambda x: x[0])
-        for i, (vnum, start, end) in enumerate(boundaries):
-            start_sample = int(start * sample_rate)
-            end_sample = int(end * sample_rate)
-            if end_sample <= start_sample:
-                continue
-            verse_audio = audio[:, start_sample:end_sample]
-            out_name = f"{book}_{chapter}_{vnum}.wav"
-            out_path = output_dir / out_name
-            torchaudio.save(str(out_path), verse_audio, sample_rate)
-            manifest.append({
-                "audio": str(out_path.absolute()),
-                "text": verses[i][1],
-            })
-        print(f"  {book} {chapter}: {len(boundaries)} verses")
-    return manifest
-def main():
-    args = parse_args()
-    temp_dir = Path(args.temp_dir)
-    audio_dir = Path(args.audio_dir)
-    output_dir = Path(args.output_dir)
-    print("Building chapter texts from Bible file...")
-    chapters = build_chapter_texts(args.bible_text)
-    print(f"  Found {len(chapters)} chapters")
-    print("Preparing MFA input...")
-    chapter_files = prepare_mfa_input(chapters, audio_dir, temp_dir)
-    print(f"  {len(chapter_files)} chapters with audio")
-    if not args.skip_mfa:
-        align_dir = run_mfa(chapter_files, temp_dir, audio_dir)
-    else:
-        align_dir = temp_dir / "aligned"
-    print("Splitting audio at verse boundaries...")
-    manifest = split_audio(chapter_files, chapters, align_dir, output_dir)
-    # Write manifest
-    manifest_path = output_dir / "manifest.jsonl"
-    with open(manifest_path, "w") as f:
-        for entry in manifest:
-            f.write(json.dumps(entry) + "\n")
-    print(f"\nDone! {len(manifest)} verses extracted.")
-    print(f"Manifest: {manifest_path}")
-    print(f"Audio:    {output_dir}/")
-    print(f"\nTotal size: {sum(os.path.getsize(e['audio']) for e in manifest) / 1e9:.1f} GB")
-if __name__ == "__main__":
-    main()

bible-tts/src/bible_tts/__init__.py DELETED Viewed

File without changes

bible-tts/src/bible_tts/data/__init__.py DELETED Viewed

File without changes

bible-tts/src/bible_tts/data/audio_dataset.py DELETED Viewed

@@ -1,81 +0,0 @@
-import json
-from pathlib import Path
-import torch
-import torchaudio
-from torch.utils.data import Dataset
-from bible_tts.data.text_processor import CharTokenizer
-class AudioDataset(Dataset):
-    """Loads verse-aligned (audio, text) pairs.
-    Manifest format: JSONL where each line has:
-      {"audio": "path/to/verse.wav", "text": "In the beginning..."}
-    Audio is resampled to target_sr if needed.
-    """
-    def __init__(
-        self,
-        manifest_path: str | Path,
-        target_sr: int = 24000,
-        max_audio_seconds: float = 30.0,
-    ):
-        self.manifest_path = Path(manifest_path)
-        self.target_sr = target_sr
-        self.max_samples = int(max_audio_seconds * target_sr)
-        self.tokenizer = CharTokenizer()
-        with open(self.manifest_path) as f:
-            self.entries = [json.loads(line) for line in f if line.strip()]
-    def __len__(self) -> int:
-        return len(self.entries)
-    def __getitem__(self, idx: int) -> dict:
-        entry = self.entries[idx]
-        audio_path = Path(entry["audio"])
-        text = entry["text"]
-        waveform, sr = torchaudio.load(str(audio_path))
-        if sr != self.target_sr:
-            waveform = torchaudio.functional.resample(waveform, sr, self.target_sr)
-        if waveform.shape[-1] > self.max_samples:
-            waveform = waveform[..., : self.max_samples]
-        text_tokens = self.tokenizer.encode(text)
-        return {
-            "audio": waveform.squeeze(0),  # [T]
-            "text_tokens": text_tokens,  # [S]
-            "text": text,
-            "audio_path": str(audio_path),
-        }
-def collate_audio(batch: list[dict]) -> dict:
-    """Pads audio and text tokens to max length in batch."""
-    max_audio_len = max(item["audio"].shape[-1] for item in batch)
-    max_text_len = max(item["text_tokens"].shape[-1] for item in batch)
-    audio_padded = torch.zeros(len(batch), max_audio_len)
-    text_padded = torch.zeros(len(batch), max_text_len, dtype=torch.long)
-    texts = []
-    paths = []
-    for i, item in enumerate(batch):
-        audio_len = item["audio"].shape[-1]
-        text_len = item["text_tokens"].shape[-1]
-        audio_padded[i, :audio_len] = item["audio"]
-        text_padded[i, :text_len] = item["text_tokens"]
-        texts.append(item["text"])
-        paths.append(item["audio_path"])
-    return {
-        "audio": audio_padded,
-        "text_tokens": text_padded,
-        "text": texts,
-        "audio_paths": paths,
-    }

bible-tts/src/bible_tts/data/text_processor.py DELETED Viewed

@@ -1,41 +0,0 @@
-import torch
-PAD_TOKEN = "<PAD>"
-BOS_TOKEN = "<BOS>"
-EOS_TOKEN = "<EOS>"
-CHAR_SET = sorted(set(
-    "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
-    ".,!?;:-\"'()[]{}@#$%&*+=/\\<>~`| "
-))
-PAD_IDX = 0
-BOS_IDX = 1
-EOS_IDX = 2
-SPECIAL = {PAD_TOKEN, BOS_TOKEN, EOS_TOKEN}
-class CharTokenizer:
-    def __init__(self):
-        self.chars = [PAD_TOKEN, BOS_TOKEN, EOS_TOKEN] + CHAR_SET
-        self.char_to_idx = {c: i for i, c in enumerate(self.chars)}
-        self.idx_to_char = {i: c for i, c in enumerate(self.chars)}
-    @property
-    def vocab_size(self) -> int:
-        return len(self.chars)
-    def encode(self, text: str) -> torch.Tensor:
-        tokens = [BOS_IDX]
-        for c in text:
-            tokens.append(self.char_to_idx.get(c, PAD_IDX))
-        tokens.append(EOS_IDX)
-        return torch.tensor(tokens, dtype=torch.long)
-    def decode(self, tokens: torch.Tensor) -> str:
-        chars = []
-        for t in tokens.tolist():
-            c = self.idx_to_char.get(t, "")
-            if c in SPECIAL:
-                continue
-            chars.append(c)
-        return "".join(chars)

bible-tts/src/bible_tts/inference/__init__.py DELETED Viewed

@@ -1,3 +0,0 @@
-from bible_tts.inference.pipeline import TTSPipeline
-__all__ = ["TTSPipeline"]

bible-tts/src/bible_tts/inference/export_onnx.py DELETED Viewed

@@ -1,96 +0,0 @@
-import torch
-import onnxruntime
-from bible_tts.models.flow_lm import FlowLMModel
-from bible_tts.models.vq_vae import VQVAE
-def export_flow_lm_onnx(checkpoint_path: str, output_path: str):
-    model = FlowLMModel()
-    state = torch.load(checkpoint_path, map_location="cpu")
-    model.load_state_dict(state["model"])
-    model.eval()
-    class FlowLMInferenceWrapper(torch.nn.Module):
-        def __init__(self, flow_model):
-            super().__init__()
-            self.model = flow_model
-        def forward(self, text_tokens, code_tokens, temp=1.0, noise_clamp=0.0, eos_threshold=0.5):
-            next_latent, is_eos = self.model(
-                text_tokens=text_tokens,
-                code_tokens=code_tokens,
-                lsd_steps=4,
-                temp=temp,
-                noise_clamp=noise_clamp if noise_clamp > 0 else None,
-                eos_threshold=eos_threshold,
-            )
-            return next_latent, is_eos
-    wrapper = FlowLMInferenceWrapper(model)
-    wrapper.eval()
-    text_tokens = torch.randint(0, 88, (1, 50), dtype=torch.long)
-    code_tokens = torch.randint(0, 512, (1, 1), dtype=torch.long)
-    torch.onnx.export(
-        wrapper,
-        (text_tokens, code_tokens),
-        output_path,
-        input_names=["text_tokens", "code_tokens"],
-        output_names=["next_latent", "is_eos"],
-        dynamic_axes={
-            "text_tokens": {1: "text_length"},
-            "code_tokens": {1: "code_length"},
-        },
-        opset_version=17,
-    )
-    print(f"FlowLM ONNX model exported to {output_path}")
-def export_vqvae_decoder_onnx(checkpoint_path: str, output_path: str):
-    model = VQVAE()
-    state = torch.load(checkpoint_path, map_location="cpu")
-    model.load_state_dict(state["model"])
-    model.eval()
-    class VQDecoderWrapper(torch.nn.Module):
-        def __init__(self, codebook, decoder):
-            super().__init__()
-            self.codebook = codebook
-            self.decoder = decoder
-        def forward(self, indices: torch.Tensor) -> torch.Tensor:
-            z_q = self.codebook(indices)
-            z_q = z_q.permute(0, 2, 1)
-            return self.decoder(z_q)
-    wrapper = VQDecoderWrapper(model.quantizer.codebook, model.decoder)
-    wrapper.eval()
-    indices = torch.randint(0, 512, (1, 375), dtype=torch.long)
-    torch.onnx.export(
-        wrapper,
-        indices,
-        output_path,
-        input_names=["indices"],
-        output_names=["audio"],
-        dynamic_axes={
-            "indices": {1: "num_frames"},
-        },
-        opset_version=17,
-    )
-    print(f"VQ Decoder ONNX model exported to {output_path}")
-def validate_onnx(onnx_path: str, sample_inputs: list):
-    session = onnxruntime.InferenceSession(onnx_path)
-    input_names = [inp.name for inp in session.get_inputs()]
-    output_names = [out.name for out in session.get_outputs()]
-    feed = dict(zip(input_names, sample_inputs))
-    outputs = session.run(output_names, feed)
-    print(f"ONNX validation passed for {onnx_path}")
-    return outputs

bible-tts/src/bible_tts/inference/pipeline.py DELETED Viewed

@@ -1,87 +0,0 @@
-import torch
-from bible_tts.data.text_processor import CharTokenizer
-from bible_tts.models.vq_vae import VQVAE
-from bible_tts.models.flow_lm import FlowLMModel
-class TTSPipeline:
-    """Full text-to-speech inference pipeline.
-    Text -> CharTokenizer -> FlowLM -> latent codes -> VQ Decoder -> audio
-    """
-    def __init__(
-        self,
-        vqvae_checkpoint: str,
-        flow_checkpoint: str,
-        device: str = "cpu",
-        lsd_steps: int = 4,
-    ):
-        self.device = torch.device(device)
-        self.lsd_steps = lsd_steps
-        self.tokenizer = CharTokenizer()
-        self.vqvae = VQVAE().to(self.device)
-        vq_state = torch.load(vqvae_checkpoint, map_location=self.device)
-        self.vqvae.load_state_dict(vq_state["model"])
-        self.vqvae.eval()
-        self.flow = FlowLMModel().to(self.device)
-        flow_state = torch.load(flow_checkpoint, map_location=self.device)
-        self.flow.load_state_dict(flow_state["model"])
-        self.flow.eval()
-    @torch.no_grad()
-    def generate(self, text: str) -> torch.Tensor:
-        text_tokens = self.tokenizer.encode(text).unsqueeze(0).to(self.device)
-        max_frames = int(300)
-        code_tokens = torch.zeros((1, 0), dtype=torch.long, device=self.device)
-        for step in range(max_frames):
-            if step == 0:
-                current_codes = torch.zeros((1, 1), dtype=torch.long, device=self.device)
-            else:
-                current_codes = code_tokens[:, -1:]
-            next_latent, is_eos = self.flow(
-                text_tokens=text_tokens,
-                code_tokens=current_codes,
-                lsd_steps=self.lsd_steps,
-            )
-            dist = torch.cdist(
-                next_latent.unsqueeze(1),
-                self.flow.code_embed.weight.unsqueeze(0),
-            )
-            next_code = dist.argmin(dim=-1)
-            code_tokens = torch.cat([code_tokens, next_code], dim=-1)
-            if is_eos.item() and step > 10:
-                break
-        audio = self.vqvae.decode_from_indices(code_tokens)
-        return audio
-    @torch.no_grad()
-    def generate_fast(self, text: str) -> torch.Tensor:
-        text_tokens = self.tokenizer.encode(text).unsqueeze(0).to(self.device)
-        text_emb = self.flow.char_embed(text_tokens)
-        text_emb = self.flow.text_proj(text_emb)
-        cond = text_emb[:, -1]
-        noise = torch.randn(1, self.flow.latent_dim, device=self.device)
-        def flow_fn(t_prev, t_curr, x_t):
-            return self.flow.flow_mlp(cond, t_prev, t_curr, x_t)
-        from bible_tts.models.lsd import lsd_decode
-        generated = lsd_decode(flow_fn, noise, self.lsd_steps)
-        dist = torch.cdist(
-            generated.unsqueeze(1),
-            self.flow.code_embed.weight.unsqueeze(0),
-        )
-        code = dist.argmin(dim=-1)
-        audio = self.vqvae.decode_from_indices(code)
-        return audio

bible-tts/src/bible_tts/models/__init__.py DELETED Viewed

File without changes

bible-tts/src/bible_tts/models/flow_lm.py DELETED Viewed

@@ -1,221 +0,0 @@
-import torch
-from torch import nn
-import einops
-from bible_tts.models.lsd import lsd_decode
-class StreamingMultiheadAttention(nn.Module):
-    def __init__(self, d_model: int, num_heads: int, max_period: float = 10000.0):
-        super().__init__()
-        self.d_model = d_model
-        self.num_heads = num_heads
-        self.head_dim = d_model // num_heads
-        self.max_period = max_period
-        self.wq = nn.Linear(d_model, d_model, bias=False)
-        self.wk = nn.Linear(d_model, d_model, bias=False)
-        self.wv = nn.Linear(d_model, d_model, bias=False)
-        self.wo = nn.Linear(d_model, d_model, bias=False)
-    def _rope(self, x: torch.Tensor, offset: int = 0) -> torch.Tensor:
-        """Apply rotary position embeddings."""
-        B, T, H, D = x.shape
-        positions = torch.arange(offset, offset + T, device=x.device, dtype=torch.float32)
-        freqs = torch.exp(
-            -torch.arange(0, D, 2, device=x.device, dtype=torch.float32)
-            * (torch.log(torch.tensor(self.max_period)) / D)
-        )
-        angles = positions[:, None] * freqs[None, :]
-        cos, sin = angles.cos()[:, None, :], angles.sin()[:, None, :]
-        x_rot = x.clone()
-        x_rot[..., 0::2] = x[..., 0::2] * cos - x[..., 1::2] * sin
-        x_rot[..., 1::2] = x[..., 1::2] * cos + x[..., 0::2] * sin
-        return x_rot
-    def forward(self, x: torch.Tensor, kv_cache: torch.Tensor | None = None) -> tuple[torch.Tensor, torch.Tensor]:
-        B, T, _ = x.shape
-        q = einops.rearrange(self.wq(x), "B T (H D) -> B T H D", H=self.num_heads)
-        k = einops.rearrange(self.wk(x), "B T (H D) -> B T H D", H=self.num_heads)
-        v = einops.rearrange(self.wv(x), "B T (H D) -> B T H D", H=self.num_heads)
-        q = self._rope(q)
-        k = self._rope(k)
-        if kv_cache is not None:
-            k = torch.cat([kv_cache[0], k], dim=1)
-            v = torch.cat([kv_cache[1], v], dim=1)
-        attn = torch.einsum("bthd,bThd->bhtT", q, k) / (self.head_dim ** 0.5)
-        attn = torch.softmax(attn, dim=-1)
-        out = torch.einsum("bhtT,bThd->bthd", attn, v)
-        out = einops.rearrange(out, "B T H D -> B T (H D)")
-        out = self.wo(out)
-        new_cache = torch.stack([k, v], dim=0)
-        return out, new_cache
-class StreamingTransformerLayer(nn.Module):
-    def __init__(self, d_model: int, num_heads: int, ff_hidden: int, max_period: float = 10000.0):
-        super().__init__()
-        self.attention = StreamingMultiheadAttention(d_model, num_heads, max_period)
-        self.norm1 = nn.LayerNorm(d_model)
-        self.ffn = nn.Sequential(
-            nn.Linear(d_model, ff_hidden),
-            nn.GELU(),
-            nn.Linear(ff_hidden, d_model),
-        )
-        self.norm2 = nn.LayerNorm(d_model)
-    def forward(self, x: torch.Tensor, kv_cache: torch.Tensor | None = None) -> tuple[torch.Tensor, torch.Tensor]:
-        attn_out, new_cache = self.attention(self.norm1(x), kv_cache)
-        x = x + attn_out
-        x = x + self.ffn(self.norm2(x))
-        return x, new_cache
-class StreamingTransformer(nn.Module):
-    def __init__(self, d_model: int, num_layers: int, num_heads: int, ff_hidden: int):
-        super().__init__()
-        self.layers = nn.ModuleList([
-            StreamingTransformerLayer(d_model, num_heads, ff_hidden)
-            for _ in range(num_layers)
-        ])
-    def forward(self, x: torch.Tensor, kv_caches: list[torch.Tensor] | None = None) -> tuple[torch.Tensor, list[torch.Tensor]]:
-        new_caches = []
-        for i, layer in enumerate(self.layers):
-            cache = kv_caches[i] if kv_caches is not None else None
-            x, new_cache = layer(x, cache)
-            new_caches.append(new_cache)
-        return x, new_caches
-class FlowMLP(nn.Module):
-    """AdaLN-conditioned MLP for flow prediction.
-    Takes conditioning vector c and input x_t, predicts flow direction.
-    """
-    def __init__(self, d_model: int, latent_dim: int, hidden: int = 1024):
-        super().__init__()
-        self.cond_proj = nn.Linear(d_model, latent_dim * 2)  # scale + shift
-        self.net = nn.Sequential(
-            nn.Linear(latent_dim + 2, hidden),  # +2 for t_prev, t_curr
-            nn.GELU(),
-            nn.Linear(hidden, hidden),
-            nn.GELU(),
-            nn.Linear(hidden, latent_dim),
-        )
-    def forward(self, c: torch.Tensor, t_prev: torch.Tensor, t_curr: torch.Tensor, x_t: torch.Tensor) -> torch.Tensor:
-        scale, shift = self.cond_proj(c).chunk(2, dim=-1)
-        cond = torch.cat([t_prev, t_curr], dim=-1)
-        inp = torch.cat([cond, x_t], dim=-1)
-        h = self.net(inp)
-        return h * (1 + scale) + shift
-class FlowLMModel(nn.Module):
-    def __init__(
-        self,
-        vocab_size: int = 88,
-        d_embed: int = 128,
-        d_model: int = 256,
-        latent_dim: int = 64,
-        num_layers: int = 3,
-        num_heads: int = 4,
-        ff_hidden: int = 1024,
-        mlp_hidden: int = 1024,
-        codebook_size: int = 512,
-    ):
-        super().__init__()
-        self.latent_dim = latent_dim
-        self.d_model = d_model
-        self.char_embed = nn.Embedding(vocab_size, d_embed)
-        self.text_proj = nn.Linear(d_embed, d_model)
-        self.code_embed = nn.Embedding(codebook_size, latent_dim)
-        self.input_proj = nn.Linear(latent_dim, d_model)
-        self.transformer = StreamingTransformer(d_model, num_layers, num_heads, ff_hidden)
-        self.out_norm = nn.LayerNorm(d_model)
-        self.flow_mlp = FlowMLP(d_model, latent_dim, mlp_hidden)
-        self.eos_head = nn.Linear(d_model, 1)
-        self.bos_code = nn.Parameter(torch.randn(latent_dim))
-    def forward(
-        self,
-        text_tokens: torch.Tensor,
-        code_tokens: torch.Tensor,
-        kv_caches: list[torch.Tensor] | None = None,
-        lsd_steps: int = 4,
-        temp: float = 1.0,
-        noise_clamp: float | None = None,
-        eos_threshold: float = 0.5,
-    ) -> tuple[torch.Tensor, torch.Tensor]:
-        """
-        Args:
-            text_tokens: [B, S] character token ids
-            code_tokens: [B, T] codec token ids (previous audio latents)
-            kv_caches: optional KV caches for streaming
-            lsd_steps: number of LSD steps (0 = training, >0 = inference)
-        Returns:
-            next_latent: [B, latent_dim] predicted latent
-            is_eos: [B, 1] EOS prediction
-        """
-        B = text_tokens.shape[0]
-        text_emb = self.char_embed(text_tokens)
-        text_emb = self.text_proj(text_emb)
-        code_emb = self.code_embed(code_tokens)
-        code_emb = self.input_proj(code_emb)
-        combined = torch.cat([text_emb, code_emb], dim=1)
-        out, new_caches = self.transformer(combined, kv_caches)
-        out = self.out_norm(out)
-        last_out = out[:, -1]
-        is_eos = (self.eos_head(last_out) > eos_threshold).float()
-        if lsd_steps == 0:
-            return last_out, self.eos_head(last_out)  # return logits, not thresholded
-        noise_shape = (B, self.latent_dim)
-        noise = torch.randn(noise_shape, device=text_tokens.device)
-        if noise_clamp is not None:
-            noise = noise.clamp(-noise_clamp, noise_clamp)
-        noise = noise * temp
-        def flow_fn(t_prev, t_curr, x_t):
-            return self.flow_mlp(last_out, t_prev, t_curr, x_t)
-        next_latent = lsd_decode(flow_fn, noise, lsd_steps)
-        return next_latent, is_eos
-    def sample_latent(
-        self,
-        text_tokens: torch.Tensor,
-        code_tokens: torch.Tensor,
-        kv_caches: list[torch.Tensor] | None = None,
-        lsd_steps: int = 4,
-        temp: float = 1.0,
-        noise_clamp: float | None = None,
-        eos_threshold: float = 0.5,
-    ) -> tuple[torch.Tensor, torch.Tensor, list[torch.Tensor]]:
-        next_latent, is_eos = self(
-            text_tokens=text_tokens,
-            code_tokens=code_tokens,
-            kv_caches=kv_caches,
-            lsd_steps=lsd_steps,
-            temp=temp,
-            noise_clamp=noise_clamp,
-            eos_threshold=eos_threshold,
-        )
-        return next_latent, is_eos

bible-tts/src/bible_tts/models/lsd.py DELETED Viewed

@@ -1,26 +0,0 @@
-import torch
-def lsd_decode(
-    v_t: callable,
-    x_0: torch.Tensor,
-    num_steps: int = 4,
-) -> torch.Tensor:
-    """Lagrangian Self Distillation.
-    Args:
-        v_t: function (t_prev, t_curr, x_t) -> velocity
-        x_0: starting noise [B, D]
-        num_steps: number of integration steps
-    """
-    current = x_0
-    for i in range(num_steps):
-        s = i / num_steps
-        t = (i + 1) / num_steps
-        flow_dir = v_t(
-            s * torch.ones_like(current[..., :1]),
-            t * torch.ones_like(current[..., :1]),
-            current,
-        )
-        current = current + flow_dir / num_steps
-    return current

bible-tts/src/bible_tts/models/vq_vae.py DELETED Viewed

@@ -1,174 +0,0 @@
-import torch
-from torch import nn
-from torch.nn import functional as F
-class ResBlock(nn.Module):
-    def __init__(self, dim: int, kernel_size: int = 3, dilation: int = 1, compress: int = 2):
-        super().__init__()
-        hidden = dim // compress
-        self.conv1 = nn.Conv1d(dim, hidden, kernel_size, padding="same", dilation=dilation)
-        self.conv2 = nn.Conv1d(hidden, dim, kernel_size, padding="same")
-        self.activation = nn.ELU()
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        residual = x
-        x = self.activation(x)
-        x = self.conv1(x)
-        x = self.activation(x)
-        x = self.conv2(x)
-        return x + residual
-class Encoder(nn.Module):
-    """Strided conv encoder: downsamples audio to latent codes.
-    Args:
-        in_channels: 1 for mono audio
-        base_channels: starting channel count (doubles per stride)
-        latent_dim: output latent dimension
-        strides: list of stride factors per layer
-        n_resblocks: number of ResBlocks per level
-    """
-    def __init__(
-        self,
-        in_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-    ):
-        super().__init__()
-        self.total_stride = 1
-        for s in strides:
-            self.total_stride *= s
-        layers: list[nn.Module] = [
-            nn.Conv1d(in_channels, base_channels, kernel_size=7, padding="same")
-        ]
-        mult = 1
-        for stride in strides:
-            for _ in range(n_resblocks):
-                layers.append(ResBlock(mult * base_channels))
-            layers.append(nn.ELU())
-            layers.append(
-                nn.Conv1d(
-                    mult * base_channels,
-                    mult * base_channels * 2,
-                    kernel_size=2 * stride,
-                    stride=stride,
-                    padding=stride // 2,
-                )
-            )
-            mult *= 2
-        layers.append(nn.Conv1d(mult * base_channels, latent_dim, kernel_size=3, padding="same"))
-        self.layers = nn.Sequential(*layers)
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        return self.layers(x)
-class VectorQuantizer(nn.Module):
-    def __init__(self, codebook_size: int = 512, dim: int = 64, beta: float = 0.25):
-        super().__init__()
-        self.codebook_size = codebook_size
-        self.dim = dim
-        self.beta = beta
-        self.codebook = nn.Embedding(codebook_size, dim)
-        nn.init.uniform_(self.codebook.weight, -1.0 / codebook_size, 1.0 / codebook_size)
-    def forward(self, z: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
-        # z: [B, D, T]
-        z_perm = z.permute(0, 2, 1)  # [B, T, D]
-        B, T, D = z_perm.shape
-        flat = z_perm.reshape(-1, D)  # [B*T, D]
-        # distances: [B*T, codebook_size]
-        dist = torch.cdist(flat, self.codebook.weight)
-        indices = dist.argmin(dim=-1)  # [B*T]
-        # quantize
-        z_q = self.codebook(indices)  # [B*T, D]
-        z_q = z_q.reshape(B, T, D).permute(0, 2, 1)  # [B, D, T]
-        # commitment loss
-        loss = F.mse_loss(z_q.detach(), z) + self.beta * F.mse_loss(z_q, z.detach())
-        # straight-through estimator
-        z_q = z + (z_q - z).detach()
-        return z_q, indices.reshape(B, T), loss
-class Decoder(nn.Module):
-    """Transposed conv decoder: upsamples latent codes to audio."""
-    def __init__(
-        self,
-        out_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-    ):
-        super().__init__()
-        mult = 2 ** len(strides)
-        layers: list[nn.Module] = [
-            nn.Conv1d(latent_dim, mult * base_channels, kernel_size=3, padding="same")
-        ]
-        for stride in strides:
-            layers.append(nn.ELU())
-            layers.append(
-                nn.ConvTranspose1d(
-                    mult * base_channels,
-                    mult * base_channels // 2,
-                    kernel_size=2 * stride,
-                    stride=stride,
-                    padding=stride // 2,
-                )
-            )
-            for _ in range(n_resblocks):
-                layers.append(ResBlock(mult * base_channels // 2))
-            mult //= 2
-        layers.append(nn.Conv1d(base_channels, out_channels, kernel_size=7, padding="same"))
-        self.layers = nn.Sequential(*layers)
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        return self.layers(x)
-class VQVAE(nn.Module):
-    def __init__(
-        self,
-        in_channels: int = 1,
-        base_channels: int = 32,
-        latent_dim: int = 64,
-        codebook_size: int = 512,
-        strides: list[int] = [4, 4, 4],
-        n_resblocks: int = 2,
-        beta: float = 0.25,
-    ):
-        super().__init__()
-        self.encoder = Encoder(in_channels, base_channels, latent_dim, strides, n_resblocks)
-        self.quantizer = VectorQuantizer(codebook_size, latent_dim, beta)
-        self.decoder = Decoder(in_channels, base_channels, latent_dim, strides, n_resblocks)
-    @property
-    def frame_rate(self) -> float:
-        return 24000 / self.encoder.total_stride
-    def forward(self, x: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
-        z_e = self.encoder(x)
-        z_q, indices, vq_loss = self.quantizer(z_e)
-        x_hat = self.decoder(z_q)
-        return x_hat, indices, vq_loss
-    def encode_to_indices(self, x: torch.Tensor) -> torch.Tensor:
-        z_e = self.encoder(x)
-        _, indices, _ = self.quantizer(z_e)
-        return indices
-    def decode_from_indices(self, indices: torch.Tensor) -> torch.Tensor:
-        z_q = self.quantizer.codebook(indices)
-        z_q = z_q.permute(0, 2, 1)
-        return self.decoder(z_q)

bible-tts/src/bible_tts/training/__init__.py DELETED Viewed

File without changes

bible-tts/src/bible_tts/training/train_flow.py DELETED Viewed

@@ -1,209 +0,0 @@
-import argparse
-import json
-import logging
-import shutil
-from pathlib import Path
-import torch
-from torch import nn, optim
-from torch.utils.data import Dataset, DataLoader
-from torch.optim.lr_scheduler import CosineAnnealingLR
-from bible_tts.data.text_processor import CharTokenizer
-from bible_tts.models.flow_lm import FlowLMModel
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
-logger = logging.getLogger(__name__)
-class CodecDataset(Dataset):
-    """Dataset of (text, codec_token_sequence) pairs from encoded JSONL."""
-    def __init__(self, encoded_path: str, max_code_len: int = 1000):
-        self.tokenizer = CharTokenizer()
-        self.max_code_len = max_code_len
-        with open(encoded_path) as f:
-            self.entries = [json.loads(line) for line in f if line.strip()]
-    def __len__(self):
-        return len(self.entries)
-    def __getitem__(self, idx):
-        entry = self.entries[idx]
-        text_tokens = self.tokenizer.encode(entry["text"])
-        codes = torch.tensor(entry["codes"], dtype=torch.long)
-        if codes.shape[-1] > self.max_code_len:
-            codes = codes[:self.max_code_len]
-        return {"text_tokens": text_tokens, "codes": codes}
-def collate_fn(batch):
-    max_text = max(item["text_tokens"].shape[-1] for item in batch)
-    max_code = max(item["codes"].shape[-1] for item in batch)
-    text_padded = torch.zeros(len(batch), max_text, dtype=torch.long)
-    code_padded = torch.zeros(len(batch), max_code, dtype=torch.long)
-    code_masks = torch.zeros(len(batch), max_code, dtype=torch.bool)
-    for i, item in enumerate(batch):
-        tl = item["text_tokens"].shape[-1]
-        cl = item["codes"].shape[-1]
-        text_padded[i, :tl] = item["text_tokens"]
-        code_padded[i, :cl] = item["codes"]
-        code_masks[i, :cl] = True
-    return {"text_tokens": text_padded, "codes": code_padded, "code_masks": code_masks}
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--encoded-data", type=str, required=True)
-    parser.add_argument("--checkpoint-dir", type=str, default="checkpoints")
-    parser.add_argument("--batch-size", type=int, default=32)
-    parser.add_argument("--lr", type=float, default=2e-4)
-    parser.add_argument("--weight-decay", type=float, default=1e-4)
-    parser.add_argument("--max-steps", type=int, default=300000)
-    parser.add_argument("--warmup-steps", type=int, default=5000)
-    parser.add_argument("--log-interval", type=int, default=100)
-    parser.add_argument("--save-interval", type=int, default=10000)
-    parser.add_argument("--seed", type=int, default=42)
-    parser.add_argument("--resume", type=str, default=None)
-    return parser.parse_args()
-def train():
-    args = parse_args()
-    torch.manual_seed(args.seed)
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    logger.info(f"Using device: {device}")
-    checkpoint_dir = Path(args.checkpoint_dir)
-    checkpoint_dir.mkdir(parents=True, exist_ok=True)
-    dataset = CodecDataset(args.encoded_data)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=True,
-        num_workers=4,
-        pin_memory=True,
-        collate_fn=collate_fn,
-    )
-    logger.info(f"Dataset: {len(dataset)} examples")
-    model = FlowLMModel().to(device)
-    optimizer = optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
-    scheduler = CosineAnnealingLR(optimizer, T_max=args.max_steps - args.warmup_steps)
-    scaler = torch.amp.GradScaler("cuda" if torch.cuda.is_available() else "cpu")
-    start_step = 0
-    if args.resume:
-        state = torch.load(args.resume, map_location=device)
-        model.load_state_dict(state["model"])
-        optimizer.load_state_dict(state["optimizer"])
-        if "scheduler" in state:
-            scheduler.load_state_dict(state["scheduler"])
-        start_step = state["step"]
-        logger.info(f"Resumed from step {start_step}")
-    step = start_step
-    best_loss = float("inf")
-    while step < args.max_steps:
-        for batch in loader:
-            if step >= args.max_steps:
-                break
-            text = batch["text_tokens"].to(device)
-            codes = batch["codes"].to(device)
-            # Flow matching training
-            B, T = codes.shape
-            latent_dim = model.latent_dim
-            # Convert code tokens to latent embeddings
-            code_emb = model.code_embed(codes)  # [B, T, latent_dim]
-            # Sample random timestep for flow matching
-            t = torch.rand(B, 1, device=device)
-            noise = torch.randn(B, latent_dim, device=device)
-            with torch.amp.autocast(device_type="cuda" if torch.cuda.is_available() else "cpu"):
-                # Text embedding
-                text_emb = model.char_embed(text)
-                text_emb = model.text_proj(text_emb)
-                # Concatenate and run transformer
-                combined = torch.cat([text_emb, code_emb], dim=1)
-                out, _ = model.transformer(combined)
-                out = model.out_norm(out)
-                # For each position, predict velocity
-                cond = out[:, -1]  # last position -> conditioning
-                t_prev = t * 0  # starting from noise
-                t_curr = t
-                x_t = (1 - t) * noise + t * code_emb[:, -1, :]
-                v_pred = model.flow_mlp(cond, t_prev, t_curr, x_t)
-                v_target = code_emb[:, -1] - noise  # ground truth
-                flow_loss = nn.functional.mse_loss(v_pred, v_target)
-                # Placeholder EOS loss — simplified for now
-                eos_logits = model.eos_head(out[:, -1])
-                eos_loss = nn.functional.binary_cross_entropy_with_logits(
-                    eos_logits, torch.zeros(B, 1, device=device)
-                )
-                total_loss = flow_loss + eos_loss
-            scaler.scale(total_loss).backward()
-            scaler.unscale_(optimizer)
-            nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-            scaler.step(optimizer)
-            scaler.update()
-            optimizer.zero_grad()
-            if step < args.warmup_steps:
-                lr_scale = min(1.0, (step + 1) / args.warmup_steps)
-                for pg in optimizer.param_groups:
-                    pg["lr"] = args.lr * lr_scale
-            else:
-                scheduler.step()
-            if step % args.log_interval == 0:
-                logger.info(
-                    f"Step {step}/{args.max_steps} | flow: {flow_loss.item():.4f} | "
-                    f"eos: {eos_loss.item():.4f} | total: {total_loss.item():.4f}"
-                )
-            if step % args.save_interval == 0 and step > 0:
-                state = {
-                    "model": model.state_dict(),
-                    "optimizer": optimizer.state_dict(),
-                    "scheduler": scheduler.state_dict(),
-                    "step": step,
-                    "loss": total_loss.item(),
-                }
-                ckpt_path = checkpoint_dir / f"flow_step_{step}.pt"
-                torch.save(state, ckpt_path)
-                logger.info(f"Saved checkpoint to {ckpt_path}")
-                if total_loss.item() < best_loss:
-                    best_loss = total_loss.item()
-                    shutil.copy(ckpt_path, checkpoint_dir / "flow_best.pt")
-                    logger.info(f"New best loss: {best_loss:.4f}")
-            step += 1
-    state = {
-        "model": model.state_dict(),
-        "optimizer": optimizer.state_dict(),
-        "scheduler": scheduler.state_dict(),
-        "step": step,
-        "loss": total_loss.item(),
-    }
-    torch.save(state, checkpoint_dir / "flow_final.pt")
-    logger.info(f"Training complete. Final model saved to {checkpoint_dir / 'flow_final.pt'}")
-if __name__ == "__main__":
-    train()

bible-tts/src/bible_tts/training/train_vqvae.py DELETED Viewed

@@ -1,138 +0,0 @@
-import argparse
-import logging
-import shutil
-from pathlib import Path
-import torch
-from torch import nn, optim
-from torch.utils.data import DataLoader
-from torch.optim.lr_scheduler import CosineAnnealingLR
-from bible_tts.data.audio_dataset import AudioDataset, collate_audio
-from bible_tts.models.vq_vae import VQVAE
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
-logger = logging.getLogger(__name__)
-def parse_args():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--manifest", type=str, required=True, help="Path to manifest JSONL")
-    parser.add_argument("--checkpoint-dir", type=str, default="checkpoints")
-    parser.add_argument("--batch-size", type=int, default=16)
-    parser.add_argument("--lr", type=float, default=3e-4)
-    parser.add_argument("--weight-decay", type=float, default=1e-4)
-    parser.add_argument("--max-steps", type=int, default=200000)
-    parser.add_argument("--warmup-steps", type=int, default=5000)
-    parser.add_argument("--log-interval", type=int, default=100)
-    parser.add_argument("--save-interval", type=int, default=10000)
-    parser.add_argument("--seed", type=int, default=42)
-    parser.add_argument("--resume", type=str, default=None)
-    return parser.parse_args()
-def train():
-    args = parse_args()
-    torch.manual_seed(args.seed)
-    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-    logger.info(f"Using device: {device}")
-    checkpoint_dir = Path(args.checkpoint_dir)
-    checkpoint_dir.mkdir(parents=True, exist_ok=True)
-    dataset = AudioDataset(args.manifest)
-    loader = DataLoader(
-        dataset,
-        batch_size=args.batch_size,
-        shuffle=True,
-        num_workers=4,
-        pin_memory=True,
-        collate_fn=collate_audio,
-    )
-    logger.info(f"Dataset: {len(dataset)} examples")
-    model = VQVAE().to(device)
-    optimizer = optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
-    scheduler = CosineAnnealingLR(optimizer, T_max=args.max_steps - args.warmup_steps)
-    scaler = torch.amp.GradScaler("cuda" if torch.cuda.is_available() else "cpu")
-    start_step = 0
-    if args.resume:
-        state = torch.load(args.resume, map_location=device)
-        model.load_state_dict(state["model"])
-        optimizer.load_state_dict(state["optimizer"])
-        if "scheduler" in state:
-            scheduler.load_state_dict(state["scheduler"])
-        start_step = state["step"]
-        logger.info(f"Resumed from step {start_step}")
-    step = start_step
-    best_loss = float("inf")
-    while step < args.max_steps:
-        for batch in loader:
-            if step >= args.max_steps:
-                break
-            audio = batch["audio"].unsqueeze(1).to(device)  # [B, 1, T]
-            crop_len = min(audio.shape[-1], 48000)  # 2 sec at 24kHz
-            if audio.shape[-1] > crop_len:
-                offset = torch.randint(0, audio.shape[-1] - crop_len, (1,)).item()
-                audio = audio[..., offset : offset + crop_len]
-            with torch.amp.autocast(device_type="cuda" if torch.cuda.is_available() else "cpu"):
-                x_hat, indices, vq_loss = model(audio)
-                recon_loss = nn.functional.l1_loss(x_hat, audio)
-                total_loss = recon_loss + vq_loss
-            scaler.scale(total_loss).backward()
-            scaler.unscale_(optimizer)
-            nn.utils.clip_grad_norm_(model.parameters(), 1.0)
-            scaler.step(optimizer)
-            scaler.update()
-            optimizer.zero_grad()
-            if step < args.warmup_steps:
-                lr_scale = min(1.0, (step + 1) / args.warmup_steps)
-                for pg in optimizer.param_groups:
-                    pg["lr"] = args.lr * lr_scale
-            else:
-                scheduler.step()
-            if step % args.log_interval == 0:
-                logger.info(
-                    f"Step {step}/{args.max_steps} | recon: {recon_loss.item():.4f} | "
-                    f"vq: {vq_loss.item():.4f} | total: {total_loss.item():.4f}"
-                )
-            if step % args.save_interval == 0 and step > 0:
-                state = {
-                    "model": model.state_dict(),
-                    "optimizer": optimizer.state_dict(),
-                    "scheduler": scheduler.state_dict(),
-                    "step": step,
-                    "loss": total_loss.item(),
-                }
-                ckpt_path = checkpoint_dir / f"vqvae_step_{step}.pt"
-                torch.save(state, ckpt_path)
-                logger.info(f"Saved checkpoint to {ckpt_path}")
-                if total_loss.item() < best_loss:
-                    best_loss = total_loss.item()
-                    shutil.copy(ckpt_path, checkpoint_dir / "vqvae_best.pt")
-                    logger.info(f"New best loss: {best_loss:.4f}")
-            step += 1
-    state = {
-        "model": model.state_dict(),
-        "optimizer": optimizer.state_dict(),
-        "scheduler": scheduler.state_dict(),
-        "step": step,
-        "loss": total_loss.item(),
-    }
-    torch.save(state, checkpoint_dir / "vqvae_final.pt")
-    logger.info(f"Training complete. Final model saved to {checkpoint_dir / 'vqvae_final.pt'}")
-if __name__ == "__main__":
-    train()

bible-tts/tests/__init__.py DELETED Viewed

File without changes

bible-tts/tests/test_flow_lm.py DELETED Viewed

@@ -1,69 +0,0 @@
-import torch
-from bible_tts.models.flow_lm import (
-    StreamingMultiheadAttention,
-    StreamingTransformerLayer,
-    StreamingTransformer,
-    FlowMLP,
-    FlowLMModel,
-)
-from bible_tts.models.lsd import lsd_decode
-def test_attention_shapes():
-    attn = StreamingMultiheadAttention(d_model=256, num_heads=4)
-    x = torch.randn(2, 10, 256)
-    out, cache = attn(x)
-    assert out.shape == x.shape
-    assert cache.shape == (2, 2, 10, 4, 64)
-def test_attention_with_cache():
-    attn = StreamingMultiheadAttention(d_model=256, num_heads=4)
-    x1 = torch.randn(1, 5, 256)
-    x2 = torch.randn(1, 3, 256)
-    _, cache = attn(x1)
-    out2, cache2 = attn(x2, cache)
-    assert out2.shape == x2.shape
-    assert cache2.shape[2] == 8
-def test_transformer_layer():
-    layer = StreamingTransformerLayer(d_model=256, num_heads=4, ff_hidden=1024)
-    x = torch.randn(2, 10, 256)
-    out, cache = layer(x)
-    assert out.shape == x.shape
-def test_transformer():
-    tf = StreamingTransformer(d_model=256, num_layers=3, num_heads=4, ff_hidden=1024)
-    x = torch.randn(2, 10, 256)
-    out, caches = tf(x)
-    assert len(caches) == 3
-    assert out.shape == x.shape
-def test_flow_mlp():
-    mlp = FlowMLP(d_model=256, latent_dim=64)
-    c = torch.randn(2, 256)
-    t1 = torch.randn(2, 1)
-    t2 = torch.randn(2, 1)
-    x = torch.randn(2, 64)
-    out = mlp(c, t1, t2, x)
-    assert out.shape == x.shape
-def test_flow_lm_forward():
-    model = FlowLMModel()
-    text = torch.randint(0, 88, (1, 50))
-    codes = torch.randint(0, 512, (1, 100))
-    last_out, is_eos = model(text, codes, lsd_steps=0)
-    assert last_out.shape == (1, 256)
-    assert is_eos.shape == (1, 1)
-def test_lsd_decode():
-    def flow_fn(t_prev, t_curr, x_t):
-        return -x_t
-    x_0 = torch.randn(2, 64)
-    result = lsd_decode(flow_fn, x_0, num_steps=4)
-    assert result.shape == x_0.shape

bible-tts/tests/test_text_processor.py DELETED Viewed

@@ -1,26 +0,0 @@
-import torch
-from bible_tts.data.text_processor import CharTokenizer
-def test_tokenizer_roundtrip():
-    tokenizer = CharTokenizer()
-    text = "In the beginning God created the heavens and the earth."
-    tokens = tokenizer.encode(text)
-    decoded = tokenizer.decode(tokens)
-    assert decoded == text, f"Roundtrip failed: {decoded} != {text}"
-def test_tokenizer_unknown_chars():
-    tokenizer = CharTokenizer()
-    text = "Hello\nWorld\t!"
-    tokens = tokenizer.encode(text)
-    decoded = tokenizer.decode(tokens)
-    assert "\n" not in decoded and "\t" not in decoded
-def test_tokenizer_vocab_size():
-    tokenizer = CharTokenizer()
-    assert tokenizer.vocab_size == 3 + len(
-        "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
-        ".,!?;:-\"'()[]{}@#$%&*+=/\\<>~`| "
-    )

bible-tts/tests/test_vq_vae.py DELETED Viewed

@@ -1,53 +0,0 @@
-import torch
-from bible_tts.models.vq_vae import VQVAE, ResBlock, Encoder, Decoder, VectorQuantizer
-def test_resblock_shapes():
-    block = ResBlock(dim=64)
-    x = torch.randn(2, 64, 100)
-    out = block(x)
-    assert out.shape == x.shape
-def test_encoder_output_shape():
-    enc = Encoder()
-    x = torch.randn(1, 1, 24000)
-    z = enc(x)
-    # stride 64 => 24000/64 = 375 frames
-    assert z.shape[-1] == 375
-    assert z.shape[1] == 64
-def test_quantizer_roundtrip():
-    quant = VectorQuantizer(codebook_size=512, dim=64)
-    z = torch.randn(2, 64, 100)
-    z_q, indices, loss = quant(z)
-    assert z_q.shape == z.shape
-    assert indices.shape == (2, 100)
-    assert loss.item() >= 0
-def test_decoder_output_shape():
-    dec = Decoder()
-    z = torch.randn(1, 64, 375)
-    x = dec(z)
-    assert x.shape[-1] == 24000
-    assert x.shape[1] == 1
-def test_vqvae_forward():
-    model = VQVAE()
-    x = torch.randn(1, 1, 24000)
-    x_hat, indices, vq_loss = model(x)
-    assert x_hat.shape == x.shape
-    assert indices.shape == (1, 375)
-    assert vq_loss.item() >= 0
-def test_vqvae_encode_decode():
-    model = VQVAE()
-    x = torch.randn(1, 1, 24000)
-    indices = model.encode_to_indices(x)
-    x_recon = model.decode_from_indices(indices)
-    assert indices.shape == (1, 375)
-    assert x_recon.shape == x.shape

only-bible-app v3.7.2+24

git clone https://git.pyrossh.dev/only-bible-app

The only bible app you will ever need. No ads. No in-app purchases. No distractions.