[WIP] [Model] Index tts

[divyanshsinghvi]

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Resolves #229 . Integrates index-tts model https://github.com/index-tts/index-tts

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

[divyanshsinghvi]

@hsliuustc0106

Thinking of structuring in 3 steps like qwen3-omni. But confused about this specific files below should I be dumping all of them together in PR? Find few which match some implementation and replace those rest dump it with this PR?

├── __pycache__
├── gpt
│   ├── __init__.py
│   ├── conformer
│   │   ├── __init__.py
│   │   ├── attention.py
│   │   ├── embedding.py
│   │   └── subsampling.py
│   ├── conformer_encoder.py
│   ├── model_v2.py
│   └── perceiver.py
├── index_tts.py
├── index_tts_config.py
├── s2mel
│   ├── dac
│   │   ├── __init__.py
│   │   ├── __main__.py
│   │   ├── model
│   │   │   ├── __init__.py
│   │   │   ├── base.py
│   │   │   ├── dac.py
│   │   │   ├── discriminator.py
│   │   │   └── encodec.py
│   │   ├── nn
│   │   │   ├── __init__.py
│   │   │   ├── layers.py
│   │   │   ├── loss.py
│   │   │   └── quantize.py
│   │   └── utils
│   │       ├── __init__.py
│   │       ├── decode.py
│   │       └── encode.py
│   └── modules
│       ├── audio.py
│       ├── bigvgan
│       │   ├── activations.py
│       │   ├── alias_free_activation
│       │   │   ├── cuda
│       │   │   │   ├── __init__.py
│       │   │   │   ├── activation1d.py
│       │   │   │   ├── anti_alias_activation.cpp
│       │   │   │   ├── anti_alias_activation_cuda.cu
│       │   │   │   ├── compat.h
│       │   │   │   ├── load.py
│       │   │   │   └── type_shim.h
│       │   │   └── torch
│       │   │       ├── __init__.py
│       │   │       ├── act.py
│       │   │       ├── filter.py
│       │   │       └── resample.py
│       │   ├── bigvgan.py
│       │   ├── config.json
│       │   ├── env.py
│       │   ├── meldataset.py
│       │   └── utils.py
│       ├── campplus
│       │   ├── DTDNN.py
│       │   ├── classifier.py
│       │   └── layers.py
│       ├── commons.py
│       ├── diffusion_transformer.py
│       ├── encodec.py
│       ├── flow_matching.py
│       ├── gpt_fast
│       │   ├── generate.py
│       │   ├── model.py
│       │   └── quantize.py
│       ├── layers.py
│       ├── length_regulator.py
│       ├── quantize.py
│       ├── vocos
│       │   ├── __init__.py
│       │   ├── heads.py
│       │   ├── helpers.py
│       │   ├── loss.py
│       │   ├── models.py
│       │   ├── modules.py
│       │   ├── pretrained.py
│       │   └── spectral_ops.py
│       └── wavenet.py
├── utils
│   ├── __init__.py
│   ├── arch_util.py
│   ├── checkpoint.py
│   ├── common.py
│   ├── feature_extractors.py
│   ├── front.py
│   ├── maskgct
│   │   └── models
│   │       ├── codec
│   │       │   ├── __init__.py
│   │       │   ├── amphion_codec
│   │       │   │   ├── codec.py
│   │       │   │   ├── quantize
│   │       │   │   │   ├── __init__.py
│   │       │   │   │   ├── factorized_vector_quantize.py
│   │       │   │   │   ├── lookup_free_quantize.py
│   │       │   │   │   ├── residual_vq.py
│   │       │   │   │   └── vector_quantize.py
│   │       │   │   └── vocos.py
│   │       │   ├── codec_dataset.py
│   │       │   ├── codec_inference.py
│   │       │   ├── codec_sampler.py
│   │       │   ├── codec_trainer.py
│   │       │   ├── kmeans
│   │       │   │   ├── repcodec_model.py
│   │       │   │   └── vocos.py
│   │       │   └── melvqgan
│   │       │       └── melspec.py
│   │       └── tts
│   │           └── maskgct
│   │               ├── ckpt
│   │               │   └── wav2vec2bert_stats.pt
│   │               ├── llama_nar.py
│   │               └── maskgct_s2a.py
│   ├── maskgct_utils.py
│   ├── qwen0.6bemo4mergeconfig.json
│   ├── qwen_emotion.py
│   ├── tagger_cache
│   │   ├── zh_tn_tagger.fst
│   │   └── zh_tn_verbalizer.fst
│   ├── typical_sampling.py
│   ├── utils.py
│   └── xtransformers.py```
 

[hsliuustc0106]

this one looks quite complex :) maybe we can try a simple one as a start in tts #315

[divyanshsinghvi]

this one looks quite complex :) maybe we can try a simple one as a start in tts #315

Will raise a PR for it today. Ironing out lots of issues that came up while adding #315.

[divyanshsinghvi]

reopen

[hsliuustc0106]

@linyueqian PTAL

[divyanshsinghvi]

I still have to push a working pr, had to clean up, will have to wait for 2 days for me to finish it up.