⚠️ Under Development
Modular-GenAI is a library designed for Generative AI (GenAI) research. Our goal is to provide a flexible, clean and easy-to-use set of components that can be combined in various ways to enable experimentation with different generative AI algorithms. The repository includes both PyTorch and Jax implementations of the algorithms.
We are currently focusing on fine-tuning and search algorithms to improve reasoning and long-term planning in large language models.
| Transformers |  |
 |
|---|---|---|
| LLM Transformers | ||
| DeepSeek v3 | ❌ | ❌ |
| Mistral | ❌ | ❌ |
| Llama v3.2 | ❌ | ❌ |
| Llama v2 | ❌ | ✔️ |
| Decision Transformers | ||
| Behavior Clonning | ❌ | ✔️ |
| Decision Transformer | ❌ | ✔️ |
| Q-Transformer | ❌ | ❌ |
| Vision Transformers | ||
| ViT | ❌ | ❌ |
| MaxViT | ❌ | ❌ |
| Diffusion Transformers | ||
| DiT | ❌ | ❌ |
| Multimodal Transformers | ||
| Qwen2.5-VL | ❌ | ❌ |
| Other Specialized Transformers | ||
| Mamba | ❌ | ❌ |
| Diffusion Models | |
|
|---|---|---|
| Image Generation | ||
| Stable Diffusion | ❌ | ✔️ |
| Audio Generation (Optional) | ||
| AudioLDM | ❌ | ❌ |
| Video Generation | ||
| Video Diffusion Models | ❌ | ❌ |
| 3D Generation (Optional) | ||
| DreamFusion | ❌ | ❌ |
| Components | |
|
|---|---|---|
| Fine Tuning | ||
| PPO | ✔️ | ✔️ |
| DPO | ❌ | ❌ |
| LoRA | ❌ | ❌ |
| RLOO | ❌ | ❌ |
| Reverse Curriculum Reinforcement Learning | ❌ | ❌ |
| Attention Modules | ||
| Sliding Window Attention | ❌ | ❌ |
| Encoders | ||
| VQ-VAE | ❌ | ❌ |
| VAE | ❌ | ✔️ |
| CLIP | ❌ | ✔️ |
| Schedulers | ||
| DDPM | ❌ | ✔️ |
| Search Algorithms | ||
| Monte Carlo Tree Search | ❌ | ❌ |