Overview

EasyR1 is an efficient, scalable, multi-modality RL training framework based on veRL. It is designed to support vision language models and provides a range of features, including supported models, algorithms, datasets, and tricks. The framework is built on top of the HybridEngine and vLLM's SPMD mode, making it efficient and scalable. EasyR1 also provides a range of tools and scripts to make it easy to use and customize. The project is open-source and has a large community of contributors and users. EasyR1 has been used in a variety of applications, including multimodal reasoning, visual question answering, and image-text retrieval. The project is well-documented and has a range of tutorials and examples to help users get started. Overall, EasyR1 is a powerful and flexible framework for RL training and has the potential to be used in a wide range of applications. EasyR1 is efficient and scalable due to the design of HybridEngine and the latest release of vLLM's SPMD mode. The framework supports a range of models, including Llama3, Qwen2, Qwen2.5, Qwen3, Qwen2-VL, Qwen2.5-VL, Qwen3-VL, and DeepSeek-R1. It also supports a range of algorithms, including GRPO, DAPO, Reinforce++, ReMax, RLOO, GSPO, and CISPO. EasyR1 supports a range of datasets, including any text, vision-text dataset in a specific format. The framework also provides a range of tricks, including padding-free training, LoRA training, resuming from the latest/best checkpoint, and Wandb & SwanLab & Mlflow & Tensorboard tracking. EasyR1 has a range of requirements, including Python 3.9+, transformers, flash-attn, and vllm. The framework provides a range of tutorials and examples to help users get started, including a tutorial on running Qwen2.5-VL GRPO on the Geometry3K dataset. EasyR1 also has a range of performance baselines, including baselines for multimodal reasoning, visual question answering, and image-text retrieval. The project has a range of known bugs and limitations, including issues with vision language models and ulysses parallelism. Despite these limitations, EasyR1 is a powerful and flexible framework for RL training and has the potential to be used in a wide range of applications.

Pros & Cons