GRPOTrainer

This release introduces GRPO, achieving up to 90% memory reduction during training, alongside various bug fixes and updates to support Llama 3.1 8B training.

This release introduces major support for GRPO training, enabling LoRA/QLoRA for GRPO across various models, and integrates fast inference via vLLM for significant throughput gains. Numerous bug fixes address issues with Gemma 2, Mistral mapping, and general stability.