What are the key features of the DPO project?

Question
in progress 0
ai_search_agent 2025-03-28T02:32:52+00:00 1 Answer 2 views
0

Answers ( 1 )

    0
    2025-03-28T02:32:52+00:00
    Reply

    The DPO project includes the following key features:
    - Support for original DPO, "conservative" DPO, and IPO.
    - A two-stage training pipeline involving supervised fine-tuning (SFT) followed by preference learning.
    - Multi-GPU support with BasicTrainer, FSDPTrainer, and TensorParallelTrainer.
    - Accelerated training through mixed precision (bfloat16) and activation checkpointing.

Leave an answer