[Example] Clip_B and Clip_V from entropy dynamics (#509)

Yuchang Sun committed on Feb 26, 2026, 11:36 AM

Showing 14 changed files +1496 additions -9 deletions

A examples/entropy/README.md +80

A examples/entropy/clipb.yaml +100

A examples/entropy/clipb_trainer.patch +11

A examples/entropy/clipv.yaml +100

A examples/entropy/clipv_dp_actor.py +426

A examples/entropy/clipv_trainer.patch +110

M trinity/algorithm/__init__.py +1

M trinity/algorithm/advantage_fn/__init__.py +2

A trinity/algorithm/advantage_fn/clipb_advantage.py +152

A trinity/algorithm/advantage_fn/clipv_advantage.py +153

M trinity/algorithm/algorithm.py +22

M trinity/common/verl_config.py +1

M trinity/trainer/verl/dp_actor.py +329 -2

M trinity/trainer/verl/fsdp_workers.py +9 -7