A new method, ActObs, is introduced to supervise both agent action tokens and observation tokens during reinforcement learning. This approach improves performance in tasks such as code editing and retains more entropy in the policy, leaving it closer to its initialization.