Skip to content

Linear Loss Tracker #2688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pbontrager opened this issue May 7, 2025 · 0 comments
Open

Linear Loss Tracker #2688

pbontrager opened this issue May 7, 2025 · 0 comments

Comments

@pbontrager
Copy link
Contributor

pbontrager commented May 7, 2025

We recently rolled out linear losses which can further save memory over just chunked loss. It's been rolled out to all of the SFT recipes but there still remains followup work to implement it in a few more recipes and get it fully interoperable with compile + TP. These changes need to land before we can fully deprecate CEWithChunkedOutputLoss.

Original PRs:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant