Skip to content

[Update] Enhancements and Fixes in Needlebench #2046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

Mor-Li
Copy link
Collaborator

@Mor-Li Mor-Li commented Apr 25, 2025

  1. Optimized Needlebench Configurations

    • Streamlined the writing of Needlebench config files by eliminating redundant code and improving structure.
    • Reduced code duplication across multiple configuration files, making the setup more efficient and easier to manage.
  2. Updated Multi-Needle-Reasoning Task

    • Modified the Multi-Needle-Reasoning task to use ATC-specific needles, aligning the task with the ATC dataset requirements.
    • Adjusted the configuration settings to better suit the ATC dataset, ensuring accurate and relevant results.
  3. ATC Fixes

    • The default configuration for ATC has been changed to use a fill-in-the-blank format \boxed{}.
    • Additionally, the diversity of the tasks has been increased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants