Skip to content

Fix mkdir hardcode in runners and partitioners #2060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

BIGWangYuDong
Copy link
Contributor

@BIGWangYuDong BIGWangYuDong commented Apr 28, 2025

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

  1. The tmp file is hard-coded in the runners, add tmp_dir, which can setting in the config.
  2. In partitioners, is hardcoded to mkdir('.cache') which may get error if set other dataset_size_path

Modification

  1. add tmp_dir in the runner.
  2. update mkdir logic in partitioners

BC-breaking (Optional)

seems no BC-breaking.

@BIGWangYuDong BIGWangYuDong changed the title add tmp_dir in runners to avoid hard-code [WIP] add tmp_dir in runners to avoid hard-code Apr 28, 2025
@BIGWangYuDong BIGWangYuDong changed the title [WIP] add tmp_dir in runners to avoid hard-code Fix mkdir hardcode in runners and partitioners Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants