Reduce JsonlDataset memory by using mmap array and use npy to store jsonl meta#1604
Merged
jayhenry merged 9 commits intoInternLM:mainfrom Mar 20, 2026
Merged
Reduce JsonlDataset memory by using mmap array and use npy to store jsonl meta#1604jayhenry merged 9 commits intoInternLM:mainfrom
jayhenry merged 9 commits intoInternLM:mainfrom
Commits
Commits on Mar 19, 2026
- committed
- committed
- committed
- committed
- committed
- committed
