HuggingFaceFW/finepdfs
Viewer • Updated • 476M • 57.8k • 947
AI & ML interests
We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)
Recent Activity
View all activity
Papers
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
HuggingFaceFW/finephrase
Viewer • Updated • 1.02B • 155k • 150
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 56.1M • 1.54k • 2
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 56.1M • 5.11k
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT-shuffled
Viewer • Updated • 62.1M • 1.19k • 6
HuggingFaceFW/finepdfs_50BT-dclm_30BT-fineweb_edu_20BT
Viewer • Updated • 62.1M • 2.43k • 3
HuggingFaceFW/finepdfs_edu_100BT-shuffled
Viewer • Updated • 17.8M • 812
HuggingFaceFW/finepdfs_edu_100BT
Viewer • Updated • 17.8M • 1.55k • 1
HuggingFaceFW/finepdfs_100BT-shuffled
Viewer • Updated • 14.6M • 683
HuggingFaceFW/finepdfs_100BT
Viewer • Updated • 29.9M • 1.58k • 1
HuggingFaceFW/fineweb_edu_100BT-shuffled
Viewer • Updated • 102M • 2.41k • 6
HuggingFaceFW/fineweb_edu_100BT
Preview • Updated • 3.1k • 4
HuggingFaceFW/fineweb_100BT-shuffled
Viewer • Updated • 161M • 834 • 1
HuggingFaceFW/fineweb_100BT
Viewer • Updated • 161M • 551 • 4
HuggingFaceFW/dclm_100BT-shuffled
Viewer • Updated • 89.3M • 822 • 4
HuggingFaceFW/dclm_100BT
Viewer • Updated • 89.3M • 1.75k • 2
HuggingFaceFW/finetranslations-edu
Viewer • Updated • 109M • 1.89k • 30
HuggingFaceFW/finetranslations
Viewer • Updated • 3.33B • 26.7k • 302
HuggingFaceFW/admin
Viewer • Updated • 18 • 16.7k • 3
HuggingFaceFW/CommonsenseQA
Viewer • Updated • 1k • 60 • 2
HuggingFaceFW/MMLU-Redux-2.0-Generative
Viewer • Updated • 5.43k • 220 • 2
HuggingFaceFW/ARC-Generative
Viewer • Updated • 7.79k • 40
HuggingFaceFW/finepdfs-edu
Viewer • Updated • 49.5M • 14.2k • 101
HuggingFaceFW/fineweb-2
Viewer • Updated • 4.48B • 113k • 895
HuggingFaceFW/finewiki
Viewer • Updated • 61.6M • 11.4k • 325
HuggingFaceFW/clean-wikipedia
Viewer • Updated • 61.2M • 1.53k • 24
HuggingFaceFW/finepdfs_lang_classification_tmp
Updated • 5
HuggingFaceFW/ocr-annotations
Viewer • Updated • 1.62k • 99 • 19
HuggingFaceFW/finepdfs_lang_classification
Viewer • Updated • 3.08M • 24.2k • 4
HuggingFaceFW/finepdfs_eng_Latn_labeled
Viewer • Updated • 1.3M • 110 • 3