An automated pipeline that spins out a growing DPO preference dataset. The same model answers every prompt twice under two system prompts, one expert and one terse, and the gap between the answers becomes the training signal. No human annotation is needed.
Snapshot from the pipeline's local stats, covering runs from 2026-06-08 to 2026-08-18. The Hub dataset is the source of truth for its current size.
How it works
topics.py picks a domain, a topic (about 50 per domain) and one of 30 prompt templates, such as “What is {topic} and why does it matter?”
generator.py asks Llama 3.1 8B Instruct through the Hugging Face Inference API: once as a thorough expert (chosen, up to 512 tokens) and once as a hurried assistant (rejected, up to 256 tokens).
A pair is dropped if the chosen answer is under 200 characters, the rejected under 30, or the two are identical.
Prompts that are already in the dataset are skipped, so the dataset only ever grows with new rows.
Accepted pairs go to a pending file. Every 50 pairs, uploader.py merges them into a Parquet file and pushes it to the Hugging Face Hub.
main.py shows a live terminal dashboard. Ctrl+C stops cleanly and uploads the final batch.
The dataset
prompt, chosen, rejected, topic and timestamp. Load it with load_dataset("iPwnds/Project-Silkworm") and use it straight away.
The format works with TRL's DPOTrainer, so it plugs into preference-tuning pipelines without conversion.
The dataset is CC BY 4.0, plus the Llama 3.1 Community License because the outputs come from Llama 3.1. The code is MIT.
Limitations
“Chosen” means “answered under the expert prompt”, not human-verified correctness, so answers can contain factual errors.
The rejected side is deliberately low quality, so a model trained on it learns mostly a length and effort preference.
Everything is generated by a single 8B model, so the dataset inherits that model's biases and gaps.
Run your own
HF_MODEL sets the generation modelUPLOAD_EVERY defaults to 50 pairsMIN_CHOSEN_LENGTH 200 and MIN_REJECTED_LENGTH 30REQUEST_DELAY defaults to 1.5 secondspip install -r requirements.txt.env.example to .env and set HF_TOKEN and HF_DATASET_REPOpython main.pygenerator.py chosen and rejected generation, quality filtertopics.py topic lists and prompt templatesuploader.py local storage, stats and Hub uploadtests/ covers sampling, the filter and storageExplore
Project Silkworm is open source. The code is MIT, the dataset is CC BY 4.0, and the dataset lives on the Hugging Face Hub.