PRX Part 4: Our Data Strategy

Summary

Photoroom’s data pipeline for pre-training the 7B PRX text-to-image diffusion model begins with a deliberate focus on breadth rather than per-image quality. The team assembled a large corpus from a combination of public and internal sources, prioritizing existing curation work on quality filtering, deduplication, and removal of NSFW or personal content. Long, accurate captions were treated as the central lever for learning visual concepts, composition, and lighting; short or inconsistent captions from source datasets were discarded in favor of uniform re-captioning.

All metadata and images were first ingested into Lance tables using Ray Data for parallel processing across the cluster. Lance’s columnar layout, predicate pushdown, and support for full-text and vector indexes enabled interactive exploration of hundreds of millions of rows, including resolution profiling, caption search, and nearest-neighbor browsing via CLIP-style embeddings. Lessons on fragmentation were learned in practice: initial shards of 100 k rows produced thousands of fragments that slowed queries; compaction to roughly one million rows per fragment reduced the count to around a thousand and restored acceptable scan performance.

Every image was then re-captioned with Qwen2.5-VL-7B to enforce consistent length and fidelity. An early ablation trained a smaller diffusion model on the same images using either these new captions or shorter LLaVA-1.5-LLaMA3-8B captions; the Qwen captions produced lower FID, CMMD, and DINO-MMD scores at every checkpoint up to 100 k steps. The resulting captions also allow the model to treat text, logos, or screenshots as controllable attributes rather than artifacts to be filtered out.

Images were stored as JPEGs at quality 92 after measurements showed that a single high-quality re-encode introduces no perceptible change relative to the already-compressed sources. Training two identical 1024 px models—one on PNG, one on JPEG—yielded statistically indistinguishable outputs and comparable estimated JPEG artifact levels, confirming that the storage choice carries negligible downstream effect. Text latents were computed on the fly during training rather than pre-materialized, trading a measured 3–4 % throughput penalty for substantially smaller MDS shards and the ability to swap text encoders without rewriting terabytes of data.

The curated Lance tables were finally converted to Mosaic Data Shards for efficient streaming and shuffling during distributed training. Light filtering removed only unusable content below the 512²-pixel training bucket (with a 4/3 upscaling cap), while deduplication addressed near-duplicates identified during exploratory browsing. The pipeline therefore separates the concerns of scale and diversity in pre-training from the later, narrower work of aesthetic fine-tuning.

Why it matters

Strong match for Hands-on model tooling category: concrete metrics, architectural choices (caption length, on-the-fly encoding), distributed training practices, and measurable impact on model quality vs. compute cost. Directly actionable for Dutch ML teams building or fine-tuning diffusion models.