Pro tip: back up your training data before you let an AI clean it, just lost 40 hours of labeling

Derek Lee · 2026-09-30T07:35:05.407803+00:00

Last Tuesday I ran a dedup script through my fine tuning set and the model quietly wiped out about 3,000 rows from my dataset, including every example I had labeled by hand over 40 hours. I didn't notice until training came back with garbage results and I checked the row count, and of course I had no backup because I figured the script was safe. Has anyone else had an AI tool eat part of their dataset, and is there a versioning setup you actually trust?

Vivisector

Loading…

Pro tip: back up your training data before you let an AI clean it, just lost 40 hours of labeling | AI Innovations - Vivisector