data pruning and deduplication for AI