[heartbeat] starting: python3 /home/fedora/prep_co1024.py 1000000 5200 /data/co1024_1m
[heartbeat] interval: 60s
[heartbeat] label:    python3
/home/fedora/.local/lib/python3.14/site-packages/huggingface_hub/constants.py:302: FutureWarning: The `HF_HUB_ENABLE_HF_TRANSFER` environment variable is deprecated as 'hf_transfer' is not used anymore. Please use `HF_XET_HIGH_PERFORMANCE` instead to enable high performance transfer with Xet. Visit https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfxethighperformance for more details.
  warnings.warn(
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
  shard 0: corpus 99943/1000000 queries 0/5200 2s
  shard 1: corpus 199883/1000000 queries 0/5200 3s
  shard 2: corpus 299814/1000000 queries 0/5200 5s
  shard 3: corpus 399664/1000000 queries 0/5200 7s
  shard 4: corpus 499455/1000000 queries 0/5200 9s
  shard 5: corpus 599337/1000000 queries 0/5200 10s
  shard 6: corpus 699271/1000000 queries 0/5200 12s
  shard 7: corpus 799246/1000000 queries 0/5200 14s
  shard 8: corpus 899214/1000000 queries 0/5200 15s
  shard 9: corpus 999159/1000000 queries 0/5200 17s
  shard 10: corpus 1000000/1000000 queries 5200/5200 18s
vectors saved 22s
. [python3 19:22:22 elapsed]
GT done 60s
copy files done 78s
{"null": 0, "badlen": 0, "zero": 0, "corpus_dup_dropped": 841, "query_dup_dropped": 0, "query_eq_corpus": 0, "rows_read": 1006041, "max_abs_norm_dev_corpus": 1.4543587822934967e-07, "max_abs_norm_dev_queries": 1.2242529900952093e-07, "query_eq_corpus_qids": [], "tie_ambiguous_k10": 3, "tie_ambiguous_k100": 24, "queries_with_dist0_top1": 0}
done 85s
[heartbeat] DONE rc=0 (clean exit)
