hotcoco¶
Fast enough for every epoch, lean enough for every dataset.
A drop-in replacement for pycocotools that doesn't become the bottleneck — in your training loop or at foundation model scale. Up to 23× faster on standard COCO, 39× faster on Objects365, and fits comfortably in memory where alternatives run out.
Eval in under a second
Up to 23× faster than pycocotools. Eval goes from a bottleneck to background noise.
Your metrics, unchanged
10,000+ parity tests against pycocotools. Your AP scores won't budge.
More than a metric
TIDE error breakdown, confusion matrix, per-category AP, confidence calibration, and publication-quality plots built in. Find out why your model falls short, not just by how much.
Already works with your stack
init_as_pycocotools() patches imports in-place. Detectron2, mmdetection, RF-DETR — no code changes.
Quick start¶
pip install hotcoco
from hotcoco import COCO, COCOeval
coco_gt = COCO("instances_val2017.json")
coco_dt = coco_gt.load_res("detections.json")
ev = COCOeval(coco_gt, coco_dt, "bbox")
ev.run()
from hotcoco import init_as_pycocotools
init_as_pycocotools()
# All pycocotools imports now resolve to hotcoco
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
coco eval --gt instances_val2017.json --dt detections.json --iou-type bbox
use hotcoco::{COCO, COCOeval};
use hotcoco::params::IouType;
use std::path::Path;
let coco_gt = COCO::new(Path::new("instances_val2017.json"))?;
let coco_dt = coco_gt.load_res(Path::new("detections.json"))?;
let mut ev = COCOeval::new(coco_gt, coco_dt, IouType::Bbox);
ev.evaluate();
ev.accumulate();
ev.summarize();
Performance¶
Benchmarked on COCO val2017 (5,000 images, 36,781 synthetic detections), Apple M1 MacBook Air:
| Eval Type | pycocotools | faster-coco-eval | hotcoco |
|---|---|---|---|
| bbox | 9.46s | 2.45s (3.9×) | 0.41s (23.0×) |
| segm | 9.16s | 4.36s (2.1×) | 0.49s (18.6×) |
| keypoints | 2.62s | 1.78s (1.5×) | 0.21s (12.7×) |
All 12 metrics verified against pycocotools on COCO val2017 with a 10,000+ case parity test suite.
License¶
MIT