Skip to content

hotcoco

Fast enough for every epoch, lean enough for every dataset.

A drop-in replacement for pycocotools that doesn't become the bottleneck — in your training loop or at foundation model scale. Up to 23× faster on standard COCO, 39× faster on Objects365, and fits comfortably in memory where alternatives run out.

Eval in under a second

Up to 23× faster than pycocotools. Eval goes from a bottleneck to background noise.

Your metrics, unchanged

10,000+ parity tests against pycocotools. Your AP scores won't budge.

More than a metric

TIDE error breakdown, confusion matrix, per-category AP, confidence calibration, and publication-quality plots built in. Find out why your model falls short, not just by how much.

Already works with your stack

init_as_pycocotools() patches imports in-place. Detectron2, mmdetection, RF-DETR — no code changes.

Quick start

pip install hotcoco
from hotcoco import COCO, COCOeval

coco_gt = COCO("instances_val2017.json")
coco_dt = coco_gt.load_res("detections.json")

ev = COCOeval(coco_gt, coco_dt, "bbox")
ev.run()
from hotcoco import init_as_pycocotools
init_as_pycocotools()

# All pycocotools imports now resolve to hotcoco
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
coco eval --gt instances_val2017.json --dt detections.json --iou-type bbox
use hotcoco::{COCO, COCOeval};
use hotcoco::params::IouType;
use std::path::Path;

let coco_gt = COCO::new(Path::new("instances_val2017.json"))?;
let coco_dt = coco_gt.load_res(Path::new("detections.json"))?;

let mut ev = COCOeval::new(coco_gt, coco_dt, IouType::Bbox);
ev.evaluate();
ev.accumulate();
ev.summarize();

Performance

Benchmarked on COCO val2017 (5,000 images, 36,781 synthetic detections), Apple M1 MacBook Air:

Eval Type pycocotools faster-coco-eval hotcoco
bbox 9.46s 2.45s (3.9×) 0.41s (23.0×)
segm 9.16s 4.36s (2.1×) 0.49s (18.6×)
keypoints 2.62s 1.78s (1.5×) 0.21s (12.7×)

All 12 metrics verified against pycocotools on COCO val2017 with a 10,000+ case parity test suite.

License

MIT