[ReID part 1] Standalone ReIDModel, architectures, and gallery eval by AlexBodner · Pull Request #480 · roboflow/trackers

AlexBodner · 2026-06-29T13:25:42Z

What does this PR do?

Standalone ReID foundation for Trackers — model loading, OSNet / timm: architectures, and gallery evaluation. Tracker association utilities and BoT-SORT wiring land in #501.

ReIDModel with curated OSNet alias and save_pretrained / from_pretrained
Clean-room OSNet + optional timm: backbones
Market-1501 / MSMT17 loaders + ReIDEvaluator (CMC / mAP / mINP)
Gallery encoder protocol: ReIDPathEncoder
Optional extra: pip install 'trackers[reid]'
Developer docs on the ReID API page: load scenarios and adding architectures

Intentionally not in this PR (deferred to #501): FeatureBank, appearance_similarity, ReIDEncoder, and BoT-SORT association.

Benchmarks

Same-domain OSNet x1.0 eval from notebooks/eval_reid.ipynb, compared to the torchreid model zoo (zoo numbers are euclidean).

Dataset	Distance	Rank-1	mAP	Zoo Rank-1	Zoo mAP
Market-1501	euclidean	94.6%	82.9%	94.2%	82.6%
Market-1501	cosine	94.9%	84.5%	—	—
MSMT17	euclidean	75.0%	43.7%	74.9%	43.8%
MSMT17	cosine	75.9%	47.8%	—	—

Euclidean matches the published protocol within ~0.5 pp. Cosine is our default association distance and is a different protocol (typically a bit higher).

Type of Change

New feature (non-breaking change that adds functionality)
Documentation update

Testing

I have tested this change locally
I have added/updated tests for this change

Test details:

NumPy suite: tests/core/reid/test_reid.py (gallery eval only; no trackers[reid] required)
Model / architecture suite: tests/core/reid/test_reid_model.py (needs uv sync --group dev --extra reid)
Main CI installs --extra reid and runs pytest -m "not integration"
Integration CI collects only tests/core/test_tracker_integration.py and tests/eval/test_integration.py

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code where necessary, particularly in hard-to-understand areas
My changes generate no new warnings or errors
I have updated the documentation accordingly (if applicable)

Additional Context

Follow-up: #501 adds shared tracker association utilities under trackers.core.reid, wires BoT-SORT, FastReID SBS, CLI, and demos on top of this branch.

Copilot

Pull request overview

Adds a ReID (appearance) subsystem to Trackers and integrates it into BoT-SORT and ByteTrack association so tracks can be maintained/recovered using appearance embeddings in addition to IoU.

Changes:

Introduces a trackers.core.reid package: OSNet clean-room backbone, model loading (hf:// + local), preprocessing, feature bank (EMA), similarity + fusion utilities, and offline eval (CMC/mAP/mINP).
Extends BoTSORTTracker and ByteTrackTracker to optionally use a ReIDModel for first-stage association (and BoT-SORT lost-track appearance recovery), with per-track EMA feature banks.
Adds tests for ReID model/config/loader behavior and fusion helpers, plus docs + notebooks demonstrating evaluation and webcam usage.

Reviewed changes

Copilot reviewed 28 out of 31 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
uv.lock	Adds locked dependency metadata for the new `reid` extra.
pyproject.toml	Introduces a `reid` optional-dependency extra (torch/torchvision/timm/hf hub).
src/trackers/init.py	Exposes `ReIDModel` at the package top level.
src/trackers/core/reid/init.py	Public ReID API surface (`ReIDModel`, eval helpers, similarity, etc.).
src/trackers/core/reid/model.py	ReID model wrapper: device selection, feature extraction, save/load entrypoints.
src/trackers/core/reid/feature_bank.py	Per-track EMA embedding storage.
src/trackers/core/reid/distance.py	Cosine similarity matrix computation for tracks × detections.
src/trackers/core/reid/extraction.py	Convenience wrapper to embed detection boxes from a frame.
src/trackers/core/reid/fusion_methods.py	Shared IoU/appearance fusion helpers (masking + weighted blend).
src/trackers/core/reid/models/init.py	Exposes registry/preprocessing/loader utilities.
src/trackers/core/reid/models/loaders.py	HF/local weights resolution + state-dict loading/reporting.
src/trackers/core/reid/models/preprocessing.py	Torchvision preprocessing pipeline config + serialization.
src/trackers/core/reid/models/registry.py	Alias/model-card registry + `reid_config.json` (de)serialization.
src/trackers/core/reid/architectures/init.py	Architecture builder for OSNet variants and `timm:` backbones.
src/trackers/core/reid/architectures/osnet.py	OSNet backbone implementation (adapted, licensed).
src/trackers/core/reid/eval/init.py	Public eval API exports.
src/trackers/core/reid/eval/datasets.py	Market-1501/MSMT17 dataset split loaders/parsers.
src/trackers/core/reid/eval/evaluator.py	End-to-end embedding extraction + metric computation pipeline.
src/trackers/core/reid/eval/metrics.py	Retrieval metric computation (CMC/mAP/mINP).
src/trackers/core/botsort/tracklet.py	Adds optional per-track `FeatureBank`.
src/trackers/core/botsort/tracker.py	Integrates ReID into BoT-SORT (gated-min fusion + lost recovery).
src/trackers/core/bytetrack/tracklet.py	Adds optional per-track `FeatureBank`.
src/trackers/core/bytetrack/tracker.py	Integrates ReID into ByteTrack first-stage association (weighted fusion).
tests/core/test_reid_fusion.py	Unit tests for fusion helpers + BoT-SORT gated-min behavior.
tests/core/reid/test_reid.py	Tests for preprocessing, registry/config, loaders, and model smoke/roundtrip.
tests/core/reid/init.py	Initializes the ReID test package.
docs/trackers/botsort.md	Documents new BoT-SORT ReID parameters and behavior.
notebooks/webcam_reid_demo.py	End-to-end demo script for webcam tracking with BoT-SORT + ReID.
notebooks/eval_reid.ipynb	ReID evaluation notebook for Market-1501/MSMT17.
.gitignore	Ignores local `rfcs/` directory.

CLAassistant · 2026-07-15T13:28:09Z

All committers have signed the CLA.

socket-security · 2026-07-15T13:28:24Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	gdown@6.1.0

View full report

Introduce the optional trackers[reid] stack with OSNet and FastReID SBS backbones, FeatureBank, Market-1501/MSMT17 evaluation, and lazy imports so base trackers stay torch-free. Tracker association wiring is deferred. Co-authored-by: Alexander Bodner <AlexBodner@users.noreply.github.com>

Match changelog style, narrow the public ReID exports, clarify docs as standalone embeddings/eval, and fix the eval notebook install pin. Co-authored-by: Alexander Bodner <AlexBodner@users.noreply.github.com>

Skip ReID src/tests during collection when optional deps are missing so integration CI (dev group only) no longer errors on doctest imports. Also apply pre-commit formatting fixes. Co-authored-by: Alexander Bodner <AlexBodner@users.noreply.github.com>

Keep FeatureBank/eval tests always runnable without trackers[reid]. Move model/architecture/loader coverage to test_reid_model.py, which CI runs with --extra reid instead of importorskip guards. Co-authored-by: Alexander Bodner <AlexBodner@users.noreply.github.com>

Allow DEFAULT_MODEL strict loads by building backbone-only OSNet when num_classes=0, and drop unused package exports / Reid* aliases. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop the separate _lazy helper in favor of package __getattr__, merge distance/extraction into appearance.py, and rename protocols to encoder. Co-authored-by: Cursor <cursoragent@cursor.com>

Default local pytest skips integration and slow; main CI runs slow ReID weight smokes while still excluding dataset integration. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep part 1 focused on OSNet, timm backbones, and gallery eval; FastReID MOT17 SBS moves with association wiring in the follow-up. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop part-2 scaffolding, make architecture-only loads offline, trim the public surface, and remove the unused slow-test policy. Co-authored-by: Cursor <cursoragent@cursor.com>

Integration CI collects only under tests/, and heavy ReID model tests use importorskip so missing trackers[reid] no longer needs a custom collect hook. Co-authored-by: Cursor <cursoragent@cursor.com>

Collect only the integration modules in CI/docs so heavy ReID tests are never imported when trackers[reid] is not installed. Co-authored-by: Cursor <cursoragent@cursor.com>

Remove the subprocess import guards and the dedicated CI step; optional ReID loading stays as implementation, without dedicated tests for it. Co-authored-by: Cursor <cursoragent@cursor.com>

Normalize with an eps floor like torch, raise on non-finite or shape mismatch instead of quietly ignoring bad inputs. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop silent NaN→zero and dim-mismatch→zero paths; require finite matrices and matching feature dims so association failures surface immediately. Co-authored-by: Cursor <cursoragent@cursor.com>

Explain architectures vs curated ModelCard aliases on the API page and in module docstrings so registry.py is not mistaken for the backbone factory. Co-authored-by: Cursor <cursoragent@cursor.com>

Replace the loading diagram and “common confusion” note with plain load paths and a short how-to for registering a new backbone. Co-authored-by: Cursor <cursoragent@cursor.com>

Extract query/gallery embeddings independently in ReIDEvaluator, document --extra reid for local pytest, and cover partial embedding reuse in tests. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep release notes out of this branch until the feature is ready to ship. Co-authored-by: Cursor <cursoragent@cursor.com>

Assert hand-computed Rank-1, mAP, and mINP for a one-query gallery that drops the same-(pid, camid) junk entry before scoring. Co-authored-by: Cursor <cursoragent@cursor.com>

Normalize detection rows in appearance_similarity, require max_rank >= 10, validate provided embedding lengths, and stop model importing appearance's private helper. Co-authored-by: Cursor <cursoragent@cursor.com>

Cosine association already normalizes both sides; store raw EMA features and emit raw embeddings unless normalize_embeddings / normalize is opted in. Co-authored-by: Cursor <cursoragent@cursor.com>

Keep each docstring focused on what that API does, not where L2 lives elsewhere. Co-authored-by: Cursor <cursoragent@cursor.com>

FeatureBank only stores EMA features; cosine L2 belongs with appearance_similarity. Co-authored-by: Cursor <cursoragent@cursor.com>

Prefer plain backticks like SORT/ByteTrack instead of :class:/:meth: markup. Co-authored-by: Cursor <cursoragent@cursor.com>

Short Google-style summaries, plain backticks, fuller Returns/Raises/Notes, and a leaner API page without ownership essays. Co-authored-by: Cursor <cursoragent@cursor.com>

Match eval/IO Raises style for Market-1501 and MSMT17 entry points. Co-authored-by: Cursor <cursoragent@cursor.com>

Use Sequence[str] and normalize=False so stubs structurally match the protocol. Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

Pull request overview

Copilot reviewed 29 out of 31 changed files in this pull request and generated 3 comments.

+            resolved_arch = architecture if architecture is not None else card.architecture
+            resolved_weights = card.weights
+            resolved_preprocessing = preprocessing if preprocessing is not None else card.preprocessing
+            resolved_warning = card.domain_warning


+                img = PILImage.open(p).convert("RGB")
+                crop = self._preprocessing.resize_crop(np.asarray(img))
+                tensors.append(self._transforms(PILImage.fromarray(crop)))


+    if not os.path.exists(source):
+        raise FileNotFoundError(f"Weights file not found: {source}")
+    return source


Keep part 1 scoped to the model and gallery-eval stack. FeatureBank, appearance similarity, and ReIDEncoder belong with tracker wiring. Co-authored-by: Cursor <cursoragent@cursor.com>

Replace ReIDPathEncoder with one encoder interface that covers frame and path embedding. Co-authored-by: Cursor <cursoragent@cursor.com>

AlexBodner changed the title ~~Feat/core/reid~~ feat/ReID base Jun 29, 2026

AlexBodner force-pushed the feat/core/reid branch from 40f6c3e to 4372c3a Compare June 29, 2026 14:27

Borda requested a review from Copilot July 1, 2026 21:40

Copilot started reviewing on behalf of Borda July 1, 2026 21:40 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

AlexBodner mentioned this pull request Jul 15, 2026

feat(botsort-reid): wire ReID association, CLI, and demos #499

Closed

2 tasks

cursor Bot force-pushed the feat/core/reid branch from 6625885 to d567e99 Compare July 15, 2026 13:51

cursor Bot changed the title ~~feat/ReID base~~ feat(reid): standalone ReIDModel, architectures, and gallery eval Jul 15, 2026

cursor Bot changed the title ~~feat(reid): standalone ReIDModel, architectures, and gallery eval~~ [ReID part 1] Standalone ReIDModel, architectures, and gallery eval Jul 15, 2026

AlexBodner and others added 5 commits July 15, 2026 11:40

docs(reid): tighten foundation PR for review

77e75e3

Match changelog style, narrow the public ReID exports, clarify docs as standalone embeddings/eval, and fix the eval notebook install pin. Co-authored-by: Alexander Bodner <AlexBodner@users.noreply.github.com>

fix(reid): omit OSNet classifier for inference and trim public surface

6158f70

Allow DEFAULT_MODEL strict loads by building backbone-only OSNet when num_classes=0, and drop unused package exports / Reid* aliases. Co-authored-by: Cursor <cursoragent@cursor.com>

AlexBodner force-pushed the feat/core/reid branch from 0b61cce to 6158f70 Compare July 15, 2026 14:40

AlexBodner and others added 4 commits July 15, 2026 11:44

refactor(reid): collapse slim modules into appearance and encoder

0eb5343

Drop the separate _lazy helper in favor of package __getattr__, merge distance/extraction into appearance.py, and rename protocols to encoder. Co-authored-by: Cursor <cursoragent@cursor.com>

test: replace network mark with slow and run it in main CI

630bc0b

Default local pytest skips integration and slow; main CI runs slow ReID weight smokes while still excluding dataset integration. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(reid): defer FastReID SBS architecture to BoT-SORT PR

dc9d62f

Keep part 1 focused on OSNet, timm backbones, and gallery eval; FastReID MOT17 SBS moves with association wiring in the follow-up. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

13fdd91

AlexBodner mentioned this pull request Jul 15, 2026

[ReID part 2] Wire BoT-SORT association, CLI, and demos #501

Draft

7 tasks

AlexBodner and others added 8 commits July 15, 2026 12:42

fix(reid): land part-1 review cleanups

5c4c258

Drop part-2 scaffolding, make architecture-only loads offline, trim the public surface, and remove the unused slow-test policy. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

aa3b9d9

refactor(ci): drop root conftest; ignore ReID via pytest paths

4a54f91

Integration CI collects only under tests/, and heavy ReID model tests use importorskip so missing trackers[reid] no longer needs a custom collect hook. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(ci): drop importorskip; run integration tests by path

ce1c298

Collect only the integration modules in CI/docs so heavy ReID tests are never imported when trackers[reid] is not installed. Co-authored-by: Cursor <cursoragent@cursor.com>

test(reid): drop lazy-import isolation suite

2899204

Remove the subprocess import guards and the dedicated CI step; optional ReID loading stays as implementation, without dedicated tests for it. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(reid): drop silent FeatureBank skips for zero vectors

cf0da04

Normalize with an eps floor like torch, raise on non-finite or shape mismatch instead of quietly ignoring bad inputs. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

15b5f5d

refactor(reid): raise on bad appearance embeddings instead of sanitizing

0b5e70f

Drop silent NaN→zero and dim-mismatch→zero paths; require finite matrices and matching feature dims so association failures surface immediately. Co-authored-by: Cursor <cursoragent@cursor.com>

AlexBodner and others added 17 commits July 15, 2026 13:25

docs(reid): document package layout and registry for contributors

86cc225

Explain architectures vs curated ModelCard aliases on the API page and in module docstrings so registry.py is not mistaken for the backbone factory. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

b79a1a2

docs(reid): clarify load scenarios and adding architectures

f8781a0

Replace the loading diagram and “common confusion” note with plain load paths and a short how-to for registering a new backbone. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

1902ae2

fix(reid): address part-1 review should-fix items

35b199a

Extract query/gallery embeddings independently in ReIDEvaluator, document --extra reid for local pytest, and cover partial embedding reuse in tests. Co-authored-by: Cursor <cursoragent@cursor.com>

docs: drop ReID Unreleased CHANGELOG entries from part-1 PR

eb84f04

Keep release notes out of this branch until the feature is ready to ship. Co-authored-by: Cursor <cursoragent@cursor.com>

test(reid): replace placeholder metrics test with controlled ranking

9e36e20

Assert hand-computed Rank-1, mAP, and mINP for a one-query gallery that drops the same-(pid, camid) junk entry before scoring. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(reid): make appearance cosine own both-side L2 and tighten contracts

661fd09

Normalize detection rows in appearance_similarity, require max_rank >= 10, validate provided embedding lengths, and stop model importing appearance's private helper. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

7bfae7d

fix(reid): stop default L2 in model and FeatureBank

843f1e8

Cosine association already normalizes both sides; store raw EMA features and emit raw embeddings unless normalize_embeddings / normalize is opted in. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

32ed3d2

docs(reid): drop cross-module ownership notes from docstrings

11655e8

Keep each docstring focused on what that API does, not where L2 lives elsewhere. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(reid): move L2 helper off FeatureBank into appearance

02540d7

FeatureBank only stores EMA features; cosine L2 belongs with appearance_similarity. Co-authored-by: Cursor <cursoragent@cursor.com>

docs(reid): drop Sphinx roles from ReID docstrings

879684b

Prefer plain backticks like SORT/ByteTrack instead of :class:/:meth: markup. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

7aa5ea0

docs(reid): align docstrings with Roboflow tracker house style

f289129

Short Google-style summaries, plain backticks, fuller Returns/Raises/Notes, and a leaner API page without ownership essays. Co-authored-by: Cursor <cursoragent@cursor.com>

fix(pre_commit): 🎨 auto format pre-commit hooks

677a153

AlexBodner marked this pull request as ready for review July 15, 2026 17:50

AlexBodner requested a review from SkalskiP as a code owner July 15, 2026 17:50

AlexBodner requested a review from Borda July 15, 2026 17:50

docs(reid): document FileNotFoundError on dataset loaders

af1f47c

Match eval/IO Raises style for Market-1501 and MSMT17 entry points. Co-authored-by: Cursor <cursoragent@cursor.com>

AlexBodner requested a review from Copilot July 15, 2026 17:51

Copilot started reviewing on behalf of AlexBodner July 15, 2026 17:51 View session

fix(reid): align test encoder stubs with ReIDPathEncoder for mypy

8603dea

Use Sequence[str] and normalize=False so stubs structurally match the protocol. Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot AI reviewed Jul 15, 2026

View reviewed changes

AlexBodner and others added 2 commits July 16, 2026 11:41

refactor(reid): defer tracker association utilities to BoT-SORT

e34e4df

Keep part 1 scoped to the model and gallery-eval stack. FeatureBank, appearance similarity, and ReIDEncoder belong with tracker wiring. Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(reid): use single ReIDEncoder protocol

9df1aa7

Replace ReIDPathEncoder with one encoder interface that covers frame and path embedding. Co-authored-by: Cursor <cursoragent@cursor.com>

This was referenced Jul 16, 2026

[ReID part 3] MOT17 crop fine-tuning pipeline #502

Draft

feat(reid): appearance ReID association via standalone reid package #511

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ReID part 1] Standalone ReIDModel, architectures, and gallery eval#480

[ReID part 1] Standalone ReIDModel, architectures, and gallery eval#480
AlexBodner wants to merge 38 commits into
developfrom
feat/core/reid

AlexBodner commented Jun 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Jul 15, 2026 •

edited

Loading

Uh oh!

socket-security Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

AlexBodner commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Benchmarks

Type of Change

Testing

Checklist

Additional Context

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

socket-security Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlexBodner commented Jun 29, 2026 •

edited

Loading

CLAassistant commented Jul 15, 2026 •

edited

Loading

socket-security Bot commented Jul 15, 2026 •

edited

Loading