traccuracy.metrics

Package Contents

Classes

BasicMetrics	Generates basic statistics describing node and edge errors
CellCycleAccuracy	The CCA metric captures the ability of a method to identify a distribution of cell
CHOTAMetric	Cell Higher Order Tracking Accuracy.
CompleteTracks	The fraction of tracklets and lineages that are completely correctly reconstructed.
CompleteTracksByLength	Fraction of fully correct tracks as a function of track length.
AOGMMetrics	Computes the Acyclic Oriented Graph Measure (AOGM), along with the error counts
CTCMetrics	Computes the original Cell Tracking Challenging metrics: TRA, DET, LNK.
DivisionMetrics	Computes division summary metrics with an optional frame tolerance.
Results	The Results object collects information about the pipeline used
TrackOverlapMetrics	Calculate metrics for longest track overlaps.

class traccuracy.metrics.BasicMetrics

Generates basic statistics describing node and edge errors

If relax_skips_gt or relax_skips_pred is True, we can match skip edges in the prediction to a series of edges in the gt, or vice versa. The total number of skip TPs/FNs/FPs will be reported and these counts will be incorporated in the calculation of precision/recall/F1.

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CellCycleAccuracy

The CCA metric captures the ability of a method to identify a distribution of cell cycle lengths that matches the distribution present in the ground truth. The evaluation is done on distributions and therefore does not require a matching of solution to the ground truth. It ranges from [0,1] with higher values indicating better performance.

This metric is part of the biologically inspired metrics introduced by the CTC and defined in Ulman 2017.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CHOTAMetric

Cell Higher Order Tracking Accuracy. https://arxiv.org/pdf/2408.11571

Reference implementation: https://github.com/CellTrackingChallenge/py-ctcmetrics/blob/main/ctc_metrics/metrics/hota/chota.py

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CompleteTracks(error_type: str = 'basic')

The fraction of tracklets and lineages that are completely correctly reconstructed.

If the reconstruction continues beyond the ground truth track, this is NOT counted as incorrect, nor are false positive tracks penalized, making this suitable for evaluating with sparse ground truth annotations.

If a False Positive Division occurs within the ground truth track (or, for the CTC errors, a wrong semantic edge), this IS counted as incorrect.

Parameters:

error_type (str, optional) – Whether to use “basic” or “ctc” errors for computing if tracks are correct or not. Defaults to “basic”.

The compute function returns a results dictionary with the following entries:

• total_lineages - the number of connected components in the ground truth graph

• correct_lineages - the number of fully correct connected components

• complete_lineages - correct_lineages / total_lineages, or np.nan if total_lineages is 0

• total_tracklets - the number of tracklets in the ground truth graph, defined as the connected components of the graph after division edges are removed. Division edges are not included in the tracklets, or counted at all in the tracklet metrics.

• correct_tracklets - the number of fully correct tracklets

• complete_tracklets - correct_tracklets / total_tracklets, or np.nan if total_tracklets is 0

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CompleteTracksByLength(max_length: int | None = None, lineages: bool = True, error_type: Literal[basic, ctc] = 'basic')

Fraction of fully correct tracks as a function of track length.

This is the CTC-BIO Complete Tracks metric generalized to every track length: for each length from 1 to max_length (in frames), it computes the accuracy (number correct / number total) of the ground truth track segments that span that many frames. At the maximum length this reduces to CompleteTracks.

Length is measured in frames (time difference), not edge count. A segment of length N spans N frames from start to end. For example:

• A segment of length 1 spans 1 frame (node at t=0 to node at t=1)

• A segment of length 2 spans 2 frames (node at t=0 to node at t=2)

Skip edges that span multiple frames are decomposed into individual single-frame edges, each carrying the skip edge’s correctness. For example, a skip edge from t=0 to t=3 contributes three length-1 segments (t=0->1, t=1->2, t=2->3), and likewise contributes to the length-2 and length-3 totals.

At division points, all branches are included in the same segment - a segment is only correct if all branches are correct.

A segment is counted as correct if:

• The starting node is a true positive

• All edges along the path are true positives

Important counting rules:

• Isolated nodes (nodes with no outgoing edges) are NOT counted

• Skip edges are decomposed into single-frame edges, so they contribute at every length they span (see above)

• Tracks shorter than length N still contribute 1 to the total for length N (correct iff the entire track is correct)

This metric helps identify whether tracking errors occur more frequently in short or long tracks, providing granular insight into tracking quality at different temporal scales.

Parameters:

• max_length (int | None) – Maximum track length in frames to evaluate. The default is None, which uses gt_tracks.end - gt_tracks.start

• lineages – If True, evaluate on full lineages (connected components). If False, evaluate on tracklets (segments between divisions).

• error_type – “basic” or “ctc” error classification scheme

The compute function returns a results dictionary with three lists, each indexed by track length (index 0 = length 1, index 1 = length 2, etc.):

• correct - number of correct segments at each length

• total - total number of segments at each length

• accuracy - correct/total at each length, or np.nan if total is 0

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.AOGMMetrics(vertex_ns_weight: float = 1, vertex_fp_weight: float = 1, vertex_fn_weight: float = 1, edge_fp_weight: float = 1, edge_fn_weight: float = 1, edge_ws_weight: float = 1)

Computes the Acyclic Oriented Graph Measure (AOGM), along with the error counts

The AOGM metric is a generalized graph measure that allows users to define their own error weights for each type of node and edge error. The AOGM is simply the weighted sum of all errors.

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

Parameters:

• vertex_ns_weight (float) – Weight for vertex/node non-split errors. Defaults to 1

• vertex_fp_weight (float) – Weight for false positive vertex/node errors. Defaults to 1

• vertex_fn_weight (float) – Weight for false negative vertex/node errors. Defaults to 1

• edge_fp_weight (float) – Weight for false positive edge errors. Defaults to 1

• edge_fn_weight (float) – Weight for false negative edge errors. Defaults to 1

• edge_ws_weight (float) – Weight for wrong semantic edge errors. Defaults to 1

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CTCMetrics

Computes the original Cell Tracking Challenging metrics: TRA, DET, LNK. These metrics are based on the more general AOGM metric.

• DET: Assesses detection performance

• LNK: Assesses linking performance by measuring only edge errors

• TRA: Assesses both detection and tracking performance

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.DivisionMetrics(max_frame_buffer: int = 0, zero_division: float = np.nan)

Computes division summary metrics with an optional frame tolerance.

Computes the following metrics: - Division Recall - Division Precision - Division F1 Score (also Branching Correctness) - Mitotic Branching Correctness: TP / (TP + FP + FN) as defined by Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Frontiers in Computer Science 3, 734559 (2021).

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

Parameters:

• max_frame_buffer (int, optional) – Maximum value of frame buffer to use in correcting shifted divisions. Divisions will be evaluated for all integer values of frame buffer between 0 and max_frame_buffer

• zero_division (float, optional) – Value to return for metrics that result in a 0/0 division. Defaults to np.nan. Set to 0.0 to return 0 and raise a warning instead, similar to scikit-learn’s zero_division parameter.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.Results(results: dict, matcher_info: dict | None, metric_info: dict, gt_info: dict | None = None, pred_info: dict | None = None)

The Results object collects information about the pipeline used to generate the metric results

Parameters:

• results (dict) – Dictionary with metric output

• matcher_info (dict) – Dictionary with matcher name and parameters

• metric_info (dict) – Dictionary with metric name and parameters

• gt_info (dict) – Dictionary with ground truth graph info (name, border_margin, etc.)

• pred_info (dict) – Dictionary with predicted graph info (name, border_margin, etc.)

property version: str

Return current traccuracy version

to_dict() → dict[str, Any]

Returns all attributes that are not None as a dictionary

Returns:

Dictionary of Results attributes

Return type:

dict

class traccuracy.metrics.TrackOverlapMetrics(include_division_edges: bool = True)

Calculate metrics for longest track overlaps.

• Target Effectiveness: fraction of longest overlapping prediction

  tracklets on each GT tracklet

• Track Purityfraction of longest overlapping GT

  tracklets on each prediction tracklet

Parameters:

• matched_data (traccuracy.matchers.Matched) – Matched object for set of GT and Pred data

• include_division_edges (bool, optional) – If True, include edges at division.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

• matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

• override_matcher (bool) – If True, the metric will not validate the matcher type

• relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

• relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results