traccuracy.metrics

Package Contents

Classes

BasicMetrics

Generates basic statistics describing node and edge errors

CellCycleAccuracy

The CCA metric captures the ability of a method to identify a distribution of cell

CHOTAMetric

Cell Higher Order Tracking Accuracy.

CompleteTracks

The fraction of tracklets and lineages that are completely correctly reconstructed.

AOGMMetrics

Computes the Acyclic Oriented Graph Measure (AOGM), along with the error counts

CTCMetrics

Computes the original Cell Tracking Challenging metrics: TRA, DET, LNK.

DivisionMetrics

Computes division summary metrics with an optional frame tolerance.

Results

The Results object collects information about the pipeline used

TrackOverlapMetrics

Calculate metrics for longest track overlaps.
class traccuracy.metrics.BasicMetrics[source]

Generates basic statistics describing node and edge errors

If relax_skips_gt or relax_skips_pred is True, we can match skip edges in the prediction to a series of edges in the gt, or vice versa. The total number of skip TPs/FNs/FPs will be reported and these counts will be incorporated in the calculation of precision/recall/F1.

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CellCycleAccuracy[source]

The CCA metric captures the ability of a method to identify a distribution of cell cycle lengths that matches the distribution present in the ground truth. The evaluation is done on distributions and therefore does not require a matching of solution to the ground truth. It ranges from [0,1] with higher values indicating better performance.

This metric is part of the biologically inspired metrics introduced by the CTC and defined in Ulman 2017.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CHOTAMetric[source]

Cell Higher Order Tracking Accuracy. https://arxiv.org/pdf/2408.11571

Reference implementation: https://github.com/CellTrackingChallenge/py-ctcmetrics/blob/main/ctc_metrics/metrics/hota/chota.py

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CompleteTracks(error_type: str = 'basic')[source]

The fraction of tracklets and lineages that are completely correctly reconstructed.

If the reconstruction continues beyond the ground truth track, this is NOT counted as incorrect, nor are false positive tracks penalized, making this suitable for evaluating with sparse ground truth annotations.

If a False Positive Division occurs within the ground truth track (or, for the CTC errors, a wrong semantic edge), this IS counted as incorrect.

Parameters:

error_type (str, optional) – Whether to use “basic” or “ctc” errors for computing if tracks are correct or not. Defaults to “basic”.

The compute function returns a results dictionary with the following entries:

  • total_lineages - the number of connected components in the ground truth graph

  • correct_lineages - the number of fully correct connected components

  • complete_lineages - correct_lineages / total_lineages, or np.nan if total_lineages is 0

  • total_tracklets - the number of tracklets in the ground truth graph, defined as the connected components of the graph after division edges are removed. Division edges are not included in the tracklets, or counted at all in the tracklet metrics.

  • correct_tracklets - the number of fully correct tracklets

  • complete_tracklets - correct_tracklets / total_tracklets, or np.nan if total_tracklets is 0

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.AOGMMetrics(vertex_ns_weight: float = 1, vertex_fp_weight: float = 1, vertex_fn_weight: float = 1, edge_fp_weight: float = 1, edge_fn_weight: float = 1, edge_ws_weight: float = 1)[source]

Computes the Acyclic Oriented Graph Measure (AOGM), along with the error counts

The AOGM metric is a generalized graph measure that allows users to define their own error weights for each type of node and edge error. The AOGM is simply the weighted sum of all errors.

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

Parameters:

  • vertex_ns_weight (float) – Weight for vertex/node non-split errors. Defaults to 1

  • vertex_fp_weight (float) – Weight for false positive vertex/node errors. Defaults to 1

  • vertex_fn_weight (float) – Weight for false negative vertex/node errors. Defaults to 1

  • edge_fp_weight (float) – Weight for false positive edge errors. Defaults to 1

  • edge_fn_weight (float) – Weight for false negative edge errors. Defaults to 1

  • edge_ws_weight (float) – Weight for wrong semantic edge errors. Defaults to 1

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.CTCMetrics[source]

Computes the original Cell Tracking Challenging metrics: TRA, DET, LNK. These metrics are based on the more general AOGM metric.

  • DET: Assesses detection performance

  • LNK: Assesses linking performance by measuring only edge errors

  • TRA: Assesses both detection and tracking performance

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.DivisionMetrics(max_frame_buffer: int = 0, zero_division: float = np.nan)[source]

Computes division summary metrics with an optional frame tolerance.

Computes the following metrics: - Division Recall - Division Precision - Division F1 Score (also Branching Correctness) - Mitotic Branching Correctness: TP / (TP + FP + FN) as defined by Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Frontiers in Computer Science 3, 734559 (2021).

These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives.

Parameters:

  • max_frame_buffer (int, optional) – Maximum value of frame buffer to use in correcting shifted divisions. Divisions will be evaluated for all integer values of frame buffer between 0 and max_frame_buffer

  • zero_division (float, optional) – Value to return for metrics that result in a 0/0 division. Defaults to np.nan. Set to 0.0 to return 0 and raise a warning instead, similar to scikit-learn’s zero_division parameter.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results

class traccuracy.metrics.Results(results: dict, matcher_info: dict | None, metric_info: dict, gt_info: dict | None = None, pred_info: dict | None = None)[source]

The Results object collects information about the pipeline used to generate the metric results

Parameters:

  • results (dict) – Dictionary with metric output

  • matcher_info (dict) – Dictionary with matcher name and parameters

  • metric_info (dict) – Dictionary with metric name and parameters

  • gt_info (dict) – Dictionary with ground truth graph info (name, border_margin, etc.)

  • pred_info (dict) – Dictionary with predicted graph info (name, border_margin, etc.)

property version: str

Return current traccuracy version

to_dict() → dict[str, Any][source]

Returns all attributes that are not None as a dictionary

Returns:

Dictionary of Results attributes

Return type:

dict

class traccuracy.metrics.TrackOverlapMetrics(include_division_edges: bool = True)[source]

Calculate metrics for longest track overlaps.

  • Target Effectiveness: fraction of longest overlapping prediction

    tracklets on each GT tracklet

  • Track Purityfraction of longest overlapping GT

    tracklets on each prediction tracklet

Parameters:

  • matched_data (traccuracy.matchers.Matched) – Matched object for set of GT and Pred data

  • include_division_edges (bool, optional) – If True, include edges at division.

property info: dict[str, Any]

Dictionary with Metric name and any parameters

compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) → traccuracy.metrics._results.Results

The compute methods of Metric objects return a Results object populated with results and associated metadata

Parameters:

  • matched (traccuracy.matchers.Matched) – Matched data object to compute metrics on

  • override_matcher (bool) – If True, the metric will not validate the matcher type

  • relax_skips_gt (bool) – If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph

  • relax_skips_pred (bool) – If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph

Returns:

Object containing metric results

and associated pipeline metadata

Return type:

traccuracy.metrics._results.Results