:py:mod:`traccuracy.metrics` ============================ .. py:module:: traccuracy.metrics Package Contents ---------------- Classes ~~~~~~~ .. autoapisummary:: traccuracy.metrics.BasicMetrics traccuracy.metrics.CellCycleAccuracy traccuracy.metrics.CHOTAMetric traccuracy.metrics.CompleteTracks traccuracy.metrics.AOGMMetrics traccuracy.metrics.CTCMetrics traccuracy.metrics.DivisionMetrics traccuracy.metrics.Results traccuracy.metrics.TrackOverlapMetrics .. py:class:: BasicMetrics Generates basic statistics describing node and edge errors If `relax_skips_gt` or `relax_skips_pred` is True, we can match skip edges in the prediction to a series of edges in the gt, or vice versa. The total number of skip TPs/FNs/FPs will be reported and these counts will be incorporated in the calculation of precision/recall/F1. These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives. .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: CellCycleAccuracy The CCA metric captures the ability of a method to identify a distribution of cell cycle lengths that matches the distribution present in the ground truth. The evaluation is done on distributions and therefore does not require a matching of solution to the ground truth. It ranges from [0,1] with higher values indicating better performance. This metric is part of the biologically inspired metrics introduced by the CTC and defined in Ulman 2017. .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: CHOTAMetric Cell Higher Order Tracking Accuracy. https://arxiv.org/pdf/2408.11571 Reference implementation: https://github.com/CellTrackingChallenge/py-ctcmetrics/blob/main/ctc_metrics/metrics/hota/chota.py .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: CompleteTracks(error_type: str = 'basic') The fraction of tracklets and lineages that are completely correctly reconstructed. If the reconstruction continues beyond the ground truth track, this is NOT counted as incorrect, nor are false positive tracks penalized, making this suitable for evaluating with sparse ground truth annotations. If a False Positive Division occurs within the ground truth track (or, for the CTC errors, a wrong semantic edge), this IS counted as incorrect. :param error_type: Whether to use "basic" or "ctc" errors for computing if tracks are correct or not. Defaults to "basic". :type error_type: str, optional The compute function returns a results dictionary with the following entries: - `total_lineages` - the number of connected components in the ground truth graph - `correct_lineages` - the number of fully correct connected components - `complete_lineages` - `correct_lineages` / `total_lineages`, or np.nan if `total_lineages` is 0 - `total_tracklets` - the number of tracklets in the ground truth graph, defined as the connected components of the graph after division edges are removed. Division edges are not included in the tracklets, or counted at all in the tracklet metrics. - `correct_tracklets` - the number of fully correct tracklets - `complete_tracklets` - `correct_tracklets` / `total_tracklets`, or np.nan if `total_tracklets` is 0 .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: AOGMMetrics(vertex_ns_weight: float = 1, vertex_fp_weight: float = 1, vertex_fn_weight: float = 1, edge_fp_weight: float = 1, edge_fn_weight: float = 1, edge_ws_weight: float = 1) Computes the Acyclic Oriented Graph Measure (AOGM), along with the error counts The AOGM metric is a generalized graph measure that allows users to define their own error weights for each type of node and edge error. The AOGM is simply the weighted sum of all errors. These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives. :param vertex_ns_weight: Weight for vertex/node non-split errors. Defaults to 1 :type vertex_ns_weight: float :param vertex_fp_weight: Weight for false positive vertex/node errors. Defaults to 1 :type vertex_fp_weight: float :param vertex_fn_weight: Weight for false negative vertex/node errors. Defaults to 1 :type vertex_fn_weight: float :param edge_fp_weight: Weight for false positive edge errors. Defaults to 1 :type edge_fp_weight: float :param edge_fn_weight: Weight for false negative edge errors. Defaults to 1 :type edge_fn_weight: float :param edge_ws_weight: Weight for wrong semantic edge errors. Defaults to 1 :type edge_ws_weight: float .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: CTCMetrics Computes the original Cell Tracking Challenging metrics: TRA, DET, LNK. These metrics are based on the more general AOGM metric. - DET: Assesses detection performance - LNK: Assesses linking performance by measuring only edge errors - TRA: Assesses both detection and tracking performance These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives. .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: DivisionMetrics(max_frame_buffer: int = 0, zero_division: float = np.nan) Computes division summary metrics with an optional frame tolerance. Computes the following metrics: - Division Recall - Division Precision - Division F1 Score (also Branching Correctness) - Mitotic Branching Correctness: TP / (TP + FP + FN) as defined by *Ulicna, K., Vallardi, G., Charras, G. & Lowe, A. R. Automated deep lineage tree analysis using a Bayesian single cell tracking approach. Frontiers in Computer Science 3, 734559 (2021).* These metrics are written assuming that the ground truth annotations are dense. If that is not the case, interpret the numbers carefully. Consider eliminating metrics that use the number of false positives. :param max_frame_buffer: Maximum value of frame buffer to use in correcting shifted divisions. Divisions will be evaluated for all integer values of frame buffer between 0 and max_frame_buffer :type max_frame_buffer: int, optional :param zero_division: Value to return for metrics that result in a 0/0 division. Defaults to np.nan. Set to 0.0 to return 0 and raise a warning instead, similar to scikit-learn's ``zero_division`` parameter. :type zero_division: float, optional .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results .. py:class:: Results(results: dict, matcher_info: dict | None, metric_info: dict, gt_info: dict | None = None, pred_info: dict | None = None) The Results object collects information about the pipeline used to generate the metric results :param results: Dictionary with metric output :type results: dict :param matcher_info: Dictionary with matcher name and parameters :type matcher_info: dict :param metric_info: Dictionary with metric name and parameters :type metric_info: dict :param gt_info: Dictionary with ground truth graph info (name, border_margin, etc.) :type gt_info: dict :param pred_info: Dictionary with predicted graph info (name, border_margin, etc.) :type pred_info: dict .. py:property:: version :type: str Return current traccuracy version .. py:method:: to_dict() -> dict[str, Any] Returns all attributes that are not None as a dictionary :returns: Dictionary of Results attributes :rtype: dict .. py:class:: TrackOverlapMetrics(include_division_edges: bool = True) Calculate metrics for longest track overlaps. - Target Effectiveness: fraction of longest overlapping prediction tracklets on each GT tracklet - Track Purity : fraction of longest overlapping GT tracklets on each prediction tracklet :param matched_data: Matched object for set of GT and Pred data :type matched_data: traccuracy.matchers.Matched :param include_division_edges: If True, include edges at division. :type include_division_edges: bool, optional .. py:property:: info :type: dict[str, Any] Dictionary with Metric name and any parameters .. py:method:: compute(matched: traccuracy.matchers._matched.Matched, override_matcher: bool = False, relax_skips_gt: bool = False, relax_skips_pred: bool = False) -> traccuracy.metrics._results.Results The compute methods of Metric objects return a Results object populated with results and associated metadata :param matched: Matched data object to compute metrics on :type matched: traccuracy.matchers.Matched :param override_matcher: If True, the metric will not validate the matcher type :type override_matcher: bool :param relax_skips_gt: If True, the metric will check if skips in the ground truth graph have an equivalent multi-edge path in predicted graph :type relax_skips_gt: bool :param relax_skips_pred: If True, the metric will check if skips in the predicted graph have an equivalent multi-edge path in ground truth graph :type relax_skips_pred: bool :returns: Object containing metric results and associated pipeline metadata :rtype: traccuracy.metrics._results.Results