Metrics

Metrics score model predictions against ground truth inside a benchmark. Summaries below are taken from each class docstring.

Area under the precision-recall curve.

class bioverse.metrics.auprc.AuprcMetric(name='AUPRC', **kwargs)

Bases: Metric

Area under the precision-recall curve.

Area under the receiver operating characteristic curve.

class bioverse.metrics.auroc.AurocMetric(name='AUROC', **kwargs)

Bases: Metric

Area under the receiver operating characteristic curve.

Balanced accuracy for binary classification.

class bioverse.metrics.balanced_binary_accuracy.BalancedBinaryAccuracyMetric(name='Bal.Acc.', threshold=0.5, **kwargs)

Bases: Metric

Balanced accuracy for binary classification.

Balanced accuracy averaged over classes.

class bioverse.metrics.balanced_multi_class_accuracy.BalancedMultiClassAccuracyMetric(name='Bal.Acc.', **kwargs)

Bases: Metric

Balanced accuracy averaged over classes.

Accuracy for binary classification tasks.

class bioverse.metrics.binary_accuracy.BinaryAccuracyMetric(name='Accuracy', threshold=0.5, **kwargs)

Bases: Metric

Accuracy for binary classification tasks.

Average BLOSUM62 substitution score between true and predicted residues.

class bioverse.metrics.blosum_score.BlosumScoreMetric(name: str = 'Blosum Score', on: int = 2, per: int = 1, **kwargs)

Bases: Metric

Average BLOSUM62 substitution score between true and predicted residues.

Coefficient of determination (R²).

class bioverse.metrics.coefficient_of_determination.CoefficientOfDeterminationMetric(name='R2', **kwargs)

Bases: Metric

Coefficient of determination (R²).

Enrichment factor at a fixed false-positive rate.

class bioverse.metrics.enrichment_factor.EnrichmentFactorMetric(name='Enrichment Factor', cutoff_fraction=0.2, **kwargs)

Bases: Metric

Enrichment factor at a fixed false-positive rate.

F1 score (harmonic mean of precision and recall).

class bioverse.metrics.f1_score.F1ScoreMetric(name='Accuracy', **kwargs)

Bases: Metric

F1 score (harmonic mean of precision and recall).

Maximum F1 score over classification thresholds.

class bioverse.metrics.fmax.FmaxMetric(name='Fmax', **kwargs)

Bases: Metric

Maximum F1 score over classification thresholds.

Macro-averaged precision over classes.

class bioverse.metrics.macro_precision.MacroPrecisionMetric(name='Precision', **kwargs)

Bases: Metric

Macro-averaged precision over classes.

Macro-averaged recall over classes.

class bioverse.metrics.macro_recall.MacroRecallMetric(name='Recall', **kwargs)

Bases: Metric

Macro-averaged recall over classes.

Mean absolute error between predictions and targets.

class bioverse.metrics.mean_absolute_error.MeanAbsoluteErrorMetric(name='MAE', **kwargs)

Bases: Metric

Mean absolute error between predictions and targets.

Mean rank of actives in virtual screening.

class bioverse.metrics.mean_active_rank.MeanActiveRankMetric(name='Mean Active Rank', **kwargs)

Bases: Metric

Mean rank of actives in virtual screening.

Mean angular error between predicted and true vectors.

class bioverse.metrics.mean_angular_error.MeanAngularErrorMetric(name='Angular MAE', **kwargs)

Bases: Metric

Mean angular error between predicted and true vectors.

Mean squared error between predictions and targets.

class bioverse.metrics.mean_squared_error.MeanSquaredErrorMetric(name='MSE', **kwargs)

Bases: Metric

Mean squared error between predictions and targets.

Multi-class classification accuracy (fraction of argmax-correct predictions).

class bioverse.metrics.multi_class_accuracy.MultiClassAccuracyMetric(name='Accuracy', **kwargs)

Bases: Metric

Multi-class classification accuracy (fraction of argmax-correct predictions).

Base class for metrics that compare predicted and true class indices via argmax. Subclassed by RecoveryMetric.

Exact-match accuracy for multi-label classification.

class bioverse.metrics.multi_label_accuracy.MultiLabelAccuracyMetric(name='Accuracy', threshold=0.5, **kwargs)

Bases: Metric

Exact-match accuracy for multi-label classification.

Pearson correlation coefficient between predictions and targets.

class bioverse.metrics.pearsons_r.PearsonsRMetric(name='Pearson', **kwargs)

Bases: Metric

Pearson correlation coefficient between predictions and targets.

Perplexity of a language-model distribution.

class bioverse.metrics.perplexity.PerplexityMetric(name='Perplexity', **kwargs)

Bases: Metric

Perplexity of a language-model distribution.

Precision (positive predictive value).

class bioverse.metrics.precision.PrecisionMetric(name='Precision', **kwargs)

Bases: Metric

Precision (positive predictive value).

Recall (true positive rate).

class bioverse.metrics.recall.RecallMetric(name='Recall', **kwargs)

Bases: Metric

Recall (true positive rate).

Sequence recovery rate (fraction of correctly predicted residues).

class bioverse.metrics.recovery.RecoveryMetric(name='Recovery', on=2, per=1, **kwargs)

Bases: MultiClassAccuracyMetric

Sequence recovery rate (fraction of correctly predicted residues).

Spearman rank correlation between predictions and targets.

class bioverse.metrics.spearmans_rho.SpearmansRhoMetric(name='Spearman', **kwargs)

Bases: Metric

Spearman rank correlation between predictions and targets.

Top-k classification accuracy.

class bioverse.metrics.top_k_accuracy.TopKAccuracyMetric(name='Accuracy', k=10, **kwargs)

Bases: Metric

Top-k classification accuracy.

Recovery metric that gives credit when the predicted residue is within the top-k BLOSUM62 substitutions for the true residue.

class bioverse.metrics.topk_recovery.TopkRecoveryMetric(name: str = 'Top-k Recovery', k: int = 1, on: int = 2, per: int = 1, **kwargs)

Bases: Metric

Recovery metric that gives credit when the predicted residue is within the top-k BLOSUM62 substitutions for the true residue.

Ranking is based on BLOSUM62 scores per true residue, with ties sharing the same rank (i.e. identical scores are treated as the same rank).

For k=1 this reduces to standard Recovery, since the true residue has the highest BLOSUM62 score with itself in the 20×20 amino-acid sub-matrix.

__init__(name: str = 'Top-k Recovery', k: int = 1, on: int = 2, per: int = 1, **kwargs)

Parameters:

• k (int) – Number of BLOSUM-based substitution ranks to treat as correct. k=1 is equivalent to standard Recovery (exact match).

• on – Passed through to the base Metric to match RecoveryMetric behaviour (defaults: on=2, per=1).

• per – Passed through to the base Metric to match RecoveryMetric behaviour (defaults: on=2, per=1).

compute(y_true: Array, y_pred: Array)

y_true: integer residue tokens in PROTEIN_ALPHABET order. y_pred: logits or probabilities over the same alphabet.