compute_performance_scores

improvelib.utils.compute_performance_scores(ytrue, y_pred, stage, metric_type, output_dir, y_prob = None)

Evaluate predictions according to specified metrics. Scores are stored in specified path and returned.

Used in train and infer.

Parameters:

y_truenp.array

Array with ground truth values.

y_prednp.array

Array with model predictions.

stagestr

Specify if evaluation is with respect to val or test set (‘val’, or ‘test’).

metric_typestr

Either ‘classification’ or ‘regression’.

output_dirstr

The output directory where the results should be saved. Should be params['output_dir'].

y_probnp.array, optional

Array with target scores from classification model, defaults to None.

Returns:

scoresdict

Metrics evaluated and corresponding scores.

Example

To evaluate validation predictions in train:

val_scores = frm.compute_performance_scores(
    y_true=val_true,
    y_pred=val_pred,
    stage="val",
    metric_type=params["metric_type"],
    output_dir=params["output_dir"]
)

When evaluating inference predictions in infer, best practice is to wrap the call to compute_performance_scores() in if params["calc_infer_scores"]: so that scores are only calculated when ground truth is available:

if params["calc_infer_scores"]:
    test_scores = frm.compute_performance_scores(
        y_true=test_true,
        y_pred=test_pred,
        stage="test",
        metric_type=params["metric_type"],
        output_dir=params["output_dir"]
    )