By Auguste Baum, Open Source Engineer at Probabl
At Probabl, we are developing skore lib, a Python library that wraps your data science pipelines to reduce boilerplate and guard against classic pitfalls. Before I dive into the latest technical updates, I’d like to say some words about why we're building skore lib.
Notebooks (Jupyter and others) let you run code interactively, in any order. That flexibility has a cost: you can train a model in a cell, delete that cell, and the model object persists in memory. Five cells later, you might have forgotten which data the model was trained with.
This is where methodology meets tool design. skore lib's EstimatorReport enforces a simple practice: it keeps the estimator and its training data together. You can't separate them, so you can't lose the link.
That design choice brings several benefits. Since the data and estimator are bundled together, it is possible to automate model evaluation (e.g. classic metrics and plots). For users, this means fewer lines of code, which leaves fewer places for bugs.
Similarly, it is common to store results to disk or remote storage to counteract the volatility of notebooks. skore lib helps with this by exposing a simple API with different backends, including disk storage, MLFlow instances and Skore Hub, Probabl's data science platform.
Good practices in data science go beyond "save every model properly". Checking for overfitting, spotting correlated features, validating on holdout data: these are patterns that recur but get implemented from scratch each time. These checks are built into skore lib, so the methodology becomes the default path instead of extra work.
This applies to human data scientists and to agents. An agent using skore inherits the same checks and organization without needing separate instructions for each project. The tool carries the methodology with it.
Without further ado, here are the latest updates in skore lib. They can also be found in skore lib’s official documentation.
The default format of report.metrics.summarize().frame() is now long-format rather than wide-format. It is now easier to retrieve data from the default DataFrame since its index is no longer a pandas MultiIndex. See #3094 by @glemaitre.
report.metrics.summarize() no longer aborts the whole computation if one of the metrics raised (e.g. if a custom scorer encounters an edge case). Instead, exceptions are caught and shown as warnings, and the output DataFrame has NaN for failed metrics. See #3124 by @auguste-probabl.
Metrics added via report.metrics.add(...) were invisible to report.metrics.help(), since they are defined dynamically. They are now listed alongside the built-ins. See #3139 by @auguste-probabl.
skore used to always include a score metric in the summary sklearn Pipeline estimators. It now inspects the Pipeline's last step to recover the default score. See #3110 by @jeromedockes.
CrossValidationReport.metrics.prediction_error(data_source="both") has worked for a long time, but it was not documented, and in fact its legend was broken. This is now fixed. See #3154 by @direkkakkar319-ops.
Computing a summary with process-based parallelism (n_jobs > 1) used to trigger infinite recursion. See #3178 by @glemaitre.
Appropriate dummy/fast/performance baselines have been added to support this ML task. See #3116 by @GaetandeCast.
When a sklearn Pipeline's preprocessing output a plain numpy array, the data-quality checks used generic column labels in their explanations. Column names are now retrieved from the pipeline's final step, so check explanations are clearer. See #3155 by @GaetandeCast.
put() no longer computes SVGs for TableReport plots, thanks to improvements in skrub 0.10. This makes put() up to 35% faster. Note that the minimum supported version of skrub is now 0.10. See #3138 by @auguste-probabl.
Listing reports from the Hub now returns all reports, rather than the 500 first ones. See #3125 and #3164 by @thomass-dev.
Thanks to the following contributors for their work this month (in no particular order):