By Adrin Jalali, VP of Labs and core maintainer of scikit-learn
At Probabl, we have a significant support for scikit-learn’s maintenance and development, and it is important to us to communicate our priorities for the broader community to know where our activities will be focused in the short term and what to expect.
We also aspire to maintain an up to date project board for each of these topics to keep track of the progress.
Priorities
1. GPU Support via Array API
- Moving from pure numpy to Array API enables using a diverse set of hardware (including GPUs) as well as native support of computations done by different backends such as pytorch or cupy. This has been a long running project, and many estimators and functions in scikit-learn already support this, but there’s still a lot of work to be done.
- Project board: https://github.com/orgs/scikit-learn/projects/12
- This work is also supported by the NASA ROSES grant.
2. Callbacks
- This work enables progress reports notably in estimators such as
GridSearchCV as well as inspection of estimators as they go through their iterative training process. We will also work on the related SLEP to get the required consensus and move this forward.
- Project board: https://github.com/orgs/scikit-learn/projects/8/views/2
- This work is also supported by a CZI-Wellcome Trust grant
3. Tree based models
- Tree based models are some of our most used estimators, and it’s important that we give the best we can to our users when it comes to these models. For this, we’d work on a variety of issues to improve them, e.g. merging Hist Gradient Boosting with Gradient Boosting.
- Project board: https://github.com/orgs/scikit-learn/projects/26/views/1
4. Displays and UX
5. Metadata Routing
- This is another long running project, which is already in a shape which enables many common usecases, however, there are areas to improve before it can become the default in the library.
- Project board: https://github.com/orgs/scikit-learn/projects/4
6. Misc / Maintenance / Release
Other areas where we keep our activity include:
- Project maintenance: it’s always crucial to maintain the project and enable other contributors to move forward their projects and we dedicate a fair amount of resources to this area.
- Free-threaded is an area supported by the NASA ROSES grant which includes maintenance of the build, as well as identifying thread safety or oversubscription issues.
- Supply chain security is an area also supported by the NASA ROSES grant, which can result in some CI refactoring and improvements in our build process.
Labs @Probabl Project Board
We also have a board to keep track of https://github.com/orgs/probabl-ai/projects/8/views/1 to view all active issues in our Labs team. Internally we assign a “champion” to each issue or pull request, which means that person is either the author or follows up on the work and makes sure the work moves forward. Whenever necessary, we also assign reviewer 1 and reviewer 2, if that’s lacking.
People mentioned in that board as a champion or a reviewer are either folks at Probabl or work very closely with us.
Note that the board includes all work done by our team on public repositories, which means not every entry is from the scikit-learn repo. Some entries are from other open source projects we support, such as skore-lib, skrub, and skops.
For more from Probabl