By Camilo Pedraza Jiménez, Applied Data Scientist at Probabl
& Cailean Osborne, Head of Ecosystem Development at Probabl

Introducing Probabl’s data deep dives

This is the first post in a series of data deep dives into trends in agentic data science. We start with a question that’s puzzling us, we share data-driven insights, and we end with new questions for the community.

In this first post, we take a look at the download boom happening in the Python data science ecosystem. Our guess: AI is driving it. Does the data support it?

TL;DR

The downloads of scikit-learn are skyrocketing. In the last year alone, scikit-learn was downloaded over 2 billion times – that’s 63 downloads per second and a 93% increase from the year before. What’s driving this download boom? Many factors could be at play (see caveats below), but our hypothesis is that AI is driving it, given that data scientists are increasingly adopting AI coding agents in their workflows. To test this hypothesis, we deep-dived into the download data of 40 popular Python libraries used in data science, covering scientific Python, machine learning, deep learning, and MLOps. Across the board, the download curves have been growing since AI coding agents entered the market. Specifically, the compound monthly growth rate of downloads increased from around 2.4% in the 16 months prior to the release of Claude Code to over 5.4% in the 16 months since its release. Furthermore, the correlations between the downloads of these Python libraries and the Python SDKs of five market-leading AI agents are not only statistically significant but also strengthening over time. Correlation is of course not causation, but these findings indicate that core Python libraries for data science like scikit-learn are being used more than ever in the age of agentic AI. We conclude with open questions about trends in agentic data science.

Our 3 data-driven observations

Observation 1: The downloads of Python libraries for data science are booming

Scikit-learn has been downloaded over 2 billion times in the last year. That’s on average 63 downloads per second and a 93% increase from the year before. That’s astonishing. But is this a one-off story for scikit-learn or a trend across the Python data science ecosystem?

We took a look at the download curves of 40 popular Python libraries used in data science in the last 5 years, spanning Scientific Python, Machine Learning, Deep Learning, and MLOps and Explainability. To do so, we used download data from PyPI, the official repository for Python packages, which are widely used as a proxy for package adoption and popularity (see caveats below). We selected the 40 libraries by identifying 10 libraries with the most PyPi downloads in the four predefined categories.

Figure 1 shows that the scikit-learn download boom isn’t strictly speaking an exception, but it isn’t a general tendency either. The blue dotted line shows that scikit-learn is in a league of its own in terms of downloads when you compare it with popular libraries in the machine learning, deep learning, and MLOps categories. One explanation may be the versatility of scikit-learn as a wide-ranging toolbox that includes over 200 algorithms for supervised and unsupervised learning as well as foundational tools for data preprocessing, model selection, model fitting, and model evaluation.

Figure 1: PyPi downloads of Python libraries used in Data Science

We zoomed into 10 popular Python libraries that show similar trends albeit on different orders of magnitude: scikit-learn, NumPy, pandas, Matplotlib, XGBoost, CatBoost, MLflow, LightGBM, SciPy, and SHAP. For analytical purposes, we added markers for the start of the LLM era (from the ChatGPT launch in November 2022), the LLM coding capability era (from GPT-4o in May 2024), and the agentic coding era (from the Claude Code preview in February 2025).

Because monthly downloads span roughly three orders of magnitude across libraries (from 1M to over 1B downloads per month), we use a log scale in Figure 2 to represent proportional rather than absolute change. On this scale, all ten libraries show a comparable upward trajectory, with growth continuing to accelerate around the start of the LLM Coding Capability Era, indicating that download gains are observed across the ecosystem rather than concentrated in the highest-volume packages. 

Figure 2: PyPi downloads of 10 popular Python libraries used in data science (log scale)

Observation 2: Download growth rates increased after AI agents arrived on the market

At first glance at Figures 1 and 2, the download growth seems to coincide with when LLMs with coding capabilities and AI coding agents entered the market.

To investigate whether this was actually the case with more precision, we examined how the download curves changed before and after the arrival of AI coding agents on the market. For fair comparison, we defined a pre-agentic era (16 months prior to the release of Claude Code preview, including part of the LLM era and the full period of the LLM coding capability era) and an agentic coding era (16 months after the release of Claude Code preview). We quantified the compound monthly growth rate (CMGR), which measures the average monthly percentage by which downloads grow over a period.

Figure 3 shows that the average CMGR of the 40 data science libraries increased on average by over 3 percentage points between the pre-agentic and agentic eras. The CMGR increased across the four categories, with Deep Learning increasing the most, by over 4.8 percentage points.

Figure 3: Compound monthly growth rate of PyPi downloads by data science category

Taking 10 popular libraries as a subsample of our analysis, we see that the CMGR values increased from an average of 2.7% in the pre-agentic era to around 6% since the start of the agentic era (see more information in Appendix A). The CMGR of scikit-learn downloads grew by nearly 50% between these periods, from around 3.7% to 6%. The exception is XGBoost, which shows positive but decelerating growth.

Observation 3: Statistically significant correlations between the downloads of data science libraries and AI agent SDKs

Given the observed positive CMGRs since the start of the agentic era, we were curious to investigate the relationship between the downloads and the adoption of AI tools by data science practitioners. Our hypothesis: If the use of AI agents is driving these downloads, we would expect to see co-movement between the downloads and the adoption of AI agents.

A major challenge for testing this hypothesis is the lack of continuous, structured time-series data about the adoption of coding agents. Through desk research, we found data scattered across the news, press releases, and quarterly earnings reports, but the data was inconsistent, sporadic, or unsuitable for the analysis. For example, we found one-off user counts from different companies at different moments: OpenAI Codex surpassed 4 million weekly developers in April 2026, Microsoft reported over 20 million users of Copilot in July 2025, and Cursor went from having 40 thousand users in August 2024 to over 1 million in April 2025. We decided to opt for the following proxy: the Python SDKs for the APIs of AI agents. We selected the SDKs of five market-leading AI agent vendors:

• anthropic: Official Python library providing access to the Claude API from Python 3.9+ applications, with both sync and async clients. Supports MCP, tool use, streaming, and Bedrock/Vertex deployments.
• openai: Official Python library providing access to the OpenAI REST API from Python 3.9+ applications, with type definitions for all request parameters and response fields, and both synchronous and asynchronous clients powered by httpx.
• google genai: Combines download data from Google’s SDK google-generativeai (deprecated at the end of 2024) and its SDK google-genai for the Gemini API. We tracked both SDKs into one, so that we are able to have more historical data and as developers still transition to the newest SDK version.
• mistralai: Python client SDK for the Mistral AI API, with v2 as the current version on PyPI; supports agents, GCP, and realtime as optional extras.
• cohere: The Cohere Python SDK allows access to Cohere models across different platforms, including the Cohere platform, AWS, Azure, GCP, and Oracle OCI.

Although Cursor is a widely adopted AI coding agent, we excluded it from the analysis because its Python SDK was released on May 23, 2026, and therefore the available download data (1 month) was insufficient for the analysis.

We computed Spearman correlations between weekly downloads and corrected for autocorrelation across the full period (from January 2022 to June 2026) as well as separately within each of the three previously mentioned eras: the LLM era (from the ChatGPT launch in November 2022), the LLM coding capability era (from GPT-4o in May 2024), and the agentic coding era (from the Claude Code preview in February 2025). Correlations were computed on detrended series to remove shared-trend effects [1], then individually bias-corrected using the Olkin & Pratt (1958) minimum-variance estimator [2] and averaged across all five SDKs to obtain a single corrected correlation per library.

To ensure we weren't simply observing two upward trends moving in parallel, we removed long-term trends from both series prior to computing correlations (known as detrending), meaning the associations reflect week-to-week co-movement in deviations, not shared directionality. Because this procedure yields a large number of correlation tests (combination of each library and SDK by era), we controlled the false discovery rate (FDR) as per Benjamini & Hochberg (1995) [3], which adjusts p-values to limit the expected proportion of false positives among significant results, rather than the probability of any single false positive. All reported significance markers reflect FDR-adjusted p-values at q < 0.05.

Figure 5 shows that on average the correlation between the 40 data science libraries and AI agent SDKs increased from 0.18 in the LLM era to 0.37 in the agentic era. Scientific Python libraries yielded the strongest correlations, averaging 0.22 in the LLM coding capability era and rising to 0.37 in the agentic coding era. MLOps and Explainability libraries showed the weakest average correlation in the agentic coding era, at only 0.19, suggesting that the adoption of some of these libraries might be independent from the usage of these Python SDKs.

Figure 4: Correlation of PyPi downloads of data science libraries and AI agent SDKs

Once again, we zoom into the 10 popular data science libraries and find that each library gains more statistically significant associations with the SDKs after AI coding agents entered the market. Figure 6 shows that the average correlation between the data science libraries and agent SDKs rose steadily, from 0.22 in the LLM era to 0.33 in the agentic era, with this last era reporting the most significant results of all three eras. In particular, scikit-learn exhibits a statistically significant positive association with all five SDKs during the agentic era (r = 0.38, p < 0.05).

Figure 5: Correlation of PyPi downloads of 10 popular data science libraries and AI agent SDKs

Open questions

This deep dive opened more questions than it closed – which is the point of our data deep dives. Four open questions stand out to us.

How are data scientists using AI coding agents? In this analysis we examined aggregate downloads of the SDKs of market-leading AI vendors. But which agents are data scientists actually using (e.g. generalist coding agents and/or specialist data science agents)? How? And at what points in their workflows? We’d like to see data-driven insights on the why’s, the how’s, and the what’s of agentic data science.

How are agents using open source data science libraries? Agents are using core data science libraries, but how? For example, are they allowing users to make the most of the libraries (e.g. leveraging advanced features) instead of only “basic” usage?

How do we ensure coding agents use the newest versions of libraries and avoid encouraging outdated practices? Core Python libraries for data science are constantly being developed, expanded, and improved. For example, scikit-learn version 1.9 was released last month, which included major improvements to many popular estimators that make them faster with GPU support, more stable, and capable of handling missing values. But could AI models trained on legacy code and versions of Python libraries encourage outdated practices over newer, more efficient ones?

If downloads are increasingly AI-driven, what does it mean to make libraries agent-ready? Libraries like scikit-learn were made for humans – with human-readable APIs, error messages, and documentation – and it will stay that way. At the same time, given that agents are using libraries, too, what types of updates should maintainers make to facilitate their use by agents (e.g. extend documentation, create agents.md files, redesign error messages, develop agent skills)?

Methodological caveats

The findings are statistically significant, but it's important to acknowledge the limitations of this analysis. Here’s our round-up:

1. PyPI downloads must be taken with a grain of salt, as noted by our colleague, Loïc Estève, in his talk at the PyData Conference in Paris [4]. PyPi downloads measure installation events, not unique users that are using the libraries. Every pip install increments the counter, whether it comes from a developer's laptop, a Docker build, or a CI pipeline running on every commit. PyPi downloads may also be inflated; for example, due to CI/CD pipelines reinstalling dependencies on every build or registries counting package metadata requests as downloads, among others. Nonetheless, PyPi downloads are useful as a signal of the total installations of libraries and ecosystem activity.
2. The AI agent SDKs are an imperfect proxy for AI agent adoption, given that these SDKs provide developers with access to the APIs of AI vendors which in turn can be used for a myriad of purposes. However, their publicly available PyPi download data was suitable for our analysis.
3. The breakpoints between eras were interpretively chosen for analytical purposes. We selected dates based on well-documented AI milestones. However, we acknowledge there’s room for debate about when the periods started or ended, and the date selection influenced the results. We welcome suggestions for alternative dates.
4. Correlation is not causation. Statistically significant correlations between the data science libraries and the SDKs indicates these trends are co-moving, but it is not evidence of a causal relationship between the adoption of AI agents and the increasing demand for open source libraries like scikit-learn.

References

[1] Granger, C. W. J., & Newbold, P (1974). Spurious Regressions in Econometrics. Journal of Econometrics, volume 2, p. 111-120.

[2] Seifer, J. (2020). Averaging Correlations - Part II. https://medium.com/@jan.seifert/averaging-correlations-part-ii-9143b546860b.

[3] Chen, X., Robinson, D. G., Storey, J. D. (2021) The functional false discovery rate with applications to genomics. https://pmc.ncbi.nlm.nih.gov/articles/PMC7846131/#:~:text=The%20false%20discovery%20rate%20(FDR,more%20accurately%20estimate%20the%20FDR.

[4] Estève, L. (2026). PyPI in the face: running jokes that PyPI download stats can play on you. https://www.youtube.com/watch?v=6SHB2v8uj84.

Appendix

Appendix A: Compound monthly growth rate of PyPi downloads of 10 popular data science libraries

Appendix B Compound monthly growth rate of PyPi downloads of 40 data science libraries in the pre-agentic and agentic era

Appendix C: Correlation of PyPi downloads of Machine Learning libraries and AI agent SDKs

For more from Probabl

• Follow our latest updates on LinkedIn
• Subscribe to our monthly newsletter
• Check out over 100 tutorial videos on our YouTube account
• Level up your machine learning skills for free with Skolar