Telemetry initiative

Created on 30 January 2018, over 6 years ago
Updated 21 May 2024, about 1 month ago

Related initiatives

Gathering some key data from end-users about how Drupal is used can give the community and the Drupal Association insights that will help us improve the product roadmap, community programs and outreach efforts by the association, and more. Right now, the only data we receive is a very limited amount of data from the sites that call home to Drupal.org for updates information.

Telmetry initiative: Gathering data about Drupal usage

The goal is to gather data about who uses Drupal, what modules they use, what modules they *don't* use, maybe basic traffic/load information, what php/db versions are in common use, etc... All of this information could be tremendously helpful in setting direction for the project.

We would want to build a modular telemetry system so that we can gather different kinds of data with each major release, if we want to focus on certain areas of the project for improvement.

---

Usecases we are targeting to improve:

  • Providing aggregate, anonymized data to core developers to help them understand real-world usage patterns with Drupal.
  • Providing the Drupal Association with data about the scope of the Drupal community outside of Drupal.org - how many sites are there really? Who are they?
  • Making all of this telemetry sending opt-outable.
  • Making sure that headless sites can be identified as Drupal sites underneath by crawlers, and by the d.o metrics system

Usecases we are not targeting to improve:

  • n/a

---

Sign-offs given:

@Dries has endorsed this idea in a call with DA staff, core maintainers, and community volunteers.

Here is the slide deck from the check-in between Core, the DA, and some interested community members about these initiatives.
---

Meet the initiative team

TBA

---

How we are working

TBA

---

How you can help

Contribute your time and skills. Below are the main planning issues that link off to individual tasks that can be worked on:

TBA

🌱 Plan
Status

Needs review

Component

Idea

Created by

🇺🇸United States hestenet Portland, OR 🇺🇸

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇫🇮Finland lauriii Finland

    There are many great ideas on this issue for telemetry data points that we could collect. To make this more feasible, we should being by experimenting with collecting telemetry with as small scope as possible. For instance, we could create some leading indicators for the current lagging indicators. One of the indicators we're following is the number of Drupal sites reported by the update status module, which is a lagging indicator. However, there are leading indicators such as how many Drupal sites are being installed or how many Drupal composer projects are being initialized using composer create-project drupal/recommended-project drupal.

    This data would be valuable when assessing the overall state of Drupal. If we could expand this to getting data on module installations too, we would have leading indicators for experimental modules as well, which would be extremely valuable for working on Drupal Core initiatives.

    While there is a lot of other useful data that we would want to collect, starting with something small like this would make the scope more achievable. Once we have figured out this use case, we can move on to the next use case and add additional data points for the other proposed ideas.

  • 🇺🇸United States hestenet Portland, OR 🇺🇸

    Some progress on the performance information side of telemetry has been made in: 📌 Add open-telemetry/sdk and open-telemetry/exporter-otlp as dev dependencies Fixed

  • 🇨🇦Canada mgifford Ottawa, Ontario

    I was just chatting with @rkoller about how nice it would be to actually know how people are using the admin breadcrumbs. Would be so useful to know if folks are going to our help.

    If there is a way to enable this so that we can learn more about how authors and admins use their sites it would be so valuable. So often we just have to make assertions based on what we think would be useful to these users. Sadly, extrapolations from developers is rarely as good as watching those trying to use their Drupal site.

  • 🇺🇸United States bkosborne New Jersey, USA

    I agree with lauriii in #44 that the scope of this needs to be reigned in. If we shoot for the moon, this initiative will never take off (that's not to knock any of the very good ideas I've read through above!!)

    I'm here because I maintain a Drupal platform of 1,000+ websites and .

    I don't have the update module enabled on any of these sites because the vast majority of the module's code (and it's a lot of code) is unrelated to sending the telemetry to Drupal.org, and it's for functionality I don't want or need. It's mostly around facilitating module updates/installs through the UI and reporting on modules that are out of date or have security issues. As a platform maintainer, I don't need that capability. I have other ways to monitor those concerns. As a platform maintainer, I try to keep unnecessary code off the platform as much as possible.

    I know other platform maintainers (there's a lot of us in Higher Ed) that also don't have the update module enabled. We're collectively contributing to a massive under-reporting of project (module) usage on Drupal.org, all because the current code for collecting module telemetry is bundled with something we don't want.

    So I feel the first step of launching this initiative is to separate the project usage reporting out of the update module and put it in core. Basically don't change any of the existing functionality of project usage reporting that Drupal.org already provides yet (I guess via the Project module?) We do that at a later time.

    I see another user already created an issue about this #3199472: Improved data sharing with d.o . I think that's a good place to start focusing the effort.

  • 🇺🇸United States bkosborne New Jersey, USA

    Actually that last issue has a broader scope and I didn't feel comfortable hijacking it entirely, so I created a new issue Moving project telemetry reporting from the update module into core Active as a child of this one.

Production build 0.69.0 2024