Can You Trust BuiltWith, W3Techs and Software Usage Statistics?

Can-you-trust-BuiltWith-W3Techs-and-software-statisticsThe Drupal Association put out a press release today, quoting some statistics about Drupal.

The press release said:

  • According to Drupal.org stats, Drupal now powers more than 1 million websites.
  • According to Builtwith Research, Drupal powers more than 12% of the world’s top 100,000 websites.

I regularly see stats like this fly around the web, particularly on Twitter.

So, I decided to use these Drupal stats as an example and answer the question: "Can you trust software usage statistics?"

Sources of usage statistics

There are three websites that collect very large amounts of information available about Drupal:

  • Drupal.org
  • W3Techs.com
  • BuiltWith.com

There are some smaller sites such as Wappalyzer and TopDrops, but I excluded those because either have much smaller data samples than the big three or didn't present their data in ways that could be easily compared.

Out of these three sites, not all the information is equally valuable. It's worth giving priority to sources who are closest to the actual software installs. Because Drupal.org connects directly to actual Drupal sites, its data is probably the most reliable.

I would give the least credence to BuiltWith. Both Drupal.org and W3Techs publish their methodology, but BuiltWith has only the following, empty statement:

"Every website gives off "signals" that they are using a particular technology. We can track these signals and determine if a site is using a particular web technology from it."

Question: How many sites use Drupal?

media_1393955131863.png
  • Drupal is used by 1 million websites according to Drupal.org.
media_1393955171282.png
  • Drupal is used by 796,045 websites according to BuiltWith.

Answer: These two numbers agree fairly closely, so we can estimate that Drupal is used by close to a million websites.

Is Drupal usage growing or shrinking?

media_1393955043492.png
  • Drupal use is trending up slightly (from 800,000 to 934,000 sites during 2013) according to Drupal.org.
media_1393954938260.png
  • Drupal use is trending up very slightly according to BuiltWith.
media_1393955011244.png

Answer: Taking an average of the three sites, Drupal usage is trending slightly up.

What percentage of the web is using Drupal?

media_1393955358275.png
  • Drupal is used by 0.3% of all websites, rising to 3.9% of the largest 10,000 websites according to BuiltWith.
media_1393955425495.png
  • "Drupal is used by 5.4% of all the websites whose content management system we know. This is 1.9% of all websites." W3Techs

Answer: This is all over the place, but the midpoint of these averages is just over 2% with a noticeable lean towards large sites.

Conclusions on the Drupal press release statistics

So what's the verdict on the Drupal press release?

  • Yes, Drupal probably does power around a million websites.
  • The 12% figure is odd. I'm not sure where it came from (BuiltWith says 3.9%). Even if accurate, it doesn't reflect the other statistics on the percentage of the web using Drupal.

Conclusions on usage statistics in general

I started this exercise as a sceptic but was pleasantly surprised by the fact that could extract some general agreement from these different sources.

There were at least three points where we could get some consensus:

  1. The total number of sites
  2. The general trend in usage
  3. The general market position

The one point where the results were radically different was the % of the web using Drupal. I wouldn't feel comfortable quoting any of that data. I suspect that BuiltWith and W3Techs have radically different ideas about the size of the internet. 

However, it's worth remembering that even the closest answers differed by at least 25%. The data gets even more wild when we start to include other sites such as Wappalyzer.

If you're doing this research, always get at least two sources of data, and if possible three or more.


About the author

Steve is the founder of OSTraining. Originally from the UK, he now lives in Sarasota in the USA. Steve's work straddles the line between teaching and web development.