Yesterday, an OpenStreetMap-themed account on LinkedIn1 has reached out to find journalists who might be willing to cover OSM and Open Data scraping. From the post:
Our senior site reliability engineer (…) is seeing unprecedented levels of bots attempting to sc[r]ape OSM data. For comparison, in recent years, he has seen 1 or a few IPs doing 10,000+ requests, but this week we’re seeing 100,0000+ IPs in a coordinated [effort?] to scrape, each IP doing a few requests.
(…)
We’re a volunteer-run service and the costs are real. We’d love to talk to a journalist about what we’re seeing + how we’re responding.
This development is (rightly, I think) linked to AI companies gathering data. Stadia Maps and other geospatial projects report similar observations.
The juxtaposition of “open data” and “scraping” is, of course, jarring: Truly open data (such as OSM) does not need be scraped but can be obtained through, for example, bulk downloads and APIs, ideally fairly used, thus not unnecessarily burdening the projects and their infrastructure.
So far, Guardian, Süddeutsche Zeitung, Lage der Nation, and others have responded. Watch your local (tech) news outlet.
Update: This reminded me of loosely related topics discussed by Stefan Verhuulst: “Five asymmetries” around AI and data and “Data Commons: The Missing Infrastructure for Public Interest Artificial Intelligence”.
Footnotes
I couldn’t find out who controls this account or if the OSM Foundation is behind it. It has many “employees” on LinkedIn. I did find the referenced Mastodon post and another one on this topic from the “OpenStreetMap Ops Team” on the “@en.osm.town” instance. And the person referenced in the LinkedIn post is part of the OSM Operations Working Group. So I do think this is legit.↩︎