Improving OpenStreetMap shop coverage with AllThePlaces

[Mateusz_Konieczny]

@Robert_Whittaker

Hi @Mateusz_Konieczny nice work here!

I’ve just had a look at your matching table for my local area:

Thanks!

@Robert_Whittaker

Travis Perkins: https://matkoniecz.codeberg.page/improving_openstreetmap_using_alltheplaces_dataset/missing_shops_travis_perkins_gb_52%201.html fails to match to Way: ‪Travis Perkins‬ (‪547315153‬) | OpenStreetMap

Mismatch may be on distance and is definitely on

shop=builders_merchant vs shop = doityourself

1. should we document shop=builders_merchant | Tags | OpenStreetMap Taginfo or retag shop=builders_merchant to some of existing tags?

2. should I consider shop=builders_merchant shop = doityourself as matching? If yes, what else may be worth matching?

@Robert_Whittaker

• Dunelm: https://matkoniecz.codeberg.page/improving_openstreetmap_using_alltheplaces_dataset/missing_shops_dunelm_gb_52%201.html fails to match to Way: ‪Dunelm‬ (‪292241201‬) | OpenStreetMap

shop=interior_decoration vs shop = houseware

1. is ATP badly classifying Dunelm? Should it be shop = houseware in ATP? Or maybe OSM is classifying it badly?

2. shop=interior_decoration and shop = houseware should be considered as matching… What else belongs in this group?

@Robert_Whittaker

• Five Guys: https://matkoniecz.codeberg.page/improving_openstreetmap_using_alltheplaces_dataset/missing_shops_five_guys_gb_52%201.html fails to match to Way: ‪Five Guys‬ (‪144745480‬) | OpenStreetMap

For the last one (Five Guys) it’s possibly the distance that’s the issue.

Yes, distance alone will consider it as separate and not matching.

@Robert_Whittaker

FWIW, in my Chains tool I consider precisely the OSM objects with the right brand:wikidata=* value for the matching, and then match based on distance within a threshold or a larger threshold if there’s also a matching (UK) postcode. For this to work well it requires some manual checking and adding of missing brand:wikidata=* tags - but the payoff is that you have much better control over the matching.

Some of problems here are that the same brand:wikidata may be for both say supermarket and fuel station (Tesco).

Or the same for fuel station, convenience shop and parcel locker (Orlen).

Also, I tried to use wikidata codes once - you ebd in a rabbit hole of some brands having separate wikidata entries for subbrands, some not. Sometimes brand:wikidata is linking dead company entry sometimes dedicated brand entry. And often multiple at once.

More importantly I have not yet found case where brand:wikidata matching would improve matching.

For UK using postcode is likely helpful but it does not work well in global coverage where postcodes are tagged fairly rare or in various formats.

And for global processing adding missing brand:wikidata as part of making this tool is juts not feasible at all.

(some prices paid for having global tool - dedicated localized one are likely to remain better)

I have major TODO for matching: consider also website tag

@Robert_Whittaker

In your table at https://matkoniecz.codeberg.page/improving_openstreetmap_using_alltheplaces_dataset/52%201_index.html I think it would be good to also have columns for the total number of branches of each chain in ATP, and the number that you’ve successfully matched.

I’m confused by what the “% of ATP matched to OSM” is supposed to be. e.g. holland_and_barrett says it’s 10%, but there are no “missing in OSM” entries. If only 10% are matched, I’d expect the other 90% to show up in the “missing in OSM” column.

I will try to reduce this confusion, I am not yet sure how.

But likely it will include customized distance thresholds for when POI is considered as gone (for some spiders it should be 15m, for some 1500m).

Next regeneration of data will be published with lower threshold for distance when object is considered as missing. If this goes well I will reduce it further, maybe eliminating gray area completely.

Or maybe it should be listed as a separate category?

---

I just drafted something that should report it as a separate category, lets see how well it will work.

@Robert_Whittaker

I think the maps would be more useful if they also showed matches as well as the missing stores.

for now it is kind of blocked by misusing several tools in my tech stack

one of funniest part is that commit of multitude of generated static files for publication takes multiple hours

I will try gray area listing of away, but not very far away, if that will not explode processing I will try listing also successes.

@Robert_Whittaker

(Sometimes a match fails because there’s a duplicate in ATP, and this would allow you to spot this.)

In such case both will match the same object without spotting problem.

@Robert_Whittaker

As well as stores missing from OSM, the other data quality issue is stores on OSM that shouldn’t be there. Do you also check for OSM objects of each store type that haven’t been matched to an ATP record?

I considered doing this but ATP data in general is not good enough for that on a global scale. Too many shop on the wrong continent or misplaced by several kilometres.

It is still in vague plans but I am not planning to do any time soon. Maybe for few spiders with known excellent quality?