Focussed tag check effort in a defined community based on check_date:<selectedTag>

[tordans]

I wanted to start the conversation about using StreetComplete and the (newish) check_date:* "system" to run a kind of remove and async mapping effort in a community.

Szenario 1: For https://www.zesplus.de/forschungsprojekt, we need good OSM data for three villages/cities for certain data. The data will be used to plan a bike network, so we want to be fairly certain that it is up two date for the most important key.

Szenario 2: The same goes for parking:lane data for a district that wants what we have in Neukölln (https://supaplexosm.github.io/strassenraumkarte-neukoelln/?map=parkingmap#18/52.47395/13.44113).

Szenario 3: maxspeed, see #3638 (comment)

---

There are multiple ways to achieve this:

• Having one (paid) mapper check it all
• Create a MapRoulette challenge
• Create a TaskingManger Campaign (which can also be created per street (screenshot at Potential to use editor with tasking manager project / campaign zlant/parking-lanes#58) instead of grid based)
• Create a MapComplete Theme
• Setup a FieldPaper based process and map at home later (Great example here, Video)
• …
• And one could be to use StreetComplete.

But for this to work, there needs to be a way to configure StreetComplete to show all re-validation quests with no or a very short tag-check_date- and feature-updated_at-timespan.

For example, checking all smoothness values would right now exclude all streets that where touched in any way in the last 4 years (#3617 (comment)).

The big advantage of StreetComplete over the other tools listed above is, it is optimized for mobile. Many of those efforts are easier mapped on the go than at home.

---

Are there others that would like to use SC like this?
What are your thoughts on extending the app in such a way?

---

Update: Added maxspeed Szenario

[matkoniecz]

Technically it would be possible but would require creating a new quests copying existing one, limited to some area and set to trigger resurvey if some data is older than X.

It may be necessary to add minor functionality to trigger resurvey quest older than specific date, rather than relatively to the current date

Another option is making a custom-modified SC version.

Note one trap:

• there is old and outdated surface data
• someone fixes name tag
• now this way will not produce surface resurvey quest

unless any way without check_date:surface would be checked?

[tordans]

For such an endeavor one could simply say …

always show quests unless there is a check_date: tag that is not older than 2 years.

A result would be a clean dataset of tag + check_date: tag for the whole area.

I don't see a reason to add a bounding box as long as there is a way (Deeplink?, Setting?) to activate this for all who want to participate. The process would be accompanied by some local communication and probably kick off meeting with introduction in how to do the task.

[mnalis]

• custom compiles SC version is always a possibility, as noted. Perhaps it is a best one for such project, but...
• I recall there have been several times where something like this was asked (eg. Postbox collection times in Switzerland #2963, Region based quest to check for green waste containers? #3091, "Is this building existing" from external database - is it a good idea? #2477, Triggering resurvey for locations where brand left the country #3083, How can I resurvey all ways for lit status now? #3237, ...)
• one of the (easier) reusable solutions might be additional level of aggressive to Resurvey interval, which would ignore current logic, and always re-ask a quest for tag unless check_date:tag is existing and is very recent (1 month?). It would only solve a subset of problems mentioned in those discussions, though, but might be better than nothing (and less controversial then automated mass-editing of the area by adding faked check_date: 1970-01-01 tags)
• much more complex (but also more customizable and covering more situations) solution, would be an (expert) option of "custom quest server URL" (empty by default) which would contact specified server, and receive from it list of quests that the user should solve (it should obviously contain at least the type of quest and a node/way id in basic version, but it might also for example contain extra text to show to user). Then the admin could at server side keep all the logic of selecting which quests the users should solve, could update it dynamically for all users etc. But it requires writing an (easily customizable) SC quest server too...

[peternewman]

It may be necessary to add minor functionality to trigger resurvey quest older than specific date, rather than relatively to the current date

I don't see a reason to add a bounding box as long as there is a way (Deeplink?, Setting?) to activate this for all who want to participate. The process would be accompanied by some local communication and probably kick off meeting with introduction in how to do the task.

I've been pondering a more general problem for a while, which I think overlaps with this a lot, hence slightly hijacking this issue.

Consider broadly any type of construction work, they seem to be often knocking down buildings and erecting tower blocks in a lot of areas near me. Or more minor things such as resurfacing roads, or bits in the middle like pedestrianising parts of town (or just giving a bit more space to pedestrians); generally shaking things up significantly over a reasonably large area.

We've currently got a quest where we ask if this construction is finished, but that's basically it.

I think most people would agree that on-the-ground tools like SC (or other mobile apps) are best suited for surveying, otherwise you've essentially got to note down the same info and then process it again when you get home (on iD/JOSM or whatever), which means twice as much work and more risk of error.

Ignoring the practicalities of how it's achieved for the moment, wouldn't it be nice if I could flag up an area within OSM and say they're doing some work on this area until a predicted end date of 1 Jan 2022. I'd then get the construction quest to say have they actually finished. Maybe it overruns a bit, so construction actually finishes a month later (1 Feb 2022); at that point, everything within that area should be resurveyed if it's dated after the 1st Feb.

Because of the above, I'm very behind this idea, but I think it would make sense if it was implemented in such a way it can be used for lots of people, rather than just this specific project; because of the above I think the reality is it's happening all over the place all the time.

Particularly for the more minor pedestrianisation type cases, I don't really see what other option I have but something like this; I don't want to delete all the surface tags, as I'd need to wait until they'd completely finished to ensure I covered them all, but they've already completed some parts, so they could have already been resurveyed.

FWIW, logically to me it would be an area on the OSM map perhaps around the construction tagging; but I realise doing it that way would need a lot wider buy in. As @tordans has sort of alluded to, it would be good if you could do some filtering; e.g. in one case they've only been resurfacing and maybe moving benches etc, but in the towerblock case, whole buildings and paths may be gone and entirely reworked. @tordans request is then just a special case of what you'd do if they'd resurfaced just some roads or similar.

[mnalis]

@peternewman yes, for the construction case, construction quest in SC only will handle if the element itself is tagged with highway=construction (or building=construction), it will heed the opening_date=*.

It is more complex for landuse=construction as you want to resurvey all quest contained inside that polygon, but both my suggestions (manual via "aggressive resurvey interval" and automated via "server-side quest suggester") would be usable for that use case too:

• In manual case, user who know that area is under construction and wants to solve surfaces, would enable "aggressive resurvey interval" while there and thus be asked for all quests in that area which have not been recently asked (user would probably also want to load preset for such construction areas, as they probably don't want to be asked for all quest, just the ones that were likely to have changed in that construction project).

• automated "SC quest server" case (actually generic quest server, should it be written, as it would receive bbox area and output JSON list, so it would be usable by anyone really) could find all highway=* inside landuse=construction area, and ask specific quests for them more often, unless they have check_date:quest_tag tag with date newer than that of landuse=construction polygon; and when the landuse=construction polygon disappears, all remaining elements that were inside it should be marked as needed for resurvey (which is something that requires keeping the history of all construction areas, so really only doable correctly on server side)

Although, for landuse=construction (depending on the type of construction) there is possibly additional problem unsolvable with SC - because it needs full fledged editor (preferably one with recent post-construction aerial imagery, also preferably with using something like tasking manager). Because new ways could've been created and old deleted, old buildings razed and new ones built, parks and playgrounds being build and basically any feature might have appeared or disappeared.

[smichel17]

My technical thoughts as a programmer but OSM outsider are below. If my OSM-ignorance means this is totally non-workable, then please don't waste time correcting me (to show I mean this and preserve that ignorance, I'm unsubscribing from the issue and don't plan to comment further).

I could imagine implementing a project like @tordans or @peternewman describe as follows:

1. Create a relation which includes all nodes/ways that we care about.
   • For @tordans, this would be all the data you want to ensure has up-to-date tags.
   • For @peternewman, this would be all the places which are currently under construction.
2. Add tags to the relation:
   • description: human-readable explanation of what the relation represents
   • resurvey_required_date: This is the date where "we only trust information collected more recently than this"
     • For @tordans, this would be the date when it was decided to plan the new bike network, or however long ago you are comfortable trusting the data.
     • For @peternewman, this would be the date when the construction finishes.
   • resurvey_required_tags: This is a comma-separated list of tags which should be considered outdated. For example, with re-paving a road, it might be surface,smoothness
3. StreetComplete shows a resurvey quest for each of the tags listed in resurvey_required_tags if they do not have a check_date more recent than resurvey_required_tags.
4. Once all the members of the relation have had the relevant tags updated, the relation may be deleted.

In short, this is using a relation as metadata to determine what needs resurvey. Alternatively, a similar "needs resurvey after this date" tag could be added to each node/way directly. In either case, I think it makes more sense than backdating the check_date so SC will show a resurvey quest, which produces false data and relies on SC implementation details (knowing what the resurvey interval is).

Of course, if this is not already a common practice, it would need a tagging proposal taken upstream before it can be implemented in StreetComplete.

[Lee-Carre]

Interesting idea.

I have mixed thoughts about using map data itself for organising editing work. While I can see several advantages (since a bunch of metadata (which is only of interest to mappers) is already included), other tools already exist for collaborative editing (various QA (which could be modified to be sensitive to check_date), MapRoulette).

resurvey_required_tags: This is a comma-separated list of tags which should be considered outdated. For example, with re-paving a road, it might be surface,smoothness

Based on OSM conventions for tags with multiple values, the semi-colon should be used (instead of a comma).

[tordans]

Another example of my use case: https://twitter.com/BerlinCyclist/status/1482348731248877571

Someone is working with maxspeed data and finds it lacking in quality (which it might very well be, IMO). It would be perfect to coordinate a mapping-effort with a few activist to re-survey all streets until there is a recent check_date:maxspeed and thus the data can be trusted (again).

This could be something simple as 5 people on Twitter (or a local meeting) agreeing to configure SC in for maxspeed quest in "resurvey all"-mode and go out for some mapping.

[Helium314]

This could be something simple as 5 people on Twitter (or a local meeting) agreeing to configure SC in for maxspeed quest in "resurvey all"-mode and go out for some mapping.

If you're willing to try, I just added something that would allow something like this to my fork: Helium314@9f9dcb4 / https://github.com/Helium314/StreetComplete/tree/my_modifications

There is now an "Advanced resurvey" setting, which allows specifying keys for which there will always be a resurvey if the tag has no check_date (or similar keys). If there is a check_date, the usual resurvey intervals are used.
You can also enter a date, then all of the specified keys will be eligible for resurvey if the check_date is older than the specified date.
For these resurvey-keys, check_date will always be set when answering the corresponding quest.

But your exact example will not work, because SC does not resurvey maxspeed.

[tordans]

Thanks for linking the other discussions @mnalis. I have two main takeaways from these thread:

• Mass editing the check_date:key would work; but that is not what I would like to do or consider a good solution

• westnorost hinted at a re-survey-screen in How can I resurvey all ways for lit status now? #3237 (comment)

  There is a plan to make a dedicated view for that - i.e. see in an overview how the streets are tagged currently and you can re-tag any of them.

  So maybe this will solve some of the issues here.

[FloEdelmann]

There is a plan to make a dedicated view for that

For reference, this is #2461. (I just commented it on the linked discussion, too.)

[smichel17]

@tordans Note that it may not be done on the timetable needed for this effort. If that hapens, I would suggest making a streetcomplete fork just for this event, as the most practical solution. It would be pretty trivial to modify the quest so that it asks resurvey for certain quests which have not been checked since a hard-coded date, rather than X years ago.

[tordans]

The construction-area-resurvey-szenario by @peternewman is interesting too (thanks for sharing). The way I see it, there are two ways to think about resurveying and yours is describing "the other" way:

resurveying mode	quest (tags)	area	example
by topic	fixed	flexible (where I am)	A group want's to resurvey all surface-quests in their hood
by area	flexible (which quests I activated)	fixed (eg. the construction (…) area)	A mapper wants to resurvey an area that changes a lot (park was remodeled, houses build, street (area) rededicated, …)

To keep this thread more on topic, I suggest to move further thinking about the second case to a new issue. I can comment more thought there.

[peternewman]

Now created as #3651 .

[mnalis]

BTW @tordans it would be good to open such open-ended discussions as Discussions, not as Issues. Only where there is clear and actionable idea exactly what should be done and in what way, should it be created as an issue (cf. #3450)

[westnordost]

Converted it into discussion now

[Lee-Carre]

Question:

I typically use Vespucci for surveying, now. However, I started with StreetComplete (so will always be fond of it).

I can see the merit of the strategy (check_date:[key]=[today]) suggested, here. I also add tags which I know SC uses (example: opening_hours:signed=*).

So, when I'm (re)surveying features, should (that is, would it help if) I added check_date:*=[today] for each relevant key …

• in addition to check_date=[today]
• instead of check_date=[today]
• or something else entirely

… ?

For clarity, I make a habit of adding

• check_date=[today]
• survey:date=[today]

to each element I (re)survey.

In that case, would the base / root check_date be interpreted as applying to all keys (in which case check_date:[key] is redundant) or assumed to be ambiguous (thus adding the key-specific tags makes a difference)?

[westnordost]

check_date is interpreted by the app currently only as that existence has been checked. check_date is quite unspecific as it is not clear what exactly has been checked. Maybe the person checked opening hours, maybe not. What about the phone number? The operator? The ref? Whether it has vegetarian food? It's clear that often it is simply not possible to check all the data entered for one place. So, the common denominator is at least the existence and that is how the app treats this tag.

[Lee-Carre]

Thanks for a very clear explanation, @westnordost 👍.

In that case, I'll begin adding key-specific check_date tagging 🙂.

Next challenge: automating that task, within Vespucci.

[Lee-Carre]

Sample for review: changeset #117395574.

[westnordost]

Jesus...

[tordans]

There are exiting things on the horizon, see https://www.openstreetmap.org/user/westnordost/diary/399378
I do not yet fully understand, yet, which conditions the new overlays will look at. But this has the potential to solve this ticket.

[matkoniecz]

layers will show all existing data, limited to specific topic (say: all sidewalk=* data)