UC Berkeley Code For Good Volunteering

[spwoodcock]

Pretext

• osm-tagger was developed last year with some pro-bono external support, based on initial prototyping by @emi420.
• The end goal is: upload an image, extract relevant OSM tags. Ideally it's generic, but the current demo was focused on roads only.
• It takes embedding from 9000+ road images to provide a confidence value for surface and smoothness tags.
• We want to create a generic pipeline that can take a dataset of pre-tagged images, a selection of tag options for the category, and then generate a classification pipeline based on this: e.g. building damage could be a good second target.
• For references, here is a good example of community agreed OSM tags: https://wiki.openstreetmap.org/wiki/Key:surface (note that OSM tags can be anything you like, but it's best to stick to community agreed conventions to ensure they are as useful as possible)

Code For Good

• UC Berkeley runs a program to have software engineering student contribute to open-source projects 'for good'.
• A student group will focus on a project, with set a set timeline and deliverables to push the development forward.
• osm-tagger was selected as one such project.

Future plans / challenges to address in osm-tagger:

• Quality of the result: can often produce incorrect tags, e.g. Model suggests invalid smoothness=100 tag on non-road (person) photo #40 and various other hallucinations.
• The following issues document where we should go from here:
  • Cost reduction measures #44
  • How do we set criteria based on datasets from different parts of the world? #45
  • Integration with efforts from Panoramax? #46
  • Integration pathways for other tools #47
  • Conflation and final merging of tags into OSM #48

[spwoodcock]

Based on the call with HeiGIT, we have a few avenues to explore.

Goals

Panoramax image analysis pipeline

• We will get access to the POC Hugapanoramax project.
• The goal is to:
  1. Pull Panoramax imagery via the API for a given area.
  2. Run whatever models appropriate to extract feature annotations.
  3. Push the annotations to the imagery in Panoramax via API.

CV: Research on OpenFacades & building material detection

• HeiGIT already have some research docs on this, including further questions to research.
• We should research and test the ideas in this doc, and determine things such as:
  • Can we use this approach in scenarios where we don't have great OSM building footprints already? (OpenFacades seems to rely on having OSM buildings already mapped).
  • Can we use Overture data reliably instead of OSM data? If so, how do we get it back to OSM afterwards?

LLM: cost reduction for tag extraction

• HeiGIT has already done a lot of work on road tag extraction.
• We have osm-tagger, but it was deemed too expensive
• We have two avenues to explore / research here:
  1. How to reduce costs for the inference? Are smaller models suited?
  2. Can we optimise the model to work more efficiently / have a higher accuracy for a specific tag, such as smoothness?

Useful OSM tags to extract in a humanitarian context

LLM extraction

• roads - surface
• roads - smoothness
• building:material + roof:material
• building:condition (seems like not a great choice, as it's not as widely used and very subjective)
• building:height
• building function: such as building=apartments, building=school, building=commercial, building=industrial, building=hospital

Computer vision detection

• bridge - for access bottlenecks
• power=tower
• power=pole

Todos

• Research further what metadata can be stored in Panoramax as annotations, so we can better determine what to extract (can we only annotate objects detected, or can we also annotate additional tag metadata such as building material?)
• Pick up on the HugaPanoramax project started by HeiGIT to build out the pipeline (be sure to run on a locally installed Panoramax instance and not push annotations to production!)
• Look further into OpenFacades and attempt to answer questions posed by HeiGIT team about it's usability in our contexts.
• Look into issues on the osm-tagger repo, and assess avenues for model optimisation and cost reduction for the inference.