Broken URLs in airspace list

[pyrog]

• au openair
• be openair
• ca openair
• ch openair
• cz openair
• dk openair
• es openair
• fi openair
• fr openair
• nl openair
• pl openair
• us openair

The Airspace layer seem to come from the openAIP project.
Does we could use the same data source?
xcsoar-data-repository/data/airspace.json

See also #381, #435

[Turbo87]

The Airspace layer seem to come from the openAIP project.

We are not using openAIP anywhere as far as I know.

Does we could use the same data source?
xcsoar-data-repository/data/airspace.json

yes, we should eventually switch to using that file or a file derived from that.

[pyrog]

We are not using openAIP anywhere as far as I know.

Ok, what is the source? because the bitmap layer is more complete than the clickable vector datas.
These datas "should" stay in sync to avoid double work.

PS: There is also the open flightmaps project (OFM).

[Turbo87]

Ok, what is the source?

we are essentially rendering it ourselves according to the styling rules in https://github.com/skylines-project/skylines/blob/master/mapserver/airspace/airspace.map

because the bitmap layer is more complete than the clickable vector datas.

the tiles are cached, but they should be in sync in the majority of cases since the airspace data hasn't been updated in quite a while.

[TobiasLohner]

There are a few things to consider when updating airspaces:

1. Some of the airspace files contain unknown or wrong airspace class identifiers. This could possibly clutter the map when rendering (e.g. a FIR denoted as class A)
2. SkyLines uses the gdal/ogr openair/sua parser which is not as robust as XCSoar's. Some files need manual editing before importing.
3. Some airspace files contain country borders which shouldn't be imported as well. See https://github.com/skylines-project/skylines/blob/master/mapserver/airspace/airspace_blacklist.txt for some examples. This needs manual checking before import.

I recommend to test every single new airspace flie on a QA system (e.g. your local SkyLines installation).

[pyrog]

I wrote a quick and dirty tool that upload XCsoar airspace list and output airspace_list on the standard output 😄

This snippet could be included into skylines/commands/import_/airspace.py.
(I don't do it because I haven't installed a test server)

Also see XCSoar/xcsoar-data-repository#56

import json
import os.path
import subprocess
from io import StringIO


def download_file(path, url):
    # Create data folder if necessary
    if not os.path.exists(os.path.dirname(path)):
        os.makedirs(os.path.dirname(path))

    # Download the current file
    # (only if server file is newer than local file)
    subprocess.check_call(['wget',
                           '-q',
                           '-N',
                           '--no-check-certificate',
                           '-U', 'Mozilla/5.0 (Windows NT 5.1; rv:30.0) Gecko/20100101 Firefox/30.0',
                           '-P', os.path.dirname(path),
                           '-O', path,
                           url])

    # Check if download succeeded
    if not os.path.exists(path):
        raise RuntimeError('File not found at {}'.format(path))

    # Return path to the file
    return path

filename = u'./airspace.json'
download_file(filename, 'https://raw.githubusercontent.com/XCSoar/xcsoar-data-repository/master/data/airspace.json')


f = open(filename, "r")
a = json.load(f)

print '# airspace sources file'
print '#'
print '# each line contains:'
print '# 2-letter-country-code openair/sua url'
print

last_update = str()

recs = sorted(a['records'], key=lambda r: r['area'])
for r in recs:
    try:
        t = r['type']
    except KeyError:
        t = 'unknown'

    print "%s %s %s" % (r['area'], t, r['uri'])

    if r['update'] > last_update:
        last_update = r['update']

print '# Updated '+ last_update

# airspace sources file
#
# each line contains:
# 2-letter-country-code openair/sua url

at airspace https://www.austrocontrol.at/jart/prj3/austro_control/data/dokumente/aV5XF_AustriaAirspaceMainFileeffdate01052015-27052015V.1.00.txt
au airspace http://soaringweb.org/Airspace/AU/australia_class_all_16_05_24.txt
be airspace http://soaringweb.org/Airspace/BE/BELLUX_WEEK_140501.txt
be airspace http://soaringweb.org/Airspace/BE/bellux_weekend_14_base_rev2.txt
bg airspace http://pg.v5d.org/airspace/bg_airspace_latest.txt
br airspace http://soaringweb.org/Airspace/BR/BRA13A16.txt
ca airspace http://soaringweb.org/Airspace/NA/canadian_airspace_256.air
ch airspace http://soaringweb.org/Airspace/CH/CH-Luftraum_2016_04_27.txt
cz airspace http://soaringweb.org/Airspace/CZ/Cz_all_2015_10_15.txt
de airspace http://soaringweb.org/Airspace/DE/Germany_Week22_2016.txt
de airspace http://www.daec.de/fileadmin/user_upload/files/2012/fachbereiche/luftraum/2016_April_Sprungzonen.txt
dk airspace http://soaringweb.org/Airspace/DK/DK-AllinOne-20160408.txt
es airspace http://soaringweb.org/Airspace/ES/SUASpain201610.txt
fi airspace http://soaringweb.org/Airspace/FI/finland_airspace_20141103_openair.txt
fr airspace http://s289271336.onlinehome.fr/dossiers_ffvv/files/__AIRSPACE_latest.txt
gr airspace http://soaringweb.org/Airspace/GR/greece_03.txt
hr airspace http://soaringweb.org/Airspace/HR/HRV-2012.txt
hu airspace http://soaringweb.org/Airspace/HU/Hungary_2012.txt
il airspace http://soaringweb.org/Airspace/IS/israel_2014_v01.txt
lt airspace http://soaringweb.org/Airspace/LT/LT_2012.txt
lv airspace http://soaringweb.org/Airspace/LV/lv_2014_05_04.txt
nl airspace https://dl.dropboxusercontent.com/u/27739535/databases/EHv16_3a.txt
nz airspace http://soaringweb.org/Airspace/NZ/NZ.txt
po airspace http://soaringweb.org/Airspace/PL/2016-04-21_Polska.txt
ro unknown http://soaringweb.org/Airspace/RO/romanian_airspace_2016.02.22.txt
se airspace http://soaringweb.org/Airspace/SE/Sverige-2016-WP-rev0.TXT
si airspace http://soaringweb.org/Airspace/SI/si_2016.04.txt
sk airspace http://soaringweb.org/Airspace/SK/LZBB140529.txt
uk airspace http://www.nvgc.org.uk/UK2016.txt
us airspace http://soaringweb.org/Airspace/NA/allusa.v16.09-15.2.txt
# Updated daily

[pyrog]

There are a few things to consider when updating airspaces…

Using openflightmaps, a more reliable format could avoid this problem. See #559

[Turbo87]

does gdal/ogr have a parser for their data?

[pyrog]

gdal/ogr read partially:

• AIXM: http://gdal.gloobe.org/ogr/formats/gml.html?highlight=aixm
• OpenAIR: https://gdal.gloobe.org/ogr/formats/openair.html?highlight=airspace

Also, openflightmaps provide EPSG3857 Tiles and Mapbox Tiles (mbtiles).

the tiles are cached, but they should be in sync in the majority of cases since the airspace data hasn't been updated in quite a while.

See #561

[Turbo87]

this was fixed some time in the past weeks