OSM feature detection in SceneProc

nmj · December 23, 2018 at 7:39 PM

Hi,

Quite new to the scene but I'm very interested in messing around with some scenery creation using AeroScenery and SceneProc, I've managed to get most of it working just fine but now I'm trying to delve into SceneProc for a more custom template based on my area.

I've noticed that some tags don't work at all in SceneProc, I'm not sure if anyone else experienced with this or the author has a predefined list of key/values that are accepted.

map.osm

XML

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.55.5 2ca3f387">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="2018-12-23T17:40:02Z"/>
  <bounds minlat="51.5027600" minlon="-0.0198700" maxlat="51.5029900" maxlon="-0.0180700"/>
  <node id="4814234846" visible="true" version="1" changeset="48103574" timestamp="2017-04-24T21:25:44Z" user="spiregrain" uid="251221" lat="51.5027729" lon="-0.0181121"/>
  <node id="4814234847" visible="true" version="1" changeset="48103574" timestamp="2017-04-24T21:25:44Z" user="spiregrain" uid="251221" lat="51.5029599" lon="-0.0198448"/>
  <way id="489148282" visible="true" version="1" changeset="48103574" timestamp="2017-04-24T21:25:44Z" user="spiregrain" uid="251221">
    <nd ref="4814234846"/>
    <nd ref="4814234847"/>
    <tag k="natural" v="tree_row"/>
  </way>
</osm>

Display More

SceneProc.spc file

Code

ImportOGR|D:\AeroScenery\map.osm|*|*|NOREPROJ
UnloadFeatures|natural="tree_row"

OUTPUT:
18:35   UnloadFeatures    Information    Removing features that match natural="tree_row"
18:35   UnloadFeatures    Information    Removed 0 features

Funnily enough if I modify my map.osm to change the tag key from natural to highway it works fine.

Code

ImportOGR|D:\AeroScenery\map.osm|*|*|NOREPROJ
UnloadFeatures|highway="tree_row"

Output:
18:36   UnloadFeatures   Information     Removing features that match highway="tree_row"
18:36   UnloadFeatures   Information     Removed 1 features

I could potentially rename the ones I want, but before I do that I'm unsure if this is intended behavior? The default OSM map comes with natural=tree_row tag/key pair.

Other than that much thanks to Nick (AeroScenery), Arno (SceneProc) & Crispy (SceneProc Template).

Here's what I have so far (Edinburgh in progress):

[Blocked Image: https://i.ibb.co/xFGfqXM/ed.png]

Original Google Maps 3D Aerial - still some way to go, primarily fixing vegetation and additional buildings where there should be vegetation.

Thanks.

Jetjockey10 · December 23, 2018 at 9:16 PM

Pretty amazing that anyone can do this for almost any city on earth. Good job.

Regards,

Ray

Ian C · December 23, 2018 at 11:54 PM

Not sure whether you’re aware but if you’re cultivating in the UK you can use Ordnance Survey data. It’s pretty much the Gold Standard as far as cultivation goes - every building and tree clump is included. (That is unless something better has been devised since that I’m not aware of.)

Most of the work on OS Cultivation has been done by Spit40. Search for his latest configuration file. - the one that solves the problem of the shimmering buildings.

EDIT: Just found Spit40’s instructions and config file and put it in the post below .....

Cheers

Ian

Quote from Spit40

New Simplified Workflow for OS Data Cultivation

I got ScenProc working directly from the OS data without the need to process in QGIS. Here's the full working workflow.

Go to this page on OS Site
Pick the squares you want data for (hold ctrl down for multiple) - check you're downloading data type:vector download ESRI shape
Scroll to the bottom and click Continue, enter your details
A few minutes later you get a download link for files with names like "opmplc_essh_sd.zip" (this would be for SD grid square)
Unzip these files you receive and you'll find a data folder that contains files like: SD_Road.dbf, SD_Road.prj, SD_Road.shp, SD_Road.shx
The SHP files are the primary ones, but keep the PRJ and SHX in the same folder
Install ScenProc
Now edit the file paths on lines 11,12,13,14 & 93 of the following scenproc script, save it as something like "os_data_SD.spc"
Comment out lines 18-23 as necessary depending on whether your cultivation area includes airports that you have built cultivation exclusions for
Load your SPC file into scenproc then run it to convert the OS Data into a TOC file for adding to your airport cultivation

Code

# AeroFly FS 2 scenProc script
# with function to filter out objects on airport
# Author of scenProc Arno Gerretsen
# Script amended, modified and enhanced by Rodeo
# Tweaked by Spit40 v2 uses SHP files directly
# This example for the OS grid square "SD"
# --------------------------------------------------------
#
# Loading data for the 'SD' grid - see https://upload.wikimedia.org/w…_Survey_National_Grid.svg
# The only attribute I'm bothering to load is FEATCODE
ImportOGR|D:\OSD\opmplc_essh_sd\OS OpenMap Local (ESRI Shape File) SD\data\sd_Building.shp|*|FEATCODE|AUTODETECT
ImportOGR|D:\OSD\opmplc_essh_sd\OS OpenMap Local (ESRI Shape File) SD\data\sd_road.shp|*|FEATCODE;CLASSIFICA|AUTODETECT
ImportOGR|D:\OSD\opmplc_essh_sd\OS OpenMap Local (ESRI Shape File) SD\data\sd_woodland.shp|*|FEATCODE|AUTODETECT
ImportOGR|D:\OSD\opmplc_essh_sd\OS OpenMap Local (ESRI Shape File) SD\data\sd_importantbuilding.shp|*|FEATCODE|AUTODETECT

#
# Exclude cultivation from areas wher you have airport models
ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\airport1_exclude.kml|*|*|NOREPROJ
ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\airport2_exclude.kml|*|*|NOREPROJ
# KMLs loaded now exclude
AddAttributeIfInside|FEATCODE>=15014 and FEATCODE<=15028|FROMFILE="airport1_exclude.kml"|String;skip|yes
AddAttributeIfInside|FEATCODE>=15014 and FEATCODE<=15028|FROMFILE="airport2_exclude.kml"|String;skip|yes
UnloadFeatures|skip="yes"


#
# Split the features into a grid of 0.25 x 0.25 degrees
# Do not split buildings, but filter them into the right grid cells
# This is needed to make the filter processing of special buildings/buildings complete before the universe ends
SplitGrid|0.25|*|FEATCODE>=15014 and FEATCODE<=15028

#Filter out buildings that are special buildings as they're duplicated as normal buildings
FilterFeatures|FEATCODE=15014|FEATCODE>=15018 and FEATCODE<=15028|DISTANCE_CENTER|1

# Lights
# Place point features for the lights along roads
# B roads and minor roads - skip the lights
PlacePointsAlongLine|FEATCODE=15719|SINGLE|45;55|6;6|25|String;point|plight|hdg
PlacePointsAlongLine|FEATCODE=15710|SINGLE|45;55|6;6|25|String;point|plight|hdg
PlacePointsAlongLine|FEATCODE=15723|SINGLE|45;55|6;6|25|String;point|plight|hdg
PlacePointsAlongLine|FEATCODE=15743|SINGLE|40;40|4;4|20|String;point|slight|hdg
#    PlacePointsAlongLine|FEATCODE=15750|SINGLE|30;35|4;4|10|String;point|tlight|hdg
#    PlacePointsAlongLine|FEATCODE=15762|SINGLE|30;30|4;4|10|String;point|rlight|hdg
#
# Plants
# Place point features for the plants in forest polygons
# using spacing of 0.00025 degrees and full randomness
PlacePointsInPolygon|FEATCODE=15999|0.00025;0.00025|1.0;1.0


#
# Create AF2 plants
# More broadleaf than conifer in the UK
CreateAF2Plant|FEATCODE=15999 And FRAND >= 0.25|8;25|broadleaf
CreateAF2Plant|FEATCODE=15999 And FRAND < 0.25|10;20|conifer
#
# Create AF2 lights
CreateAF2Light|point="plight"|0.9;0.8;0.7|5|0;0;1|10
CreateAF2Light|point="slight"|0.9;0.7;0.5|3|0;0;1|8
CreateAF2Light|point="tlight"|0.6;0.6;0.7|3|0;0;1|6
CreateAF2Light|point="rlight"|0.9;0.6;0.3|3|0;0;1|6

# Identify the buildings, add a BUILDTYPE attribute
# 1 = Small building
# 2 = House / Terrace
# 3 = Commercial or wide modern house
# 4 = Church
# 5 = Other special buildling
# NB: I'm assuming that length is always the longest side !!
AddAttribute|FEATCODE=15014 And FAREARAT>0.7 And FLENGTH < 10|Integer;BUILDTYPE|1
AddAttribute|FEATCODE=15014 And FLENGTH >= 10 And FWIDTH < 20|Integer;BUILDTYPE|2
AddAttribute|FEATCODE=15014 And FLENGTH >=10 and FWIDTH>=20|Integer;BUILDTYPE|3
AddAttribute|FEATCODE=15025|Integer;BUILDTYPE|4
AddAttribute|FEATCODE>=15018 and FEATCODE<=15028 and FEATCODE<>15025|Integer;BUILDTYPE|5

# Split bigger buildings into smaller if area ratio<0.7
# We store but don't actually use the SPLITBUILD attribute
ReplacePolygonByBuildingRectangles|FEATCODE>=15014 and FEATCODE<=15028 and FAREARAT<=0.7 and FLENGTH>=10 and FWIDTH>=20|0.8;4;4|0.25;2.0;0.25|Integer;SPLITBUILD|1

#
# Create AF2 buildings
CreateAF2Building|BUILDTYPE=1|1|flat|residential
CreateAF2Building|BUILDTYPE=2|1|gable|residential
CreateAF2Building|BUILDTYPE=3|3|flat|industrial
CreateAF2Building|BUILDTYPE=4|5|gable|residential
CreateAF2Building|BUILDTYPE=5|4|flat|industrial

#Merge into single file for TOC output
MergeGrid

#
# Export the AF2 TOC file
ExportTOC|D:\OSD\opmplc_essh_sd|map_sd

Display More

nmj · December 24, 2018 at 1:15 AM

Thanks Ian,

I've tried that and it seems like some of the building data is a bit off, I am expecting grass and see a building instead in some places (especially gardens). Going to have a better look to see what is causing it, need to trim the .shp files first as even one square is way too large.

Ian C · December 24, 2018 at 1:32 AM

Quote from suprep

Thanks Ian,

I've tried that and it seems like some of the building data is a bit off, I am expecting grass and see a building instead in some places (especially gardens). Going to have a better look to see what is causing it, need to trim the .shp files first as even one square is way too large.

Occasionally there's a mismatch between the aerial images and the OS data if new houses have been built on a greenfield site recently. (The OS data is more up-to-date I guess.)

The OS grid squares are very large (100x100 km). I like to have my tree separation set to 0.0001 degrees but I don't think ScenProc can cope with the large number of trees generated for some of the squares (over 15 million) - usually if there are a large numbers of buildings too. It crashes. Square TQ (containing London) is particularly problematic. I've had to reduce the tree density in some cases.

nmj · December 24, 2018 at 1:45 AM

Fair enough, I'll do some cross-referencing with the OS data to see if it's indeed the right landscape compared to what I see in-game (ignoring photoreal scenery).

Edit:

Seems I've got it working now, PlacePointsInPolygon was missing a INHERITPARENTATTR at the end and I accidentally renamed the .toc file on the output so it wasn't overwriting the old one I had (figured that out after I literally trimmed the entire area down and wondered why everything was still showing up).

I just need to play with ReplacePolygonByBuildingRectangles now as some of the more complex polygon shapes are displaying in a weird way (and with different roof types unfortunately, but unsure if I can do anything for that).

Edit 2:

I think I'm at a point where complex polygons need to be done manually and the only way to have the same roof/building is to remove all the other ones from the building_textures folder as it seems AFS2 just chooses one at random. I also noticed some rotation issues (houses on a hill), they'd either be floating or be in the ground but I don't think much can be done about that unless I design them manually and place them on the map or so.

Anyhow I'm happy with the result now - Day Screenshot / Night Screenshot. Perhaps the next step is getting the Edinburgh building textures and finally the airport.

Hartman · December 24, 2018 at 1:05 PM

Quote from suprep

Here's what I have so far (Edinburgh in progress):

Quote from suprep

Anyhow I'm happy with the result now - Day Screenshot / Night Screenshot.

Really impressive!

lenidcamper · January 19, 2019 at 5:51 PM

Quote from nmj

I've noticed that some tags don't work at all in SceneProc, I'm not sure if anyone else experienced with this or the author has a predefined list of key/values

But back to the original question.

I too have never been able to get natural="tree_row" to work either with LineToPolygon or PlacePointsAlongLine

This may be a scenProc limitation.

lenidcamper · January 20, 2019 at 10:42 PM

There is a fix for the "tree_row" issue

In the scenProc\gdal-data\osmconf.ini

In the [lines] section replace

attributes=name,highway,waterway,aerialway,barrier,man_made,power,tunnel,bridge,railway

with

attributes=name,highway,waterway,aerialway,barrier,man_made,power,tunnel,bridge,railway,natural

Arno said he will update the config for the next release

cheers

/Stu

TomB · January 21, 2019 at 6:58 PM

Why does the attached OSM file not create a toc-file when running through ScenProc (template_ver6.spc). These are all natural=tree points. The data is from osm and filtered out with osm-filter so only natural=tree is kept in order to keep the file small. The region is north of Berne.

lenidcamper · January 21, 2019 at 8:02 PM

I am not sure what is in template_ver6.spc but

ImportOGR|map.osm|*|*|NOREPROJ

CreateAF2Plant|natural="tree"|10;12|0|broadleaf

ExportTOC|.\|cultivate

produces trees from that osm snippet

TomB · January 21, 2019 at 9:33 PM

Thanks a lot. I will try tomorrow as I am not at my pc right now.

Template ScenProc

TomB · January 30, 2019 at 7:08 PM

natural=tree_row does still not work with scenproc latest dev release.

What is wrong with that:

PlacePointsInPolygon|natural="tree_row" And NOT landuse="forest"|0.00009;0.00009|1.0;1.0|INHERITPARENTATTR

CreateAF2Plant|natural="tree_row"|10;15|0|broadleaf

lenidcamper · January 30, 2019 at 7:14 PM

natural="tree_row" is a line feature

LineToPolygon|FTYPE="LINE" And natural="tree_row"|3|String;landuse|tree_row

PlacePointsInPolygon|landuse="tree_row"|0.00005;0.00005|1.0;1.0|INHERITPARENTATTR

CreateAF2Plant|landuse="tree_row" And FRAND >= 0.2|4;12|0|broadleaf

CreateAF2Plant|landuse="tree_row" And FRAND < 0.2|5;10|0|conifer

will create a 3m polygon along the tree_row features and place points in the landuse="tree_row" polygons

The PlacePointsAlongLine directive is not well suited for tree cultivation.

/Stu

TomB · January 30, 2019 at 10:03 PM

Thanks lenidcamper, works fine now.

Below is a screenshot from an area with defined tree rows in JOSM (orginal osm data from bbbike). The second picture shows the same region in AFS2 Scenery Editor after the scenproc process. The forests and buildings are not shown as I filter them out for the tests. There are very few tree_rows in osm data but one can draw them yourself (a line defined as tree_row). This is faster than drawing single trees. Can be used for alleys too.

And finally the view out of the lovely R22.

New Simplified Workflow for OS Data Cultivation

Tags