Building creation with scenProc

  • First I want to congratulate Arno the author of scenProc for his powerfull tool. And thanks him to have extended it with AFS features knowing it was initialy a tool focused on FSX/P3D.

    I am particularly attached to the building generation because it is for me a crucial point to be able to generate realistic scenes.

    And in th generation of buildings an essential point is to be able to have the heights of buildings closest to reality.

    That is why some time ago I put a lot of effort in the information of the heights of buildings in OSM. At the time, I developed scritps allowing the automatic import of these data into OSM. My scripts were based on DEM + DSM sources available in open data and by doing the subtraction I managed to determine a maximum height for each building already imported into OSM (thanks to the cadastre). There were some difficulties to manage like trees or antennae that had to be ignored to not have false values. But in the end it worked pretty well and I managed to inform the heights for all the buildings of the cities of Nice and Montpellier in France. I had also done a script that had allowed me to fill the height on almost half of the Parisian buildings, in this case it was the number of floors and not the height that I filled.

    The good news is that sceneProc is able to handle the height attribute in addition to the building: levels attribute by using a scale ratio of 0.3 (assuming that a floor is about 3 meters high).

    On the other hand, it implies having to split all the building rules in sceneProc. There is already a doubling of rules to manage the existence or not of the attribute building: levels, but we must add another one to also manage the existence or not of the attribute height.

    If we consider the following rule:

    Code
    CreateAF2Building | building = "apartments" And FAREARAT> 0.7 And FAREA> = 700 | building_levels | 1 | gable | residential | 0

    It becomes:

    Code
    CreateAF2Building | building = "apartments" And FAREARAT> 0.7 And building_levels = "*" And FAREA> = 700 | building_levels | 1 | gable | residential | 0
    CreateAF2Building | building = "apartments" And FAREARAT> 0.7 And NOT building_levels = "*" And height = "*" And FAREA> = 700 | height | 0.3 | gable | residential | 0
    CreateAF2Building | building = "apartments" And FAREARAT> 0.7 And NOT building_levels = "*" And NOT height = "*" And FAREA> = 700 | 3; 6 | 1 | gable | residential | 0

    But at the end it makes a lot of rules and it's not very readable. Would not there be a better way?

    Also if sceneProc could have a mode that would prevent it from working twice on the same OSM entity it would also simplify the rules (no need to add the conditions "And Not XXX").

    Another question if Arno read that thread: no plan to support "generic" buildings ? By generic I mean buildings which can adapt to any footprint and any height. Of course this is not a trivial point and it will be difficult to have buildings as pretty as those that are "pre-calculated" but at least we would have all buildings that exist in AFS even when their shape / footprint is not standard (ie. rectangular). So we could have in our AFS sceneries ALL the buildings that exist in the real world (well, in OSM) and with the correct shape and correct height! But currently a lot of them are ignored because they have a non rectangular footprint (I guess that most of us use the FAREARAT>0.7 condition).

    Otherwise I noticed a strange behavior: the swimming pools in OSM generate buildings ! The pools have nothing to do with the buildings (there is just leisure = swimming_pool, no building tag) so it really looks like a bug.


    Otherwise I noticed a strange behavior: the swimming pools in OSM generate buildings ! The pools have nothing to do with the buildings (there is just leisure = swimming_pool, no building tag) so it really looks like a bug.

    I can see the same issue with playgrounds (leisure=pitch) but only for certain sports such as tennis or beach volley (not for other such as football, athletics..)

  • I developed scritps allowing the automatic import of these data into OSM. My scripts were based on DEM + DSM sources

    You are so right with your findings!

    Maybe sending a copy of this to arnog s blog directly?

    https://www.fsdeveloper.com/forum/threads/…c.441173/page-7

    I am interested in your house level extract scripts. Would you mind sharing them?

    They really would add a lot to the scenery realizm!

    Cheers,

    Tom

    Cheers, Thomas


  • Hi turman,

    I'm pretty sure I came across a discussion on another forum about this subject of a script to import the heights of OSM data buildings about the city of Nice. It was a few years ago and it was in French if I remember correctly. I found this discussion particularly interesting at this time because I was looking for how to add building height data to my sources for creation of a more realistic autogen building. The DTM and DSM data comparison is a very good idea and surely allows to have an accurate result but I quickly stumbled on the lack of open data enough precise to be able to launch the work. Height accuracy needed to create a scenery for a simulator is not as demanding as adding this info to the global OSM data. So I imagined other possibilities...

    First idea, in the case where we only have a DSM sufficiently precise, the idea was to take the measure of several points taken randomly into cadastral polygon from OSM or cadastral data and keep the highest point then compare with the height data taken on several randomly chosen points around the same polygon and keep the lowest. so for each building I could theoretically recover the height (difference between max and min) and the altitude of its base (min). Working directly on a DSM raster it's not so hard to write a script to do that but there is always the problem of lack of sources. To overcome this lack, I thought to exploit the data of google-earth in areas covered by photogrametrie. Exploiting data from a hight number of points in GE is theoretically possible. A hight resolution DSM raster of an area could be generated like this. put aside the complexity of implementation, slowness of measurement, view deformation at high zoom, problem is that these data are not open because processing from protected by the copyright data.

    My other idea that I finally used for Martinique is much less accurate but already help to reinforce realism and buildings variety. Idea was exploiting roofs color of buildings. With Raster's functions of scenproc It is possible to add to the average color of each polygon so of each building. Sure there is a lot of imprecision and errors linked to ground photo quality, projected shadows, imagery & vector shifting but it helps to give variety., For residential areas, when I had rather red, blue or green roofs, it derived from simple homes (1 or 2 level) and when I had gray roofs, it crowded this with other data (type: residence, commercial, building area, etc.) I could try to vary the height of buildings for a start in realism...

    Edited 2 times, last by vogel69 (July 29, 2019 at 8:03 AM).

  • Hi turman,

    I'm pretty sure I came across a discussion on another forum about this subject of a script to import the heights of OSM data buildings about the city of Nice. It was a few years ago and it was in French if I remember correctly.

    It must be a discussion with me because I had actually discussed this a lot on the french OSM forums or mailing list in summer 2016. Internet is a small village ! ;)

    Here are some screenshots showing the results.

    Nice:

    Montpellier:

    Paris (but in that case it was a very different process because the height infos was already available as open data):


    The above screenshot are taken from https://demo.f4map.com which is for me the best way to visualize OSM buildings in 3D. Off course there's no details and textures but the footprints and heights are fully respected. I'd love to have the same thing in AFS !

    Edited once, last by turman (July 29, 2019 at 12:00 PM).

  • I'd have another suggestion for scenProc (maybe it already exists): it could be usefull if users can define a list of OSM features that must be excluded from the building generation. That way there is no need anymore to make some KML exclusions for all the implementations of that feature ! For example if I want to generate storage tanks with ObjectGen I could add the man_made='storage_tank' feature into that list.

  • This would also be helpful for airports. Ortho4XP reads in airports from an open database (you can see it in the log), so the info must be available. This might be helpful for scenproc to exclude airports automatically (as an option) as well.

    Kind regards, Michael

    Intel i7-6700K 4.0 GHz / Asus MAXIMUS VIII RANGER / Kingston 32 GB DDR4 / Samsung SSD M.2 500 GB + Samsung SSD 1 TB + Intel SSD 500 GB (AeroflyFS2) + WD HD 6 TB / EVGA GTX 1080Ti 11 GB / LG 34UM95 3440 x 1440 / HP Reverb / Win 10/64

  • Apparently it's already the case, see the comment in the template file from Crispy:

    Quote

    # LOAD SHAPEFILE that will be excluded from processing.

    # - By default this script loads all .kml exclude files in the folder. If you only want specific files, then use

    # the filename in place of "*.kml", You may also need to do the same for any of the "AddAttributeIfInside"

    # lines you are using in the FILTER section below.

    # - Buildings in airport areas are now automatically excluded, so no exclude file is required for these.

  • Ah thanks, that's great to know!

    Kind regards, Michael

    Intel i7-6700K 4.0 GHz / Asus MAXIMUS VIII RANGER / Kingston 32 GB DDR4 / Samsung SSD M.2 500 GB + Samsung SSD 1 TB + Intel SSD 500 GB (AeroflyFS2) + WD HD 6 TB / EVGA GTX 1080Ti 11 GB / LG 34UM95 3440 x 1440 / HP Reverb / Win 10/64

  • You are so right with your findings!

    Maybe sending a copy of this to arnog s blog directly?

    https://www.fsdeveloper.com/forum/threads/…c.441173/page-7

    You're right Tom, as I'm not sure that Arno will pass on the Aerofly forum in a near future I have tried ton contact him via https://www.fsdeveloper.com. But I'm unable to register, no confirmation mail received. If you or other people having an account on his forum could notice him of that thread it would be great !

    It would be a shame to let him go on vacation while we have plenty of work to give to him for working on AFS! ^^

  • So we have to wait a little bit I guess...

    About the exclusion there is actually that line in the template file which prevents to have buildings inside aerodrome:

    Code
    # FILTER out buildings
    # The line below automatically removes buildings within your airport area for the majority of airports worldwide. 
    AddAttributeIfInside|building="*"|aeroway="aerodrome"|String;skip|yes
    # The line below removes buildings that are in the area covered by your exclude file. This should only be needed 
    # if you want to exclude buildings outside an airport area.
    AddAttributeIfInside|building="*"|FROMFILE="*.kml"|String;skip|yes

    So if the aerodrome polygon is well defined in OSM there shouldn't be any problem.

    To check it you can edit OSM data with an editor or more simply you can do it from the http://www.openstreetmap.org interface: just click on the button with "?" on the right, and then click on place (anywhere!) in the aerodrome. Select the aerodrome in the item list => its outine is displayed !

  • Thanks Dave ! :thumbup:

    if I try to resume my questions / suggestions:

    - a bug with buildings generated on playgrounds (leisure=pitch) ?

    - a plan to have "generic" buildings (for any footprint and any height) in the future ?

    - a mode that would prevent scenProc from working twice on the same OSM building which could simplify the rules (no need to add the conditions "And Not X=Y") ?

    I have others about vegetation but I'll launch a separate thread for that..

  • Hi,

    I was on vacation, so hence the late reply (also I don't check this forum that often).

    I understand what you mean with the duplication of the lines. That's not very elegant indeed. Let me think a bit about it, but the only way I see to prevent that is to modify the CreateAF2Building step so that you can specify a level attribute, height attribute and fallback level value in case no attribute exists. But that would make the step a bit more complex to use for some people as well. So let me think about that.

    Making scenProc only process each feature once does not fit in the concept of how scenProc works. So that will not happen.

    For the "generic buildings" my impression is that AF2 doesn't support them (like FSX/P3D don't). It is only possible to specify rectangular buildings in the TOC file. So that's why i can't export other shapes. Or do you see options for that?

    Do you see that swimming pool issue everywhere or just with some swimming pools? If I know a location where it happens I might be able to reproduce it here.

    The filter that you use to select the features to process can also be used to exclude certain features from processing. On the other hand if you don't want to use certain features as well, you can also restrict the import in the ImportOGR step already with the required attributes value in there.

    Arno

  • I understand what you mean with the duplication of the lines. That's not very elegant indeed. Let me think a bit about it, but the only way I see to prevent that is to modify the CreateAF2Building step so that you can specify a level attribute, height attribute and fallback level value in case no attribute exists. But that would make the step a bit more complex to use for some people as well. So let me think about that.tes value in there.

    Yes that's would a good thing to add a new attribute in CreateAF2Building for the height, in addition to the level. I don't think it would make the process more complex if the fallback is well explained.

    In fact we could imagine a "one line" solution with two fallbacks for both levels and height: if (height value exists) use it else if (levels value exists) use it else use the hardcoded value (or hardcoded values range).

    That way we could divide by 3 the number of lines for the creation of buildings (or by 2 for those which take into account only the number of floors). I think it's really worth it!

    Making scenProc only process each feature once does not fit in the concept of how scenProc works. So that will not happen.

    Ok.

    For the "generic buildings" my impression is that AF2 doesn't support them (like FSX/P3D don't). It is only possible to specify rectangular buildings in the TOC file. So that's why i can't export other shapes. Or do you see options for that?

    No sorry I don't know any options. In fact I did not even know that we could only define rectangular buildings in AFS2.

    It should have confirmation from IPACS. Perhaps it's a voluntary choice for performance reasons?

    That said if we look at some commercial sceneries from IPACS or ORBX we can obviously see non-rectangular buildings. But maybe in this case it does not go through TOC files, I admit that I have not dug the subject at all ...

    Do you see that swimming pool issue everywhere or just with some swimming pools? If I know a location where it happens I might be able to reproduce it here.

    I'm not sure it concerns all the swimming pools but most of them.

    Right now I don't have access to my PC with AFS but I'm pretty sure it's the case for theses ones:

    https://www.openstreetmap.org/way/230758739

    https://www.openstreetmap.org/way/69789322

    But note that it doesn't concern swimming pools only but most of sports playgrounds too !

    For example this tennis court: https://www.openstreetmap.org/way/554017742

    Or this beach volley pitch: https://www.openstreetmap.org/way/330042924

    For me it's a major issue and I am really surprised that the problem has not already been notified. Maybe the bug is local to my system. For now I cannot tell you the exact version of scenProc I used but it was a dev version fetched in the begin of the summer.

    Edited 3 times, last by turman (August 28, 2019 at 3:32 PM).

  • Quote

    In fact we could imagine a "one line" solution with two fallbacks for both levels and height: if (height value exists) use it else if (levels value exists) use it else use the hardcoded value (or hardcoded values range).

    Since the TOC file specifies the amount of levels, I was thinking of an order levels - height - fallback.

    Quote

    That said if we look at some commercial sceneries from IPACS or ORBX we can obviously see non-rectangular buildings. But maybe in this case it does not go through TOC files, I admit that I have not dug the subject at all ...

    Those are probably custom objects, instead of cultivation buildings then. The last category can have any shape.

    Let me try to reproduce the issue on my side.

    Arno

  • Arno has made some changes to the CreateAF2Building command, to allow a fallback value when buildings levels and or height are not present in OSM data. The only issue is that this new command (and the old one as well) doesn't work well with buildings generated by the ReplacePolygonByBuildingRectangle command This command splits an individual larger building up into smaller rectangular buildings to better represent the overall building shape. But if you use the CreateAF2Building command to assign random heights for your buildings, the various smaller rectangle that make up the bigger building will have differing heights instead of a consistent height. The only solution is to set the building height before the ReplacePolygonByBuildingRectangle command, so that all the smaller rectangles generated by this command use the original building height for every piece of the larger building. Once this is done the CreateAF2Building shouldn't be allowed to vary the building height, or you defeat the purpose of using the ReplacePolygonByBuildingRectangle command.

    If not using ReplacePolygonByBuildingRectangle command then setting a range of heights in the new CreateAF2Building command will work fine by itself.

    My latest template handles building heights by default and works with Arno's latest change. But if you want to implement this solution into your own scripts, I have pasted sample code below. The ReplacePolygonByBuildingRectangle lines would sit somewhere between the 2 blocks of code below. You can create more lines like the one shown with <<<--- , to assign different building height ranges to different building types. The example just uses building="*" for simplicity.


    # - Identify buildings with height/level recorded in OSM

    AddAttribute|building="*" And building_levels="*"|String;levelid|level

    AddAttribute|building="*" And NOT building_levels="*" And height="*"|String;levelid|height

    # - Indentify buildings with height/level NOT recorded in OSM.

    AddAttribute|building="*" And levelid="height" And height="*m*"|String;levelid|unknown

    AddAttribute|building="*" And NOT building_levels="*" And NOT height="*"|String;levelid|unknown

    AddAttribute|building="*" And building_levels="0"|String;levelid|unknown

    # - Set Height/Level for Buildings (where no OSM height/level is available)

    AddAttribute|building="*" And levelid="unknown"|String;building_levels|RND(5,10) <<<------

    # - Create Buildings (updated command)

    CreateAF2Building|building="*"|building_levels|height|0.3|1;1|flat|residential|0

    Thanks, Chris

    Win 10 64-bit, 24GB RAM, i5-9400F @ 3.9, 6GB Nvidia RTX-2060

  • Hi,

    As Chris mentioned in his post I have implemented the changes in the CreateAF2Building step now. The manual has been updated and shows the new arguments the step has. There is an argument for the floors attribute, an argument for the height attribute, an argument for the scale (applied to the height attribute value) and an argument for the fallback height.

    Arno