UK Cultivation - Ordnance Survey instead of Open Streetmap

  • Update 30/12/17: Save time and skip to this post where a full simplified workflow is provided that avoids a Graphical Information System (GIS)

    I'm very impressed with my first attempt at cultivation, but the data was a bit thin for where I am in the UK, so I looked for alternatives. I found the Ordnance Survey site who offer a free download. I haven't got as far as data into FS2 yet (bit of a learning curve), but thought I'd share work in progress in case others want to contribute. This is where I'm up to:

    1. Go to the OS Site
    2. Pick the squares you want data for (ctrl for multiple) - check you're downloading data type:vector download ESRI shape
    3. Scroll to the bottom and click Continue, enter your details
    4. A few minutes later you get a download link
    5. Now you need a GIS. So download and install QGIS
    6. Now you need to get the OS data into the GIS, so unzip the OS download and follow these instructions

    Compared to what I get from OSM this is amazing. Next step is to try and get it into FS2 via ScenProc. I think I need to export to a KML, but once I've done that it would be great to do a more sophisticated mapping job, relying not just on building dimensions to guess at what type of building it is, but to use all this meta data.

    Higher level

    Zoom in a bit

    Metadata

  • ScenProc has all the tools to work directly from shapefile vector format. You only need a GIS to explore your data and find how they're referenced for setting according calls/syntax in ScenProc steps (see the manual). Not a straightforward job, especially with QGIS that quickly gets unstable with large databases...

    Cheers

    Antoine

    Config : i7 6900K - 20MB currently set at 3.20GHz, Cooling Noctua NH-U14S, Motherboard ASUS Rampage V Extreme U3.1, RAM HyperX Savage Black Edition 16GB DDR4 3000 MHz, Graphic Card Gigabyte GeForce GTX 1080 8GB, Power supply Corsair RM Series 850W, Windows 10 64 bit.

  • ScenProc has all the tools to work directly from shapefile vector format. You only need a GIS to explore your data and find how they're referenced for setting according calls/syntax in ScenProc steps (see the manual). Not a straightforward job, especially with QGIS that quickly gets unstable with large databases...

    Cheers

    Antoine

    Thanks yes I see that now - references to shp files in the manual. Once I do get my head around the nature of the data though I should be able to create a scenproc file like Rodeo's that others could use to pull in as much of the UK as they want. I think it may also need OSM data though as I'm not seeing street lighting in the OS Data.

  • Lots of progress today. I managed to decode the attributes in the OS OpenData downloads and processed the shp files that they provide directly in scenProc. As a first pass I used a modified version of Rodeo 's scenProc file (attached), but I hope to improve this with more UK style buildings - red brick and terraces. Unfortunately the data that OS provide at this level uses a single classification for buildings and woodland.

    It's not quite there yet though as I hit a stumbling block with the projection. OS maps don't store co-ordinates in the usual way but use the OS grid system. I found a conversion for it here, but something is wrong and all the houses are a few metres out of alignment. More investigation needed. Maybe it's a splitgrid issue. Any ideas anyone? Anyway, that's why my screenshots are at night !

  • Solved the co-ordinate problem of OS Data

    The way I did it was to:

    1. Take each SHP file as supplied by OS (building, woodland, road)
    2. Load it into QGIS (load all at once actually)
    3. For each one (layer), Save As a KML file and modify the CRS (cordinate system) from Ordnance Survey to ESPG

    Then I loaded these KMLs into procScen. It works perfectly. Its really quite amazing. OS maps have just about every single building in the UK, so you'll even get a small hut appearing on a golf course or bowling green. Now I need to modify the appearance for UK style buildings.

    The other thing I might try is to request a free Discovery Licence to try out their more detailed vector maps that have height data and additional detail. It would be good to autogen a tower or tall building differently from a home.

  • Finally the UK has terraced houses

    I've been doing more tweaks to Rodeo 's scenProc file and now we finally have rows of terraced houses, a familiar feature in the UK. I've just realised they need to be a bit taller but that's an easy fix. I've also noticed that FS2 boosts the vibrance of cultivated buildings quite a bit compared to the BMPs. I've been trying to take out some of the saturation of the red roofs but it's still there.

    The other thing I've been experimenting with is ReplacePolygonByBuildingRectangles in order to show some of the more interesting irregularly shaped buildings, and I came up against the "feature" that FS2 currently randomises its choice of building as you can see above where a commercial building now has a mixture of shades to its roof.

    Here's my latest scenProc file in case others want to use it with OS OpenData.

    # AeroFly FS 2 scenProc script

    # with function to filter out objects on airport

    # Author of scenProc Arno Gerretsen

    # Script amended, modified and enhanced by Rodeo

    # Tweaked by Spit40

    # --------------------------------------------------------

    #

    # KML files from OS OpenData, opened in QGIS and exported with EPSG co-ordinate system

    # Loading data for the 'SD' grid - see https://upload.wikimedia.org/wikipedia/comm…tional_Grid.svg

    # The only attribute I'm bothering to load is FEATCODE

    ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_building.kml|*|FEATCODE|NOREPROJ

    ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_road.kml|*|FEATCODE;CLASSIFICA|NOREPROJ

    ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_woodland.kml|*|FEATCODE|NOREPROJ

    ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_importantbuilding.kml|*|FEATCODE|NOREPROJ

    #

    # Load Shapefile that will be excluded from processing

    # Skipped for now

    #ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\exclude.kml|*|*|NOREPROJ

    #

    # Split the features into a grid of 0.25 x 0.25 degrees

    # Do not split buildings, but filter them into the right grid cells

    # I need to ask Rodeo why he commented this out

    # SplitGrid|0.25|*|FEATCODE="15014"

    #

    # Lights

    # Place point features for the lights along roads

    # B roads and minor roads - skip the lights

    PlacePointsAlongLine|FEATCODE=15719|SINGLE|45;55|6;6|25|String;point|plight|hdg

    PlacePointsAlongLine|FEATCODE=15710|SINGLE|45;55|6;6|25|String;point|plight|hdg

    PlacePointsAlongLine|FEATCODE=15723|SINGLE|45;55|6;6|25|String;point|plight|hdg

    PlacePointsAlongLine|FEATCODE=15743|SINGLE|40;40|4;4|20|String;point|slight|hdg

    # PlacePointsAlongLine|FEATCODE=15750|SINGLE|30;35|4;4|10|String;point|tlight|hdg

    # PlacePointsAlongLine|FEATCODE=15762|SINGLE|30;30|4;4|10|String;point|rlight|hdg

    #

    # Plants

    # Place point features for the plants in forest polygons

    # using spacing of 0.00025 degrees and full randomness

    PlacePointsInPolygon|FEATCODE=15999|0.00025;0.00025|1.0;1.0

    #

    # Filter out the buldings, lights and plants

    # that are within the exclude polygons

    # Skip for now

    #AddAttributeIfInside|building="*"|FROMFILE="exclude.kml"|String;skip|yes

    #AddAttributeIfInside|point="light"|FROMFILE="exclude.kml"|String;skip|yes

    #AddAttributeIfInside|FTYPE="POINT" And landuse="forest"|FROMFILE="exclude.kml"|String;skip|yes

    #UnloadFeatures|skip="yes"

    #

    #

    # Create AF2 plants

    # More broadleaf than conifer in the UK

    CreateAF2Plant|FEATCODE=15999 And FRAND >= 0.25|8;25|broadleaf

    CreateAF2Plant|FEATCODE=15999 And FRAND < 0.25|10;20|conifer

    #

    # Create AF2 lights

    CreateAF2Light|point="plight"|0.9;0.8;0.7|5|0;0;1|10

    CreateAF2Light|point="slight"|0.9;0.7;0.5|3|0;0;1|8

    CreateAF2Light|point="tlight"|0.6;0.6;0.7|3|0;0;1|6

    CreateAF2Light|point="rlight"|0.9;0.6;0.3|3|0;0;1|6

    # Identify the buildings, add a BUILDTYPE attribute

    # 1 = Small building

    # 2 = House / Terrace

    # 3 = Commercial or wide modern house

    # 4 = Church

    # 5 = Other special buildling

    # NB: I'm assuming that length is always the longest side !!

    AddAttribute|FEATCODE=15014 And FAREARAT>0.7 And FLENGTH < 10|Integer;BUILDTYPE|1

    AddAttribute|FEATCODE=15014 And FLENGTH >= 10 And FWIDTH < 20|Integer;BUILDTYPE|2

    AddAttribute|FEATCODE=15014 And FLENGTH >=10 and FWIDTH>=20|Integer;BUILDTYPE|3

    AddAttribute|FEATCODE=15025|Integer;BUILDTYPE|4

    AddAttribute|FEATCODE>=15018 and FEATCODE<=15028 and FEATCODE<>15025|Integer;BUILDTYPE|5

    # Split bigger buildings into smaller if area ratio<0.7

    # We store but don't actually use the SPLITBUILD attribute

    ReplacePolygonByBuildingRectangles|FEATCODE>=15014 and FEATCODE<=15028 and FAREARAT<=0.7 and FLENGTH>=10 and FWIDTH>=20|0.8;4;4|0.25;2.0;0.25|Integer;SPLITBUILD|1

    #

    # Create AF2 buildings

    CreateAF2Building|BUILDTYPE=1|1|flat|residential

    CreateAF2Building|BUILDTYPE=2|1|gable|residential

    CreateAF2Building|BUILDTYPE=3|3|flat|industrial

    CreateAF2Building|BUILDTYPE=4|5|gable|residential

    CreateAF2Building|BUILDTYPE=5|4|flat|industrial


    #

    # Export the AF2 TOC file

    ExportTOC|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd|map_sd

  • Rodeo I think there are few more tricks we can do here to fine tune things, like using Google Earth to select/create areas and KML files and use those to tag buildings in those areas to indicate that they are all caravans or all high rise or all country cottages v. suburban. It would add a little benefit now but much more if we could have more cultivation objects. Is cultivation hardwired to industrial/com and residential/res or can we introduce new codes and create objects to match them? If we could go as far as adding XREF objects this way that would be even better.

  • Phil, all cultivation tests we do is good for opening the path to our future coverage, i.e. find and mix suitable sources, get familiar with ScenProc multiple features and algorithms, as the case may be create new algorithms when it makes sense.

    But please keep in mind that IPACS is rethinking the cultivation philosophy to make it suited for large areas covering. This involves very probably new sets of objects and hopefully the ease of their creation.

    One can have a first glance at the upcoming format in Orbx's Chicago Meigs add-on, where the binary cultivation files obviously load and unload correctly while flying in an out, ensuring far visibility and smooth behaviour, unlike the text format we're currently stuck with.

    Additionally, it is of paramount importance that the upcoming compiled cultivation files format can be edited, for instance by de-compiling and uploading into ScenProc, otherwise there's no chance for the community to build-up and share sceneries.

    For instance you can release in the future a beautiful cultivation for UK, but if somebody else subsequently creates Biggin Hill and a third person a collection of UK Landmarks, their integration will either necessitate to locally edit cultivation or to have an exclude file system to disable cultivation objects where 3D objects are placed above.

    Cheers

    Antoine

    Config : i7 6900K - 20MB currently set at 3.20GHz, Cooling Noctua NH-U14S, Motherboard ASUS Rampage V Extreme U3.1, RAM HyperX Savage Black Edition 16GB DDR4 3000 MHz, Graphic Card Gigabyte GeForce GTX 1080 8GB, Power supply Corsair RM Series 850W, Windows 10 64 bit.

  • Indeed. Further advancements will take place for all cultivation efforts to make things easier and better for larger deployments.

    Just like the geoconvert tool, it's all work on progress :)

    IPACS Development Team Member

    I'm just a cook, I don't own the restaurant.
    On behalf of Torsten, Marc, and the rest of the IPACS team, we would all like to thank you for your continued support.

    Regards,

    Jeff

  • Thanks for the info gents. I'm reminded of the wisdom of the ancients - actually a very relevant aphorism for almost everyone following the journey of FS2.

    https://i.pinimg.com/originals/f0/5…02705b7e350.jpg

  • Thanks for the info gents. I'm reminded of the wisdom of the ancients - actually a very relevant aphorism for almost everyone following the journey of FS2.

    https://i.pinimg.com/originals/f0/5…02705b7e350.jpg

    This is very true.

    IPACS Development Team Member

    I'm just a cook, I don't own the restaurant.
    On behalf of Torsten, Marc, and the rest of the IPACS team, we would all like to thank you for your continued support.

    Regards,

    Jeff

  • I'm very impressed with my first attempt at cultivation, but the data was a bit thin for where I am in the UK, so I looked for alternatives. I found the Ordnance Survey site who offer a free download. I haven't got as far as data into FS2 yet (bit of a learning curve), but thought I'd share work in progress in case others want to contribute. This is where I'm up to:

    1. Go to the OS Site
    2. Pick the squares you want data for (ctrl for multiple) - check you're downloading data type:vector download ESRI shape
    3. Scroll to the bottom and click Continue, enter your details
    4. A few minutes later you get a download link
    5. Now you need a GIS. So download and install QGIS
    6. Now you need to get the OS data into the GIS, so unzip the OS download and follow these instructions

    Compared to what I get from OSM this is amazing. Next step is to try and get it into FS2 via ScenProc. I think I need to export to a KML, but once I've done that it would be great to do a more sophisticated mapping job, relying not just on building dimensions to guess at what type of building it is, but to use all this meta data.

    Higher level

    Zoom in a bit

    Metadata

    Hi Phil

    The link in step 6 "follow these instructions" seems to be broken.

    Cheers

    Ian

  • I was about to write up a simplified approach that avoids QGIS. Its not yet tested but Arno at SendProc suggested this:

    • Use the SHP files as-is, skipping steps 5 & 6
    • Change the SCP file from post #7 to replace lines as below (changes in 3 places)

    Old

    ImportOGR|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_building.kml|*|FEATCODE|NOREPROJ

    New

    ImportSHP|D:\aerofly_fs_2_sdk_tools_oct17\Cultivation\OS Download\opmplc_essh_sd\sd_building.shp|*|FEATCODE|AUTODETECT

  • Spit40

    Hi again Phil

    Just to let you know that I tried the above and wow - I can see what you mean about the quality of the OS data!

    The only thing I found was that ScenProc doesn't seem to like "ImportSHP" - I had to change it back to "ImportOGR" (while keeping the other changes you made in the new version above). Also I must have done something wrong with the Woodland KML file because I didn't get any trees - but I'm fairly confident I'll be able to fix that.

    Now I need to study what you have already done to make the buildings more suitable for the UK - the toning down of the red roofs and non-white (maybe reddish brown) walls etc. Generally they need to be more dun coloured and subdued.

    But, all in all, I'm more than satisfied with the results so far. Can't wait to see the trees in place too - probably tomorrow, as it's very late.

    Cheers

    Ian

    Edited once, last by Ian C (December 20, 2017 at 2:30 AM).

  • Great news Ian. I understand your 'wow'. The trees must be a simple thing. As for the UK buildings i photoshopped the bmp files and reconverted. You can have a copy if you like. I still find they come out very vivid though. FS2 boosts the vibrancy even though i toned down a lot.

    Thanks for confirming importogr is the way to go.

  • Great news Ian. I understand your 'wow'. The trees must be a simple thing. As for the UK buildings i photoshopped the bmp files and reconverted. You can have a copy if you like. I still find they come out very vivid though. FS2 boosts the vibrancy even though i toned down a lot.

    Thanks for confirming importogr is the way to go.

    Hi Phil - yes, it would be great to have the reconverted UK buildings. I assume they go in the building_textures folder? Do you need an email address from me? I'll send it to you anyway via this site in case you need it.

    Ian

  • Hi Phil

    Ecky thump lad, reet grately what your doing

    How about asking the gods if you can post them for everyone to use, must be someway we can do this , no point reinventing the wheel every time. Will try this cultivation lark eventually, when i get some time off from EGGP.... never again

    Cultivation for a car park, now that would help

    Steve