Posts by Ian C

  • Let's localize the building textures

    I haven't really been following posts on buildings and building textures, so forgive me if I'm asking something that has already been answered.

    My question is ..... how do you actually create the TTX files that go in the "building_textures" folder? I assume you start with a BMP or similar, but how do you get from there to TTX? I've done a bit of searching on the forum but haven't found anything so far.

  • Creating terrain heightmaps

    Quote from Spit40

    Love it! The UK has free elevation data too, but I don't know how easily contours are converted to heightmaps. I bet arnog could do it !

    https://www.ordnancesurvey.co.uk/opendatadownlo…cts.html#terr50

    Hi Phil

    The Ordnance Survey 50 m data that you mention ("OS Terrain 50") can be downloaded in ASCII (.asc) format, so it can be processed in pretty much the same way as I've processed the DEFRA 1m data in my tutorial above. The 1 m data has quite a lot of holes in it, so I was thinking of using the 50 m data as a base layer. I'm also currently investigating the Ordnance Survey 5 m data ("OS Terrain 5"). You have to fill in a form saying how you intend to use it etc - so I'm hoping they give me the go-ahead (without paying anything preferably!) If I can get hold of that it would probably be a better option than the 1 m data because, being from Ordnance Survey, I imagine it is complete and of good quality. The 1 metre data could then perhaps be used as a level 14 top layer in the areas where the scenery would benefit from it.

    Anyway, I tried the 50 m data out around Beachy Head and took 3 screen shots for comparison. The top pic is with the default mesh, the middle pic is with the OS 50 m mesh, and the bottom pic is with the DEFRA 1 m mesh. The 50 m mesh is actually a considerable improvement on the default mesh so I am hoping that, where there are holes in the 1 m data, the mesh boundary / fault line won't be so obvious with a 50 m base layer. That's the theory anyway!

  • Creating terrain heightmaps

    As promised, these are the comparison screenshots. They show the Seven Sisters - vertical chalk cliffs on the SE coast of England. The first shot is with the default mesh. The cliffs look like little more than a sloping white beach. The second shot - with the 1 metre mesh - shows them as they really are.

  • Creating terrain heightmaps

    USING ELEVATION DATA FOR GREAT BRITAIN - A TUTORIAL

    I'd like to start by thanking qwerty42, crispy136 and TomSimMuc for their tutorials on the USA, Australia and Norway respectively. I'd never have been able to come up with this stuff on my own without having previously studied their methods.


    WHAT YOU WILL NEED BEFORE YOU START

    • Aerofly SDK tools (inc Geoconvert)
    • QGIS version 2.14.21 (Essen) - available here. [I can't figure out how to make it work with the latest version.]
    • The mesh_conv.tmc & mesh_conv.bat files - as for any other region. (qwerty provided these in his tutorial package.)
    • Some means of knowing where the Aerofly tiles are in terms of latitude and longitude. I use my own spreadsheet (with formulae courtesy of rodeo) but I believe there is another tool available for download on this forum? AeroScenery is also helpful for this purpose.


    DOWNLOADING THE DATA

    Broadly speaking there are 2 different types of free data available for GB. I'll say a little about them and explain how to download them. The data is usually organised by Ordnance Survey 100 x 100 km grid squares, which have 2-letter name such as SU, TQ, NY etc and are further subdivided using numbers. There's an explanatory map here. Both types of data are in the same format (.asc files) so the PROCESSING THE DATA section which follows is the same for both.


    1. Ordnance Survey 50 m data "OS Terrain 50" This data is complete for the whole of England, Wales and Scotland and can be downloaded all in one go.

    • Go to: Ordnance Survey: OS Terrain 50.
    • Select ASCII grid and GML (grid) in the supply format menu.
    • Tick download, then scroll to the bottom of the page and click continue.
    • Fill in the form and you will be sent an email with a download link.
    • Download and unzip. The unzipped folder contains the .asc files grouped according to OS squares.
    • You may not be particularly interested in this data since higher resolutions are available, but it is actually a considerable improvement on the default mesh and can be used to cover large areas very quickly. [UPDATE: I'm getting occasional faults (deep trenches) which I think are caused by this data. This is surprising since it comes from the Ordnance Survey. I've corrected the faults by deleting the odd level 11 TTH tile here and there.]

    2. High res data from various sources (2 m, 1 m, 50 cm and 25 cm) This type of data tends to have a lot of "holes" in it and coverage may be far from complete. There are full details below on how to download the English data plus links to the Scottish and Welsh data.

    • For the English data go to: DEFRA Data Download
    • Zoom in on the map until you see the squares and click the down arrow at the top right so that you get the Download your data panel.
    • Select an area using the freehand drawing tool. You really want a square or squares but the only way of selecting them seems to be to draw a rough circle around them to begin with. The squares are 5x5 km, so a level 9 sized area is roughly 100 squares. However the site won't allow you to select that many in one go.
    • Click get available tiles. There are 3 drop down menus here with a variety of choices, so feel free to experiment. In the end I chose LIDAR composite DTM from the top menu and DTM 1M from the bottom menu. The DTM 2M data has slightly better coverage - but there's not a lot of difference. The DSM options are probably best avoided - they just seem to give random lumpiness in built up areas.
    • Click your square(s) on the list, download and unzip.
    • This data is excellent where it exists but you'll need to avoid geoconverting the "holes" otherwise you'll get flat areas at sea level in your landscape, which is not a good look.
    • Scotland: (1) Scottish Govt: Remote Sensing Data (GeoTIFF format - ie different but simple enough to process - coverage rather sparse) (2) More Scottish data (.asc format - sparse coverage)
    • Wales: Lle Geo-Portal for Wales (.asc format - similar coverage to the English data)

    Ordnance Survey 5 m data OS Terrain 5 looks promising since it is complete for the whole of Great Britain. However, when I made enquiries about obtaining a free "Data Exploration Licence" Ordnance Survey told me that the licence was only intended for commercial users who might eventually become paying customers.

    News is that ORBX are working on GB scenery. However the mesh creation I describe isn't really wasted effort because you will be able to combine the 1 metre etc mesh with ORBX's mesh (which will probably be either 5 or 10 metre.)


    PROCESSING THE DATA

    Whichever of the above data sources you use, you will end up with a large number of .asc files - so from here onwards the procedure is similar for all the data types. The first step in the procedure is "merging" them. You need to do this so that they form a big enough heightmap to be worth geoconverting. QGIS won't let you merge more than about 100 .asc files at a time, but you can always merge them in batches and then merge the resulting TIFs if you want a bigger area. If you find that the .asc files are in multiple folders you can use Windows search (for example) to find them all, copy them and put them all in one place. (Search for "asc".) This makes life easier when you do the merging.

    • Open up QGIS and select Raster > Miscellaneous > Merge.
    • Click the input files select button, navigate to the folder where you have your .asc files and select all the ones you want to merge. You also have the option here to choose input directory instead of files and to check recurse subdirectories. Like "search", this is also useful if the .asc files are in multiple folders.
    • Click the output file select button, give your file a name and choose somewhere to save it.
    • Click OK and wait. The Coordinate Reference System Selector box should appear after a short while. When it does appear, choose OSGB 1936 / British National Grid EPSG:27700 from the long list of CRS's in the 3rd panel and click OK in that box. [You can use the filter to search for it. After you have used the CRS once it will subsequently always appear in the 2nd panel from the top.] The merging should then go ahead to completion. You should see a heightmap in the main QGIS canvas when you've closed all the boxes obscuring it (OK, OK and close). [If you are using 1m data etc this is the point where you'll be able to spot the "holes". They show up as completely black areas in the heightmap.]
    • Right click on the bold, underlined title of your merged file in the bottom left panel of QGIS, and select save as. In the save as box you should have: output mode = raw data and format = GTiff - but you need to change the CRS now to Default CRS (EPSG:4326 - WGS 84). You should now see latitude and longitude values in the extent panel below, and resolution values in the panel below that. Use Browse to enter a name for your merged file and choose a place to save it. Click OK to save.
    • 1 metre etc data only: The 1 m data has lots of black (no data) areas in it, so you need to crop rectangles of good data from a TIF of level 9 size or larger, avoiding all the "no data" areas. (1) In QGIS: select layer > add layer > add raster layer. (2) Select the merged TIF that you saved as ESPG:4326. (3) Select raster > extraction > clipper. Select a destination for your output file and give it a name. Clipping mode = extent. Select a good rectangle from your Geotiff. Click OK and wait for the processing to finish. (4) Repeat 3 until you have selected all the good bits of your geotiff. You may end up with dozens of cropped bits of geotiff. (5) Select raster > conversion > translate, Check batch mode. The input directory is where you have your multiple cropped geotiffs. Output directory = input_aerial_images. Check creation options, click "+" and type "tfw" under name and "yes" under value. (6) Click OK and you should end up with all your cropped TIFs and corresponding TFW files in the input_aerial_images folder ready for geoconversion.
    • 50 metre data only: Follow steps (5) and (6) above, except that you only have one TIF (probably) and only want to produce one TFW file.
    • Edit your mesh_conv.tmc in the usual way. I assume people have some knowledge of this, but if you don't I can point you in the right direction. If you are doing a whole level 9 or 10 tile then you can use your preferred method of snapping to that tile area. I actually use Aeroscenery to get the level 9 tile co-ordinates for the .tmc file. I tick generate AID/TMC files and then copy and past the 2 lines of code that contain the coordinates from the Aeroscenery TMC into the mesh_conv.tmc. I also go to settings > geoconvert in Aeroscenery and change shrink TMC grid square by from 0.01 to 0.005 degrees. (I find that with 0.01 I sometimes have missing tiles.) The default mesh already has level 7-10 elevation (TTH) tiles. My current thinking is to use the 50 m data to create level 11 tiles for the whole terrain, and the 1 m data (where it exists) to create levels 12, 13 & 14. (Obviously many of these latter files will be masked because they come from lots of cropped geotiffs.) Anyway, the principle is to use separate levels for different resolutions.
    • Finally double click your mesh_conv.bat to run Geoconvert, and your TTH files should appear in the scenery\images folder as usual.
    • You can put the TTH files here: C:\Users\yourname\Documents\Aerofly FS 2\scenery\elevation - or you may have some alternative place for them. (I personally have them on the D-drive now.)

  • Creating terrain heightmaps

    Quote from qwerty42

    It stores certain preferences and configuration settings, and if it's gone I think AeroFly generates a new one when it starts. I can't remember if joystick mappings are stored in there, so if you try it you might want to back up your old one. Also you'll have to manually re-enter your auxiliary folder location back into it if you regenerate it. I doubt it will make a difference to what you're seeing, but it's so fast to check it's worth a try.

    OK thanks - I might give it a go.

  • Creating terrain heightmaps

    Quote from qwerty42

    Weeeeird... I have no clue why it would behave that way. Have you tried having steam do an integrity check on your base AeroFly install? I wonder if something is corrupted with one of the default tiles... :/

    Edit: Just had a thought... do you have any extra 3rd party DLC? Like ORBX stuff? I wonder if the load priorities on those are dealt with in a different way and could be conflicting?

    I'll try the integrity check thing - thanks for the tip.

    I do have ORBX True Earth Netherlands - so I guess that could be causing the problem. Also I moved the Netherlands scenery tiles - I don't know whether that's of any significance. I never touched the Netherlands TTH files however.

    What does deleting the "main.mcf" do? I have it at the back of my mind that it's a cure for some types of problem - but I can't remember which!

    Cheers

    Ian

  • Creating terrain heightmaps

    I've beeen experimenting with different locations for my elevation (TTH) tiles, and I've found that if I put my level 7-10 tiles here .....

    D:\SteamLibrary\steamapps\common\Aerofly FS 2 Flight Simulator\scenery\elevation

    or here .....

    D:\aerofly_aux_folder\scenery\elevation

    ..... Aerofly takes ages to load and eventually crashes. (Actually, to be more precise, I can put a few level 7-10 tlies in those locations but when I add more I get a crash.)

    If I put the level 7-10 tiles here .....

    C:\Users\ian\Documents\Aerofly FS 2\scenery\elevation

    everything seems OK (almost), The problem I reported in a previous post seems to be just for one one particular set of level 7-10 files that don't show up in Aerofly when I place them on the C-drive. Those particular tiles only show up on the D-drive. Another set do show up on the C-drive. I can't explain that.

    As for the level 11-14 tiles .... they don't cause a crash in any location and do always show up.

    In summary: I've partially solved my storage problem because I can keep my level 11-14 tiles on my D-drive, but it seems I have to have my level 7-10 tiles on my C-drive.

  • Creating terrain heightmaps

    Thanks for your reply qwerty42. I was aware that I could specify an additional folder for scenery - I wasn't sure that I could do the same for mesh tiles. However since I would probably put them on the D-drive anyway I was thinking that I might as well put them with the default ones that come with the Aerofly installation.

    Like you I have levels 7-14 in the USA and, as you say, there's no easy way of telling whether the lower ones are showing up in that case. However in Norway (> 60 deg) it's very obvious because the default terrain is flat and at sea level.

  • Creating terrain heightmaps

    Currently I am putting my elevation (TTH) tiles here .....

    C:\Users\ian\Documents\Aerofly FS 2\scenery\elevation

    Is there any reason why I shouldn't put them here .....

    D:\SteamLibrary\steamapps\common\Aerofly FS 2 Flight Simulator\scenery\elevation ?

    Obviously I would put them in a separate folder so that I could move them if necessary. The reason I ask is that my C-drive is rather small in capacity. The D-drive would be a much better place for them.

    Another question ..... I created a base layer of level 7-9 TTH tiles for a level 7 area north of 60 deg in Norway using 10 m data from Høydedata. Initially I put them in the C-drive location above and found that they didn't show at all in Aerofly. The terrain was completely flat - i.e. it was the default terrain at that latitude. It was only when I transferred them to the D-drive location that I saw the terrain as it should be. I noticed that the default terrain tiles on the D-drive are levels 7-10 only. I assume therefore that levels 7-10 have to be on the D-drive anyway to show up. Obviously it would make sense for me to put all levels (7-14) on the D-drive.

  • Creating terrain heightmaps

    Quote from Trespassers

    And a good 10m mesh already may provide very satisfaying results.

    Yes of course - I was forgetting - 10 m is actually a very good mesh. It produces excellent results in the USA.

    And yes as far as I'm aware there is no Aerofly mesh north of 60 deg. I certainly can't see any mountains to the north in the distance. I can see them to south because Bergen is only a little north of 60 deg.

    Cheers

    Ian

  • Creating terrain heightmaps

    Thanks Antoine - that's a very useful table.

    It looks as though I may be able to get 10 m and 50 m resolution data from the Norwegian site - but I haven't tried yet. The 10 m data would certainly be good enough for both levels 7 & 9. The 50 m data may not be quite good enough for level 9, though good enough for level 7. Having said that, the default Aerofly resolution is worse than 50 m (not sure?) so maybe it wouldn't be any worse than default?

    I have noticed that there is a fearsomely deep crevasse in the sea just off the west coast of Norway where my current data ends. It is about 100 m wide at the top and appears to extend hundreds of metres below sea level. It is genuinely scary in VR, especially if you lower the helicopter into it! I'll have to have a play around and see what I can do about it.

    Cheers

    Ian

  • Creating terrain heightmaps

    Quote from Trespassers

    Congrats Ian, it looks good.

    In such places where there is no (or poor) underlying default scenery you’ll notice even Level 9 only loads at quite short range in front of your aircraft in AFS2, thus a lower level like 7 is necessary for distant scenery.

    Level 7 tiles are huge, of course you let geoconverter create a mask for missing areas, but keep in mind that areas you don’t cover beyond N60 deg will remain flat...

    Cheers

    Antoine

    Hi Antoine - and thanks

    Yes - I have occasionally noticed that hills and mountains are suddenly appearing out of nowhere in the middle distance.

    I've found that it takes about 10 GB of 1metre data to cover a level 9 tile area. This is a manageable amount of data, so I can snap to level 9 and avoid masked tiles in levels 9-14. However I would need about 16x that amount to cover a level 7 area. There appears to be lower res data available on the Høydedata data site so I was thinking that I could maybe cover the country in level 7 tiles using lower res data. (I haven't tried it yet.) There's still going to be a problem if I go anywhere near Sweden though!

    Cheers

    Ian

  • Creating terrain heightmaps

    Hi Chris

    Well done re Cape Flattery!

    Since this thread is about terrain heightmaps I will send a direct message to you about ScenProc.

    Cheers

    Ian

  • Creating terrain heightmaps

    Hello Thomas

    I'd forgotten about the lack of terrain mesh at 60 deg plus. That would explain why I can see distant mountains to the south of the airport, but not to the north! I'll fill in the lower level TTH files from 7 upwards as you suggest.

    Regarding the tutorial, there were just 2 things that puzzled me:

    (1) The merging of the separate TIFs that one receives from Høydedata. I did "raster > miscellaneous > merge" which is pretty much what you said. I think it was the fact that you talked about "loading" them that confused me, because I thought that was a step that you had to do before you merge them.

    (2) The fact that there are 2 different resolutions - horizontal and vertical - and it is the vertical one which is negative in the TFW file.

    I've attached a screenshot taken a little to the west of Bergen airport. There's no point in posting a comparison shot for the default terrain because, as you say, there is none. The scenery looks a bit like the "limestone pavement" scenery in northern England - but I think the underlying rock is much harder.

    Cheers

    Ian

  • Creating terrain heightmaps

    OK - I've created some scenery to go with my high res terrain mesh in Norway - and the results are very mixed. I've certainly got high res areas - probably 1 metre resolution - judging from the piles of gravel modelled, the flatness of small lakes, the knobbliness of hills, etc. Unfortunately, however, there are also numerous faults in the mesh pretty randomly distributed - in the form of vertical cliffs, trenches and walls where there shouldn't be any. These are in the middle of the high res area - not at the boundary where you would expect to see these faults.

    I've tried removing all the masked files and that doesn't make any difference. (I wouldn't expect it to anyway, except at the outer boundary.) The only thing I can think of is that I should have merged the TIFs before geoconverting as TomSimMuc suggested. I wonder what I was doing wrong when I tried to merge them the first time? TomSimMuc : You said that you "loaded the relevant TIFF files into QGIS". I'm not quite sure what that means. I selected them all from the relevant directory when I merged them - but I didn't "load" them prior to that. Maybe I should have saved them in the correct format before merging them (using layer > add layer > add raster layer)? I will have to experiment. The other thing that puzzles me about merging them is that I will end up putting one pair of figures for resolution in the TFW file when the 4 unmerged TIFs have 4 slightly different resolutions. How critical are the resolution values? What part do they play in the geoconversion?

    UPDATE: SUCCESS! I had another go at merging the TIFs before saving and geoconverting and it worked this time. I swear I did the same as last time - but I guess I must have done something different. I ended up with a perfectly square composite image, which is reassuring. All the faults in the mesh seem to have disappeared, except at one point I did notice a few very deep potholes in a short line across a stretch of hillside. I guess these would be examples of the spikes (albeit inverted) that you mentioned Antoine? I'm not seeing any floating cultivation. It's difficult to judge at what distance the high res detail kicks in. I'm not noticing that it suddenly "snaps" into view - though I have noticed that some more distant hills have suddenly snapped in. I'm not sure whether that's related to the high res mesh or not. All in all I'm very pleased now. The scenery of this part of Norway really benefits from the higher res because the hills are very rocky and have lots of fine detail - as does the rocky shoreline. I'll try to take some screenshots when I get the time.

    Cheers

    Ian

  • Creating terrain heightmaps

    Thanks TomSimMuc and Trespassers - luckily I guessed right re the vertical resolution! I'm not too worried about the masked TTH tiles at the moment because I only intend to fly within my high res region, and I'll just extend it if I want to go further afield. The problem with my initial attempt was that all the TTH tiles (levels 12, 13 & 14) were masked - which can't be right for a 20 x 20 km area. On my second more successful attempt I got a mix of masked and unmasked files, which I was expecting.

    I'm still somewhat disconcerted by the fact that the 4 TIFs I've received overlap and don't form a perfect square, although I downloaded a perfect square. Either I must have received more data than I requested or less - preferably the former! If there are gaps around the edges of the square I requested it could be a problem if a want to download an adjacent square later. (I kept a record the co-ordinates in metres of the square, so I know what I've already got. It doesn't seem to be possible to download a square by specifying lon-lat co-ordinates.)

    Anyway I'll report back when I have the scenery as well

    Cheers

    Ian

  • Creating terrain heightmaps

    Quote from TomSimMuc

    Hello Hartman,

    Did you see my article about Norwegian elevation meshes?

    Creating terrain heightmaps

    Cheers,

    Thomas

    Hi again Thomas

    I've given Norway a go and obtained a healthy-looking set of TTH files for a 20x20 km area covering Bergen Airport. Many thanks for your guidance!

    I haven't got any scenery yet to test them with, but while I'm waiting for that I have a few questions/observations .....

    (1) I got 4 TIFs covering that area - and I noticed that their co-ordinates overlap. You talk about merging them, but I didn't have any success doing that using "raster > miscellaneous > merge". The merging appeared to work and the geoconversion went ahead but the TTH files were all masked and about 2 kB each - so obviously no good. I must have been doing something wrong there.

    (2) So instead I just did "layer > add layer > add raster layer" for each separate TIF, saving each one in the format you suggested and putting them all in the "input_aerial_images" folder along with the 4 corresponding TFW files.

    (3) At the "save" stage in QGIS I noticed that the horizontal and vertical resolutions were different - and also different for each TIF. In the TFW file one of the resolutions is positive and one is negative. Which is supposed to be negative - the horizontal or the vertical? (I guessed and made the vertical negative.)

    Given the guesswork above and the fact that the TIFs were overlapping I wonder whether my TTH files are really OK. We shall see!

    Cheers

    Ian

  • Creating terrain heightmaps

    Quote from TomSimMuc

    Hello Ian,

    the 1m resolution is so high, that you even can see the noise protection walls here:

    Yes indeed. The distinction between a "man-made structure" and a natural terrain feature is a bit arbitrary anyway I guess. I suppose if a structure is sufficiently "solid" it will be treated as part of the ground.

    This certainly attracts me towards giving Norway a go - expecially since I've noticed that there are now some pretty good aerial images for southern Norway.

  • Creating terrain heightmaps

    Quote from TomSimMuc

    Hello Hartman,

    Did you see my article about Norwegian elevation meshes?

    Creating terrain heightmaps

    Cheers,

    Thomas

    I'm very interested in this 1 m Norway data.

    Is this kind of data always based on a survey of the ground elevation? I have 2 reasons for asking: (1) I would have thought it would be quite difficult to "see" the ground if the area is built up. (2) If the data were high enough res (ideally better than 1m - but 1 m might suffice) it could give a 3D effect to man-made structures - which could perhaps be useful in the context of flight sims. Obviously, since the aerial images contain no info about vertical surfaces, it wouldn't be much use for buildings - but it might model structures such as Hadrian's Wall or maybe even forests.

    Ian

  • Creating terrain heightmaps

    Good point - yes - I hadn't thought about noise and spikes in the data. So far I haven't noticed any in the USA but maybe it's not an issue with approx 10 m resolution (1/3 arcsec) and level 12 compilation. (Actually level 11 would probably suffice for 10 m res in theory.) The resolution is high enough to create pleasing mountain features but not so high as to create the problems you mention.

    Cheers

    Ian