Northland/Nordland or 8th Wonder of the World/Weltwunder map.dat file format

gistfile1.txt

<!-- mediawiki -->

<!-- Note: the decoding is not completely right because on bigger maps my parser crashes. For smaller maps it works though. -->

The map.dat has no index table at the beginning of the file. It rather uses entry-headers (called "hoix"es) that state what type of content the next block contains and how long it is. The byte-order is  Little Endian.

== Hoix == 
A hoix denotes a header of a datablock in the map.dat. The file starts with this.

{| class="wikitable"
! Datatype
! Name
! Notes
|-
| char[8]
| name
| It is the name of the entry. Additionally, it determines the type of data it contains.
|-
| uint32
| unknown
| 
|-
| uint32
| blocklength
| The size of the data block in bytes that starts directly after this hoix block. A either a new hoix comes directly after or the end of file is reached.
|-
| uint32
| unknown
| 0 every time?
|-
| ubyte[3]
| unknown
| 0x3FA017 every time?
|-
| ubyte
| unknown
| 
|-
| ubyte[6]
| unknown
| 
|-
| ubyte[2]
| unknown
| Sometimes the last hoix is two bytes too short so end of file might be already reached here!
|}


The "name" field can be one of the following:
{| class="wikitable"
! Datatype
! Notes
|-
| <tt>[[#hoixigol|hoixigol]]</tt>
| 
|-
| <tt>[[#hoixmmgl|hoixmmgl]]</tt>
| 
|-
| <tt>[[#hoixzisl|hoixzisl]]</tt>
| Mapsize
|-
| <tt>[[#hoixehml|hoixehml]]</tt>
| Heights
|-
| <tt>[[#hoixapml|hoixapml]]</tt>
| 
|-
| <tt>[[#hoixbpml|hoixbpml]]</tt>
| 
|-
| <tt>[[#hoixtlml|hoixtlml]]</tt>
| Minimapcolor - normally linked with LSC (e.g. Trees are green on minimap)
|-
| <tt>[[#hoixvlml|hoixvlml]]</tt>
| Additional data for LSC - number of items on the ground; content of chests; ...
|-
| <tt>[[#hoixplml|hoixplml]]</tt>
| 
|-
| <tt>[[#hoixocml|hoixocml]]</tt>
| 
|-
| <tt>[[#hoixwtml|hoixwtml]]</tt>
| 
|-
| <tt>[[#hoixsmml|hoixsmml]]</tt>
| 
|-
| <tt>[[#hoixrpml|hoixrpml]]</tt>
| 
|-
| <tt>[[#hoixbwml|hoixbwml]]</tt>
| LSC; unknown; On trees: if harvestable=1, else=0
|-
| <tt>[[#hoixbbml|hoixbbml]]</tt>
| LSC
|-
| <tt>[[#hoixorml|hoixorml]]</tt>
| 
|-
| <tt>[[#hoixbsml|hoixbsml]]</tt>
| 
|-
| <tt>[[#hoixoaml|hoixoaml]]</tt>
| 
|-
| <tt>[[#hoixfhml|hoixfhml]]</tt>
| 
|-
| <tt>[[#hoixocal|hoixocal]]</tt>
| 
|-
| <tt>[[#hoixwsal|hoixwsal]]</tt>
| 
|-
| <tt>[[#hoixmfal|hoixmfal]]</tt>
| 
|-
| <tt>[[#hoixdnex|hoixdnex]]</tt>
| 
|-
| <tt>[[#hoixmmme|hoixmmme]]</tt>
| 
|-
| <tt>[[#hoixrbme|hoixrbme]]</tt>
| Lights
|-
| <tt>[[#hoixcvme|hoixcvme]]</tt>
| Vertexcolors
|-
| <tt>[[#hoix1mme|hoix1mme]]</tt>
| 
|-
| <tt>[[#hoiximme|hoiximme]]</tt>
| 
|-
| <tt>[[#hoixdpae|hoixdpae]]</tt>
| Ground type dict
|-
| <tt>[[#hoixapme|hoixapme]]</tt>
| Ground texture triangles
|-
| <tt>[[#hoixbpme|hoixbpme]]</tt>
| Ground texture triangles
|-
| <tt>[[#hoixdtae|hoixdtae]]</tt>
| 
|-
| <tt>[[#hoix1tme|hoix1tme]]</tt>
| 
|-
| <tt>[[#hoix2tme|hoix2tme]]</tt>
| 
|-
| <tt>[[#hoix3tme|hoix3tme]]</tt>
| 
|-
| <tt>[[#hoix4tme|hoix4tme]]</tt>
| 
|-
| <tt>[[#hoixdlae|hoixdlae]]</tt>
| 
|-
| <tt>[[#hoixalme|hoixalme]]</tt>
| LSC - Types
|-
| <tt>[[#hoixdnet|hoixdnet]]</tt>
| EOF
|}
They are in order of natural occurrences.

=== hoixzisl ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| uint32
| width
| 
|-
| uint32
| height
| 
|}
These values are equivalent to the ones seen in the [[internal Editor]] and half of the values specified in the [[external Editor]].

=== hoixehml ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Every decoded "data" field inside the [[#Common Data|COMMON_DATA]] represents the height value from 0 to 255 of one field. common_decoded holds the height data as one big array for the whole map. From this array the game builds the map from left to right, from top to bottom.
|-
|}

=== hoixtlml ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Decoded like [[#hoixehml|hoixehml]].
|-
|}

=== hoixbwml ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Decoded like [[#hoixehml|hoixehml]].
|-
|}

=== hoixbbml ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Decoded like [[#hoixehml|hoixehml]].
|-
|}

=== hoixrbme ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Exactly like [[#hoixehml|hoixehml]] except that it contains the light information instead of height.
|-
|}

=== hoixalme ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Decoding|COMMON_DECODING]]</tt>
| common_decoded
| Decoded similar to [[#hoixehml|hoixehml]] EXCEPT that it uses two bytes instead of one for the data (the "number" is not affected; still one byte).
|-
|}

== Common ==
=== Common Decoding ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| <tt>[[#Common Header|COMMON_HEADER]]</tt>
| header
| 
|-
| <tt>[[#Common Data|COMMON_DATA]]</tt>
| data
| 
|}

=== Common Header ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| ubyte
| unknown
| 
|-
| uint32
| blockLength
| represents the number of bytes that are between the end of this uint32 until the next hoix
|-
| ubyte[12]
| unknown
| 
|-
| uint32
| someLength
| ? same as blocklength before
|}

=== Common Data ===
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| ubyte
| number
| Either the number of fields for which the next byte is the data OR the number of bytes following where every byte is the data for one field.
|}

If number is > 128, the number stands for the number of fields the data is valid for,
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| ubyte
| data
| The data that is encoded here is valid for the following "number" fields. (-> run-length encoding) The data is specific to the hoix.
|}

else, "number" is the number of bytes that are following where every byte is the data for one field.
{| class="wikitable"
! Datatype
! Name
! Notes
|-
| ubyte
| data[number]
| Every data-byte is the data for exactly one field. The data is specific to the hoix. 
|}

YES YES YES! FINALLY I AM ABLE TO DECODE BMD FILES! Thanks to your C code, @martianboy! The documentation I have from Molt was wrong at two points. I ignored the docs more or less this time and just straight up ported your C to java. Finally!

@Nithanim oh wow that's awesome, I didn't know you still wanted to experiment with those! 😃 My latest code for reading BMD files are in my Rust Wasm package, here's a link: https://github.com/martianboy/cultures2-wasm/blob/master/src/bmd.rs

I'm new to Rust, though, and the code quality is still mediocre. But I figured a bunch of other details like frame type 4, and the metadata fields on frame info headers.

I'm already able to render the whole ground of the map, and in just a few days I'll be able to render landscape items (trees, meadow, goods, etc) over it as well. There are still a few bugs here and there. I can prepare a demo for you if you wanna see the progress!

By the way @Nithanim, if you like, I'll be more than glad to chat and exchange ideas regarding this over Telegram or something.

I never got BMDs working and left them because it was just a mess, until you suddenly showed up by surprise and found your reader. I will have a look at your rust code. I do like rust but I have never gotten into it but looks like a good opportunity for reading some!

You are loooong ways ahead of me in terms of reversing the game and I am pretty sure that you are the one with the most knowledge in that regard. Though, there is still the trickery of Remik's Editor where I don't know what he did.

After you showed up I wanted to get back to the c2m file but quickly ran into the issue that there is no good viewer/(un)packer available. So I am currently developing a multitool like one that exist for other games (CascView, Gw2Browser, FinalBig, ...) and then ran into the dreaded bmd format again.

Feel free to add me on Telegram! You should be able to find me with the same name/handle.