Jump to content

DOWNLOAD MODS

Are you looking for something shiny for your load order? We have many exclusive mods and resources you won't find anywhere else. Start your search now...

LEARN MODDING

Ready to try your hand at making your own mod creations? Visit the Enclave, the original ES/FO modding school, and learn the tricks of the trade from veteran modders...

JOIN THE ALLIANCE

Membership is free and registering unlocks image galleries, project hosting, live chat, unlimited downloads, & more...

TES Interior Data / Random Dungeon Generator


Narks
 Share

Recommended Posts

I've always been a fan of roguelike games - due to their random dungeon generation. Obviously, a dungeon in TES4 is a whole world more interesting then a roguelike game like Nethack.

So, I want to bring the two worlds together. A static random dungeon generator, mostly for modders, to allow for the rapid creation of large, interesting dungeons. I found that a lot of the higher quality dungeons created by users tend to have rather unique designs (such as the Lost dungeons in The Ayleid Steps), but still have basic flaws such as sections not lining up properly, overlapping sections, etc.

The purpose of the dungeon generator, would be a source of inspiration - rather then design the dungeon based on an idea, base your idea on the random dungeon layout. It also saves time, and hopefully if this reaches an advanced stage, will be capable of creating far more interesting rooms then often seen in the vanilla dungeons.

The basic concept will be that the generator will pick from a library of prebuilt rooms stored in its memory. It will then create a random dungeon layout based on user parameters, and then create the dungeon using an algorithm similar to what you may find in a roguelike game (I already have one in mind). Different libraries are for different dungeon architecture (obviously, like Ayleid [which will be the focus of the generator], Imperial keeps, or whatever) but can also be configured to focus on rooms that fit a certain theme (ie. untouched Ayleid tomb rooms only, focus more on traps, etc).

Chances are the only way to do this, will be to figure out how TES4 stores interior data, and write a 3rd party program to replicate it. I have no experience in the TES modding community - hey, someone may already know how, or maybe it's stored as plaintext (I haven't looked yet). Any information about how interior data works would be nice - but I believe that I will have to be the one who analyzes the interior data.

According to somebody in the chatbox, someone has already created a random dungeon generator (albeit apparently simple), but he did not know who or what it was called. If anyone has any information, I would be interested in knowing what you know.

tl;dr:

- does anyone know how TES4 interior data works?

- does anyone know if a random dungeon generator already exist?

- good idea? y/n

yeah, I know it's ambitious, but you gotta start somewhere

Edited by Narks
Link to comment
Share on other sites

I think there was a utility to do this at one point. I seem to remember a thread in the oblivion mods section of the beth forums. However, I am unable to come up with it now. It has been a while, so, it might have been the victim of a forum prune.

Link to comment
Share on other sites

the interior data is fully exposed/decoded by Wrye Bash currently (also Tes4Edit). However I would sugget waiting to do anything base on that until Warrudar's CBash is done (A C implementation (ie way faster than python for read and write of large files like oblivion.esm) that can plug into python)... then build a python or C based app using that to deal with the esp writing and data reading/writing.

Pacific Morrowind

Link to comment
Share on other sites

IIRC, there was a POC mod that was released sometime after Oblivion's release. It used the cellar/basement asset set to create a pseudo-random dungeon. This was done using scripts, and involved manually linking different static tiles together in a feasible order. In any case, the result was a generic looking maze (which was the goal). The idea was dropped as linking the different tilesets didn't seem like it would yield anything usable. In essence, the number of combinations that algorithm yielded was pretty low and every other seed looked similar to its older sibling.

Link to comment
Share on other sites

the interior data is fully exposed/decoded by Wrye Bash currently (also Tes4Edit). However I would sugget waiting to do anything base on that until Warrudar's CBash is done (A C implementation (ie way faster than python for read and write of large files like oblivion.esm) that can plug into python)... then build a python or C based app using that to deal with the esp writing and data reading/writing.

Pacific Morrowind

Wow, thanks, that's really helpful. I'll go investigate Wrye Bash. Chances are I will write the actual dungeon generator in Java (because that's the only real language I have any experience in), and then learn just enough Python to handle read / write operations. Where can I find information or updates on CBash?

IIRC, there was a POC mod that was released sometime after Oblivion's release. It used the cellar/basement asset set to create a pseudo-random dungeon. This was done using scripts, and involved manually linking different static tiles together in a feasible order. In any case, the result was a generic looking maze (which was the goal). The idea was dropped as linking the different tilesets didn't seem like it would yield anything usable. In essence, the number of combinations that algorithm yielded was pretty low and every other seed looked similar to its older sibling.

Well, I don't intend to put together static tiles to create a dungeon, I intend to put entire rooms (like the warehouse interior rooms), so it should not look too maze-like. I'm more worried about the dungeon becoming too linear (the algorithm I'm thinking of implementing will probably tend to create a "tree" like dungeon.)

UPDATE: Well, looking through Wrye Bash, I believe that it does not have the functionality I desire (editing references in cells) - and even if it does, chances are I won't be able to find said functionality without a lot of tedious searching through the source (or just asking Wrye, but I don't know how to contact him, and even if he would help).

So, I opened up an .esp file I made with 3 static objects placed at specific coordinates using a hex editor. I learned that TES4 likes to store coordinates as 32-bit floating point numbers... backwards. From there, I've managed to isolate what I believe is a single reference.

So far, all I've managed to do is change the coordinates of objects. I tried to duplicate a reference, but it just crashed TES4CS when I tried to open the esp (although I believe I might have screwed up the duplication itself lol). I'm going to check now if the .esp has some kind of checksum. I also have a suspicion that the cell object has a checksum or a count of how many references a cell should contain.

As usual, any information would be welcome! I'll eventually post all my findings when I feel I have enough to be worthwhile.

UPDATE:

YESYESYESYESYES

I managed to successfully duplicate a reference in a cell! TES4CS also threw me a whole bunch of errors, which while bad, also informed me a lot about how the .esp file works. Furthermore, pressing yes to all the errors didn't cause a crash, and when I opened up the interior, I saw my extra reference!

brb victory dance

Here are the errors; if anyone can recognize them, that would be helpful.

I tried to duplicate a reference, and I got an error while loading:

Duplicate form ID (01 00 0C E9) encountered.

Then I got this error:

SetFormID bashing entry in form ID map at 01 00 0C E9.

Form basher: type REFR_ID

Bashed form: REFR Form " (01 00 0C E9) to STAT form 'ARFloor03' (00 03 45 B3)

in Cell 'aaaMyDungeon' (01 00 0C E6).

Then this error:

Form counts don't match.

File = thegame.esp

Forms loaded = 10

Forms in file = 9

Do you want to correct the file header?

The .esp loaded fine, and I saw my duplicate reference. Awesome.

The second time I loaded, I got only the first two errors, not the third one.

UPDATE:

I've done a lot more research on the inner workings of the .esp. I'm nearing a point where I am confident that I know enough to create a dungeon filled with static objects (I have not investigated things such as traps, NPC's and doors - still a lot of work to do!).

This is the file with all my findings (I had to use <code> tags).

ESP files seem to store floats as 32-bit, in hex, backwards.

Use this to convert: http://babbage.cs.qc.edu/IEEE-754/Decimal.html

So, if your number in decimal is 3000, and the hex conversion is 45 3b 80 00, then it's stored as 00 80 3b 45.

44A72000

44 A7 20 00


Coordinates of the reference are stored as XYZ, one after another, as backward 32-bit floats.



After this is twelve 00's (00 00 00 00 00 00 00 00 00 00 00 00). This seems to represent the end of the reference.

These values may have some significance (probably boolean flags).


Now that we know the end of the reference, we can isolate a single reference! Yay!


// ======================================================================================

// Reference Analysis

// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E7 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 20 A7 44 00 20 A7 44 00 20 A7 44 00 00 00 00 00 00 00 00 00 00 00 00

52 45 46 52 28 00 00 00 00 00 00 00 E8 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A C4 00 00 FA 44 00 80 3B C5 00 00 00 00 00 00 00 00 00 00 00 00

52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

I---------I

Reference header - in text shows as "REFR".


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                    I---------I

This appears to be the form ID of the reference itself. Backwards, naturally.                


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                I---------I

Not sure what this is, but I saw the same sequence in the CELL and GRUP objects.

There are 10 in total - the same number of form IDs in the file.


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                            I---------I

Name header. Shows as "NAME" in text.


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                                              I---------I

I think this is the form ID of the object our reference is pointing to. Backwards.


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                                                          I---------I

This says "DATA" in text.


// ======================================================================================


52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                                                                            I--X-POS--I I--Y-POS--I I--Z-POS--I

00 00 7A 44 00 00 FA 44 00 80 3B 45

I--X-POS--I I--Y-POS--I I--Z-POS--I


This is where the reference coordinates are stored, in backward 32-bit single float, three of them.


// ======================================================================================

52 45 46 52 28 00 00 00 00 00 00 00 E9 0C 00 01 07 57 00 00 4E 41 4D 45 04 00 B3 45 03 00 44 41 54 41 18 00 00 00 7A 44 00 00 FA 44 00 80 3B 45 00 00 00 00 00 00 00 00 00 00 00 00

                                                                                                                                                I--X-ROT--I I--Y-ROT--I I--Z-ROT--I

00 00 00 00 00 00 00 00 00 00 00 00

I--X-ROT--I I--Y-ROT--I I--Z-ROT--I


Sample data:

DB 0F 49 3F E4 CB 16 40 2D D1 59 40

I--X-ROT--I I--Y-ROT--I I--Z-ROT--I


This one seems to be rotational data for the object.

Stored as radians, not as degrees. Strange.

DB 0F 49 3F = 0.7853981852531433 radians = ~45 degrees


// ======================================================================================

// Scaled Reference Objects

// ======================================================================================

Looks like modifying the scale of an object creates an "XSCL" object.

This appears after the end of the "NAME" object.


58 53 43 4C 04 00 00 00 00 40

I---------I

This just says "XSCL" - the header.


I am currently unsure how the actual scale is stored (the scale of this object is 2.00).


// ======================================================================================

// File Header Analysis

// ======================================================================================

Seems to be inside the "HEDR" object.


48 45 44 52 0C 00 00 00 80 3F 09 00 00 00 F7 0C 00 00

I---------I

Says HEDR.


// ======================================================================================


48 45 44 52 0C 00 00 00 80 3F 09 00 00 00 F7 0C 00 00

                              II ?? ?? ??

This looks like the form count. Maybe it uses 4 bytes? Probably does, but need confirmation.



// ======================================================================================

// 

// ======================================================================================



I tried to duplicate a reference, and I got an error while loading:


Duplicate form ID (01 00 0C E9) encountered.


Then I got this error:


SetFormID bashing entry in form ID map at 01 00 0C E9.

Form basher: type REFR_ID

Bashed form: REFR Form (01 00 0C E9) to STAT form 'ARFloor03' (00 03 45 B3)

in Cell 'aaaMyDungeon' (01 00 0C E6).


Then this error:


Form counts don't match.

File = thegame.esp

Forms loaded = 10

Forms in file = 9

Do you want to correct the file header?


The .esp loaded fine, and I saw my duplicate reference. Awesome.

The second time I loaded, I got only the first two errors, not the third one.

Edited by Narks
Link to comment
Share on other sites

Wow, thanks, that's really helpful. I'll go investigate Wrye Bash. Chances are I will write the actual dungeon generator in Java (because that's the only real language I have any experience in), and then learn just enough Python to handle read / write operations. Where can I find information or updates on CBash?

CBash topic over on bethsoft

UPDATE: Well, looking through Wrye Bash, I believe that it does not have the functionality I desire (editing references in cells) - and even if it does, chances are I won't be able to find said functionality without a lot of tedious searching through the source (or just asking Wrye, but I don't know how to contact him, and even if he would help).

It definitely can read/write but if you're doing Java might not help you too much; however the info contained in these classes should help you undertand the record data better: (ie jut search for these lines since depending on your version it could be different line numbers, the stuff after the colon in brackets is a little note about it)

class MreRefr(MelRecord): (the reference data for most cell references - ie most of what you want is in here)

class MreAcre(MelRecord): (the data for hand-placed creatures)

class MreAchr(MelRecord): (the data for hand-placed NPCs)

class MreRecord(object): (those three other classes inherit a lot of stuff from this... has a lot of the reading/writing code... the other reading/writing code is in the ModReader and ModWriter classes)

all the classes between 'class modReader:' and 'class MelTuple(MelBase):'

(all those classes are in bosh.py

Wrye is "kidnapped and taken to Azeroth" (his words... after this event happened) - he's pretty much totally offline... but otherwise he probably would have been helpful (he's helpful if you're polite an are at least a little bit not an idiot (he has a very low idiot tolerance (which runs into fun over at bethsoft sometimes :rofl:)). However Raziel23x, and me have taken over at least for the time being Bash's development (with contributions from haama, Lojack and Badhair as well)... any questions I'll try to answer them (though I've only been doing anything other than tescript for the last 10 months).

tried to duplicate a reference, and I got an error while loading:

Duplicate form ID (01 00 0C E9) encountered.

the formid has to be different for each and every record or you will (probably) run into all sorts of fun errors... so when duplicating I'd suggest (psuedocode):


def duplicate:

# lots of other stuff *****

tempid = newreference.formid

while tempid in listofformidsinmod:

    tempid += 1

newreference.formid = tempid

Then I got this error: SetFormID bashing entry in form ID map at 01 00 0C E9. Form basher: type REFR_ID Bashed form: REFR Form (01 00 0C E9) to STAT form 'ARFloor03' (00 03 45 B3) in Cell 'aaaMyDungeon' (01 00 0C E6).
not sure but I suspect that fixing the formid will fix this
Then this error: Form counts don't match. File = thegame.esp Forms loaded = 10 Forms in file = 9 Do you want to correct the file header? The .esp loaded fine, and I saw my duplicate reference. Awesome. The second time I loaded, I got only the first two errors, not the third one.
yes the cs autocorrects that error... but better to psuedocode:

def duplicate:

# lots of other stuff *****

tempid = newreference.formid

while tempid in listofformidsinmod:

    tempid += 1

newreference.formid = tempid

modfile.header.numrecords += 1

(see class MreTes4(MelRecord): in bosh.py for some details on that)

Hope that is helpful.

Pacific Morrowind

Link to comment
Share on other sites

So the MreRecord is the superclass of the MreRefr, MreAcre, MreAchr classes. I'm unfamiliar with python syntax, so it's a bit difficult for me to read through it - however, I was given this link: http://www.uesp.net/wiki/Tes4Mod:Mod_File_Format

It appears to be an in-depth map of all the data types and classes in the .esp (I wish I had found it earlier) - so I probably won't have to go through bosh.py. If I run into trouble, I may look again through wrye bash files, but I don't think I will need to.

Thanks for offering to answer questions - it gives me a lot of confidence that I can go off and do stuff and ask someone if I get stuck.

Wrye sounds like a pretty cool guy, but it does sound like he probably would not want to help me (I'm bound to say something idiotic or have a failure to communicate eventually, my knowledge of programming has gaping holes everywhere).

I also found out myself that the form id needed to be incremented, to obviously prevent duplicates. There also appeared to be a count of all the form id's in the HEDR object.

I'm probably going to go off and study the different data types and classes - more specifically, doors, traps and containers (and maybe those nice welkyn(sp?) stones on the stands). I am probably not going to implement random NPC generation (I can see it getting very messy) - so this generator will be for the dungeon structure only.

Thanks very much for replying!

Link to comment
Share on other sites

o, I opened up an .esp file I made with 3 static objects placed at specific coordinates using a hex editor. I learned that TES4 likes to store coordinates as 32-bit floating point numbers... backwards. From there, I've managed to isolate what I believe is a single reference.

In case you were still wondering, the floating point values are represented using the little endian byte order, where the least significant byte is stored first.
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...