How to Build Large Worldspaces: tips 'n' tricks

[Zehryo]

Ok, well, this is rather a list of hints and tips, than a real guide.

I'm going to give you a full range report on the difficulties I've encountered with building worldspaces larger than 4x4 quads and the relative solutions I've found on the Internet or figured out myself.

Point 1: What you might find useful to build a large worldspace.

If you've already fiddled a bit with CS, you've already noticed the worldheightfield editor limits the view to a total of 2x2 quads, which is ridiculous, if you plan to build a very large worldspace on a large scale (areas larger than 2x2). Not to mention how unnatural your set would look, if you have to literally draw your beaches, hills and mountains with layers of bluish colors.

Here a list of the tools you're going to need to accomplish your task:

.- A terrain generator, one that can save the heightmap as a 16bits RAW image; I use Machine World, which randomly creates terrains and lets you apply erosion and other natural "wearings" to make it more realistic. The trial version lets you save maps in a resolution of 512x512.

.- PhotoShop or any other image editor that can handle 16bits RAW images, resize images and crop your final map in tiles. PhotoShop is recommended because you can easily set guides for a much easier and faster cropping work. Remember quads are just 1024x1024 pixels.

.- TES4Gecko, nice small open source program to elaborate .esp and .esm files.

.- A lot of patience. Sorry, I had of my own, so I cant give you any URL where to download it from....

Point 2: A large world starts with a "container".

If you're reading this, you might've already noticed CS is a crash-maker-machine, it just loves inventing new reasons for crashing and it always does it when you're far ahead from your last saved checkpoint.

I've read about a solution, a workaround that lets you exploit some benefits from converting your plug-in (.esp) file into a master (.esm). This can be easily done by using TES4Gecko, a wonderful application that let's you bla bla bla and bla, read it from the linked page.

Ok, step by step:

.- Open TES CS; easy one

.- Many guides tell you to open Oblivion.esm, prior to set a new worldspace; this is because you're going to need references for items and other stuff that your empty plug-in do not have.....it's just empty, when you create it. When you open the Oblivion.esm, though, remember to unflag DEFAULT.esp, or your work will be saved in that file, instead of a new one.

.- Now that you've loaded Oblivion.esm, you're going to need your own worldspace to work with, just the worldspace you're going to save in your plug-in. I expect you to know what we're doing here, so I'll be brief: create a new worldspace and name it the way you prefer. Remember to untick "Small World", before clicking OK.

.- Ok, now just save. CS will ask you for a file name and that's it.

.- Now close CS down.

.- Now it's time to open TES4Gecko; it's just a folder with many files, one of which has a .jar extension. Open it using "Java Platform" and the application will start. It've very simple to use, it just has a handful of buttons that can do almost everything with .esp and .esm files. We're interested in the "Convert to Master" one. Click it, chose your newly made .esp and convert it. Now you can close TES4Gecko. (Notice the program doesnt delete the original .esp file, which cat still be found in the \Data folder)

Now we have our container. The benefits we're going to exploit from the .esm type of file is that you can load and work on larger worldspaces without (too many) crashes. You cant just load 64 quads in a single shot, though, so we're going to make multiple .esp files that will depend on your .esm.

Point 3: Loading your heightmap in batches.

I assume you know how to build a heightmap and crop it in separate 1024x1024 "quads", but I'll refresh your memory:

. To be usable, a heightmap must be a 16bits .raw image

. To be usable, a heightmap must be sliced in tiles of 1024x1024 pixels

. To be usable, heightmap tiles must be named in a specific way (that I'll mention in a bit)

. To be used, heightmap tiles must be placed in the \Data\Heightfield folder

Ok, let's say you've just finished elaborating your heightmap and you've exported it in a 16bits raw file. You've even already sliced it in perfect 1024x1024 squares. It's time to name them.

How to name heightmap tiles: imagine an Oblivion map like a huge square with x and y axis. Each axis have both positive and negative values and the 0_0 position is right in the middle of the map; yes and no!

Imagine you have an 8x8 tiled map you want to name with coordinates; this means the positions should go from -4(x)_-4(y) to +4(x)_+4(y). Wrong! You forgot there's a 0_0 position. So it actually goes from -4_-4 to 3_3 and the first "all positive" tile at the cross of the axis is 0_0.

Should you decide to make a 2x2 sized map, you'll just name tiles as -1_-1, 0_-1, -1_0 and 0_0. It's pretty tricky, so pay attention when you name your tiles, you could end up with huge holes in your worldspace because of bad named tiles.

For a quick visual reference check here.

Naming a 8x8 tiled map can be a tough job, must be done, the patience tool I told you about will help a lot here.

Once named do not copy all of them inside the \Data\Heightfield folder in one shot. I've managed to load a 64 tiles map in two batches of 4x8(=32) tiles, after converting the .esp to .esm. You might be not so lucky, but you can give it a try. If CS crashes or stops responding while loading the tiles I suggest you to try with 4x4 batches.

Step by step:

.- Copy the first batch of tiles inside the \Data\Heightfield folder; I suggest to chose tiles that are contiguous, like first all the positive x and then the negative x, or whatever you find more logical.

.- Open TES CS and load ONLY the data from your own .esm file, no other; wait for CS to finish loading everything..... (check DEFAULT.esp is not ticked, usually it is by default!!)

.- Now open the worldspace heightmap editor by clicking the button with the earth icon on it and select your own worldspace.

.- If you've copied any of the four central tiles inside the heightfield folder, you'll see them drawn in blue and light blue colors in the heightmap editor.

.- As you may or may not know, the freaking editor doesnt load all of the tiles inside the heightfield folder when you first open it; you have to force it to get all of the quads by opening the File menu and then clicking "Import". You will be asked if you want to import files from the heightfield folder, click Yes.

.- Now the CS might take a bit to load all of the tiles; if the program crashes or stops responding and closes down, try with a smaller batch of tiles. Notice that CS renames imported tiles with a new (actually added) ".imported" extension. Just rename each file by deleting the added extension to be able to import them again. If CS loaded them all with no problems, just save in the heightmap editor window and then in the main TES CS window, where you'll be asked to give a name to your brand new .esp plug-in file.

.- Repeat for all the other batches: open CS, load only your main .esm, copy the new batch inside heightfield flder, open worldspace editor, import tiles, save and save with a new name.

NOTICE: I suggest you to keep the heightfield folder open to check how the importing work is proceeding. If it seems like CS got sort of catatonic and is not renaming files anymore, try moving your mouse pointer over the program window; might sound stupid, but I found this to be a "wake-up call" that brings CS back to work. Do not do this too often, though, as it wont make CS to work faster, more probably it will make the program stop responding and crash.

Point 4: Generating distant LOD meshes and textures/normals for testing purposes.

If you've ever worked on a new worldspace for a mod, you'll probably know what you have to do now and how to do it. I'll just write it to be sure we both know what we're doing.

First you have to generate LOD meshes and then LOD textures/normals. To be honest, I'm not much of a one for solving the issues that you could encounter with this part of the job. Take this section as a starting point.

If you've not yet started working at the worldspace, but you're just reading, you might find more productive to generate LOD meshes each time you load a new batch of tiles (just save the new heightmap and the whole .esp, before generating LOD material).

To generate the LOD meshes, open the heightmap editor, open the File menu, click on Generate LOD and select "Full". You could actually chose to generate just a prototype, it's faster, but your worldspace is still pretty bare, so it shouldnt take long even if you go for the full version. No need to save anything, the LOD meshes are automatically saved in a separate folder (\Data\Meshes\Landscape\LOD) as .NIF files.

Now that you've generated the LOD meshes, you need to generate the textures to cover them, or everything will be radioactively purple, in the distance. You can now close the heightmap editor and start positioning your view on the 0,0 cell.

A brief explanation: you now have to "take pictures" of the land as it is so that CS will elaborate them into low res textures that will be applied over distant LOD meshes. This can be done only one quad at a time, so you'll have to manually move from quad to quad by selecting cells from the "Cell View" window.

You may or may not know that each tile (1024x1024 pixel) generates a quad made of 32x32 cells; so, to be sure you're right inside a quad, when generating LOD stuff, I suggest you to move by 32 cells starting from coordinates 16,16 instead of 0,0 (ie:16,16 | 16,-48 | 80,-16, etc etc...).

Dont worry, if you try to generate LOD textures for a quad where you've already been, nothing will happen. You'll notice the difference! Remember to turn off lights and water, before starting, I've read it's good to do so.

And you wont have to save anything as well, as the textures will be saved as .DDS files inside the "\Data\Textures\LandscapeLOD\Generated" folder.

Beware, there's an issue that involves bad file names generation for LOD stuff, I just cant remember if it was about textures or meshes. Anyway it's easy to check, just ensure that every .NIF and .DDS file name have two figures for all the coordinates: for example, if it's something like "16781011.-32.0.32.DDS", just change it to "16781011.-32.00.32.DDS".

As a last note, you may encounter the "black LOD textures" issue; awful beast. This is nothing that can be prevented nor solved if not by manually editing the .DDS files (at least for what I know). You'll find the images half black, that's the cause of your black LOD ingame. As long as you're working on a bare worldspace, all you have to do is to copy and paste the good half over the black one. With more complicated textures, where you could find different types of terrain in the same quad, I cant really help; that's beyond my personal experience.

Interlude: Testing your new worldspace.

Before merging everything in a big single .esp, you may want to test your world and see if it actually has the shape you dreamt of while generating and modifying it. I warmly recommed to test it as you might incur in some bad mesh, a hole or even a misplaced tile; at this stage you can still rework a single defective .esp or even redo everything without losing too much time.

Remember to check your .esm and .esp files are selected in the game launcher window (do I have to tell you how?). Then, once ingame, open the console and type cow WorldspaceName x,y; this will automatically teleport you to the new worldspace at the x,y cell coordinates (each quad is made of 32x32 cells).

Point 5: Packing it up

Once you're done with importing the heightmap tiles, you could actually make one file out of many by using the TES4Gecko again.

First of all make a backup of all the .esp and .esm files. Once you've saved a copy of everything, open the TES4Gecko and click on "Merge to Master", select the master file and then the plugin you wanna merge; do it for every .esp you have.

Now you have a bigger .esm file, which we're going to convert back to the .esp format. Do it by using TES4Gecko.

Here we are, now you can dispose of the .esm and .esp files you no longer need by moving them to a backup folder.

Point 6: You might find this useful.

.-| SPATIAL UNITS:

.---| Maps are made of quads.

.---| Each quad is made of 32x32 cells.

.---| Each cell corresponds to 4096x4096 CS heightmap editor units.

.---| Each cell corresponds to about 58,5x58,5 meters ingame.

.---| Each quad is 131072 CS units wide, which corresponds to about 1872 meters ingame.

.---| In the heightmap editor the max height reachable is 65535 vertical units, which corresponds to half a quad.

.---| In the heightmap editor water level is given at 4096 vertical units, that is 58,5 meters.

.---| Heightmap editor's vertical units correspond to double the horizontal ones, ingame.

.---| The water level thus corresponds to 117 meters from the very bottom and not 58, ingame.

Check here for visual reference.

When generating new terrains remember to scale both heights above and below water: remember that it's 58,5 meters to the water level and 936 meters to the sky/ceiling. If working with these numbers is too narrow, multiply both the distances for the same number ([x1] 58,5 and 936, [x2] 117 and 1872, [x3] 175,5 and 2808, [x4] 234 and 3744, etc etc....)

Edited January 8, 2012 by Zehryo