Search the Community

Greetings, all! I need some expert-level help on splitting a rather large ESP into an ESM/ESP pair, and so far neither Google nor tool documentation nor direct searching on TESA has found me a way to do this. I know that TES4Gecko can "split a mod", but it puts way more stuff into the ESM than what I want there, and it doesn't seem to allow me any manual control over that. Background (sorry, this is long, but the information matters): My ESP file defines a couple thousand new base objects, seven interlocking quests, hundreds of custom scripts, and over 3,000 lines of voiced and lip synced dialog. Among those base objects, some use vanilla meshes and textures (e.g., a vanilla door but with a custom script added requiring a base object just to add the script, but no external assets). Other base objects use custom meshes, textures, sound files, music files, or some combination of these. All told, the added meshes and textures total about 250 meg, and the voice files are about 150 meg. I deploy the mod to my alpha test team (several people) as an ESP plus three BSA files (one each for meshes, textures, and sounds which includes voices and lip syncs). Development has reached the point where the meshes and textures are quite seldom changing, but I'm still editing recorded voices and inserting them into the mod. That means that of 400 meg download, about 250 meg of it is the same for the testers every time. Looking ahead, we're just a few weeks away from a public beta release, and hopefully a lot more people downloading. There is no way the voice editing will be 100% done before the beta, so I need to make that part easy to update for testers. Furthermore, the ESM/ESP split is the right thing to do *long term* as well. The reason for this is that the music handling logic for the mod is smart enough to detect Sound Commands, Enhanced Sound and Music, or Better Music System, selecting the best available option, and falls back to vanilla StreamMusic() calls if none of those are installed. The initialization logic builds arrays of the mod's internal cells and which music files should apply to which cells (merchants, taverns, dungeon....etc.). All of that logic is something I don't want to duplicate in the (required by BMS design) separate ESP file that supports BMS. So my code builds the arrays in one place and then shares them with the optional ESP that supports BMS -- and this requires the optional ESP to have my main ESP as a master. I know how to do that, but it's an ugly hack that I'd like to avoid. If I create an ESM, all that shared logic for music handling can go into the ESM. There are other optional ESPs planned, such as one explicitly supporting CM Partners. So getting some of the global logic into an ESM has lots of benefits that have nothing to do with download size. Now, given the above, here's what I need to do: The ESM should contain only one quest, and not one of the existing ones. This quest will contain the music data arrays and initialization logic, so that MyMod-BetterMusicSystem.esp can depend on my ESM and not my ESP. The ESM quest will also have a few declared variables that are shared by the other quests, but other than that it won't have much code. The ESM should define most or all of the unscripted base objects (or base objects that use only vanilla scripts) that use custom meshes, textures, and sound FX files, but *not* voice and music files. For scripted objects such as custom activators, I'll add a dummy unscripted version that has the model attached but no script, just so the models get pulled into the BSA for the ESM file. The ESP will contain all quests except the one mentioned above, all dialog, all interior and exterior cell edits, and almost all of my custom scripts. In short, the ESM should mostly define base objects, and the ESP should have references to those base objects. That last bit is the problem. Here is what I've found so far with the tooling I have, which includes Wrye Bash, Construction Set Extended (very latest version), TES4Edit, TES4Gecko, and TES4Files, among others: TES4Gecko does a perfect job of splitting the mod, but it's way too aggressive in what goes into the ESM. With Gecko, all new records go into the ESM because they can safely load early, and override records stay in the ESP so they can load after the things they override. That is sensible enough for some situations, but not what I want to do here because all those voice files end up in the ESM. I have tried copying my ESP to a different name, using Wrye Bash to "esmify" one of them, then using either TES4Edit or CSE to delete or mark unmodified the records in each file that I want to use from the other. This basically results in vast numbers of missing records. I tried making the ESP as a clone of the ESM such that there are intentionally (and temporarily) huge numbers of ITM (identical to master) records in the ESP, the idea being then to delete out of the ESM those that should be in the ESP and vice-versa. The problem here is that when I do this, even though I created the ESM by physically copying the ESP file and esmifying it in place (that is, never opened in the CSE), neither Wrye Bash nor TES4Edit considers the duplication to be ITM records! This completely surprised me. I have tested with changing just a few objects in the ESP manually (in TES4Edit or the CSE) so that the references point to mod index 00 (implying "load this from whichever master has it defined"), and this works -- but with thousands of base objects, it doesn't scale to my situation. Using TES4Gecko's display-and-copy functionality doesn't work, either, because the export gets new formIDs for the base objects. Creating an empty (or nearly so) ESM in the CSE, then making it a master of the ESP, then using TES4Gecko's "merge to master" function doesn't help, because it uses the same decision logic as the "split plugin" feature and puts everything into the ESM that it possibly can. Looking at the references in the ESP after doing any of the above, it appears what is happening is that the ESP's references are hardwired to itself rather than pointing back to the base object that now exists in an ESM. I think Coda (the programming language embedded in CSE) could do what I want, but try as I might I can't find any documentation for Coda's callable functions. There is documentation for the language syntax but nothing I can find for available functions to do things like change the base object ID of all references of a specific type. I'm sure someone has done this before, but I spent almost an entire day trying to figure out how, and I'm stumped. I don't need (or want) the tooling to pick which objects go where -- that's got to be a manual decision. All I need is something that updates the references in the ESP correctly and lets me do a group of manually-identified base objects in a batch. Ideally, what I want is a tool where I can start with an almost-empty ESM, master it over the ESP, then use a feature like TES4Gecko's "merge to master" but *only on base objects I select* in the UI. Second choice would be more extensive documentation on Coda so I can create my own custom logic to do exactly what I need. I am an IT pro professionally, so writing code doesn't scare me -- but I need API docs and not just syntax docs. I'm sure someone has solved this before. If that person is you, please help a guy out here! As an incentive for replies, I will offer that whoever provides the solution that ultimately works for me, can be listed in the mod's credits as a technical contributor. :-) Many thanks for reading this long post, and for any advice you can offer. --Syscrusher