ESM Load Order Error

[michaelrw]

ive been using CK for quite a while and released several mods and made many more for myself. Fortunately for me ive never run into the problem im about to describe (until now).

 

Heres the scenario. I have an ESP mod with dependencies on all 3 DLCs. Then I have a patch for that mod that also has the same dependencies, plus is dependent on the main ESP. To work on the patch in CK, I run ESMify self on the main plugin in wryebash, then load up the mod, all of its dependencies, and the patch, setting the patch as the active file. I do a bunch of work to the patch then save and exit. upon loading up the mod and patch in tes5edit, i see that all of my formids are out of sync.

As it turns out, the CK load order is messed up and thus the order of the mod's dependencies is messed up. No problem though, I just go into wryebash and switch the order of the masters so that everything is where it should be:

00 Skyrim  

01 Update

02 Dawnguard

03 Hearthfires  

04 Dragonborn

05 MainMod.esp

 

Instead of:

00 Skyrim  

01 Update

02 Hearthfires 

03 Dawnguard

04 MainMod.esp

05 Dragonborn

 

However, this doesnt change the fact that all of the referenced formids in my formlists are still out of sync. A formlist with all of the dragonborn alchemy ingredients still has all of them listed with the 05 mod index instead of the 04 it should be.  Sure, I could go in and manually edit each formid but that would take forever. plus, what about the next time i want to make some changes to the file in CK?? Do i have to do this every time i want to make a change to the file? 

My first order of business was to see how the load order of CK was determined. I looked in AppData\Local\Skyrim and made sure loadorder.txt and plugins.txt were in good shape and reflecting the correct load order. Everything is fine there. I opened up the skyrim launcher, and everything is in order there. I ran BOSS through wryebash and ran it again through BOSS.exe directly. Every place is showing the proper load order EXCEPT the CK data window. I also thought that maybe it had to do with file dates, but all of the bethesda ESMs have the same dates. 

Basically I need to know how I can get the CK to recognize the proper load order so that when it comes time to save, it saves all of my records with the appropriate mod index. I can't be the first person to have experienced this.. so, what are other people doing to avoid/fix the issue.

Thanks 

[WillieSea]

I am not sure I follow exactly what your issues is.

But it is my experience that you never need to worry about the Form ID first two hex numbers, or what I think you are calling the index. Not everyone's 'index' will be the same, so the game decides what each DLC and mod will use for an index and correctly assigns them without intervention from you.

It uses the name of the master file, and then correctly assigns an index to that master file. Your mod would be 'adjusted' to the persons load order correctly.

 

Now, if your using TES5Edit or other second party programs to mess with things, then that might cause problems.

[michaelrw]

I am not sure I follow exactly what your issues is.

But it is my experience that you never need to worry about the Form ID first two hex numbers, or what I think you are calling the index. Not everyone's 'index' will be the same, so the game decides what each DLC and mod will use for an index and correctly assigns them without intervention from you.

It uses the name of the master file, and then correctly assigns an index to that master file. Your mod would be 'adjusted' to the persons load order correctly.

 

Now, if your using TES5Edit or other second party programs to mess with things, then that might cause problems

 

 

Yes, you are correct in that most of the time, the index doesnt matter because it is dynamic and will automatically change depending on your load order. When it comes to master order, i believe "relative order" is important. What i mean is, if the saved master order was:

 

Skyrim  

Update

Dawnguard

Hearthfires 

Dragonborn

MainMod.esp

 

then HF must always load after DG, but there can be other mods in between the two.. MainMod.esp must always be after DB, but it doesnt have to be immediately after. does that make sense? im not sure if this is all accurate, but my observations seem to suggest so.

 

Nonetheless, this all starts when CK saves the list of masters in a different order than they should be. I mean, there must be some validity to my claim that master file order can be important, otherwise there wouldnt be a CK wiki page dedicated to telling you how rearrange them back into the proper order if things get mixed up. 

 

I wouldnt say im using tes5edit to "mess with things" lol. im new to these forums here... is tes5edit frowned upon for some reason? if it werent for that program i wouldnt know half the things i do about esp/esm files and their structure and how mods work, etc etc. Plus there are things you can do with it that are either impossible with CK, or very time consuming. But i digress.

 

At the end of the day, there is a disconnect happening. With most other types of record i have not seen this cause issues.. simply rearranging the master order (either in CK "data" dialogue or in wryebash) will get things back on track. But whats happening here is that formids in a custom formlist are being hard-saved with a "non-dynamic" mod index. So when CK loads up the dragonborn.esm on the 6th spot, it saves all my dragonborn alchemy ingredients with an 05 index. Then, when you save and reload the file in a different program that is using the correct load order (as prescribed by the community and the BOSS team), dragonborn.esm now is in the 5th position and given the 04 index. When the plugin and all the masters are loaded, any native record or record override belonging to dragonborn.esm has the correct 04 index as well. However, if you go to the formid list you will see all of the dragonborn items with the proper IDs, but incorrect index.. still showing the 05 index. And now, since it still has the 05 index, its looking for the record in whatever plugin/master currently has the 05 index ... so naturally there is a an error message "could not find formid" or "error: could not resolve - null reference" or some junk like that.

Like i said, with most records this behavior does not happen. So far I have only experienced this with formid lists. My belief is that, since the formids in the list are not actual records themselves, they are not "as effective" at dynamically switching their indexes... for whatever reason. And, the other thing that is really irritating is i know all of this would not happen if you could rearrange the load order, and thus the masterlist save order, of mods in the CK. If I were to guess, this is not something bethesda developers ever had to mess with because they never had plugins that were dependent on 5 different master, both ESM and ESP master. At most they were dependent on skyrim.esm and update.esm, both of which *always* load 00 and 01 respectively.

[WillieSea]

TES5Edit is widely used and is a powerful and useful tool. But it can be abused if used incorrectly.

 

With a new game, there should be no problems with the index. If you have a save game, it will expect the DLC and mods in a certain order, and if you 'change' your mod list, it can mess up your save game. This much I know. Also, you must ensure your master file is actually a master file, including the change to the header which TES5Edit will do. But that can cause problems with a master file that has Dialogue changes since they can be dropped or cause it to not work at all. 

 

As for your other questions, I have never experienced a problem with the game not figuring out a mod that was dependent on a master file. Until there was a save game that expected something different. Perhaps someone else may be able to add more to the conversation.

[michaelrw]

TES5Edit is widely used and is a powerful and useful tool. But it can be abused if used incorrectly.

 

With a new game, there should be no problems with the index. If you have a save game, it will expect the DLC and mods in a certain order, and if you 'change' your mod list, it can mess up your save game. This much I know. Also, you must ensure your master file is actually a master file, including the change to the header which TES5Edit will do. But that can cause problems with a master file that has Dialogue changes since they can be dropped or cause it to not work at all. 

 

As for your other questions, I have never experienced a problem with the game not figuring out a mod that was dependent on a master file. Until there was a save game that expected something different. Perhaps someone else may be able to add more to the conversation.

 

Sure, i agree any tool can be a catalyst for disaster if used improperly. You do not know me and thus are correct to assume the possibility of me having screwed things up myself. For as much as its worth, i assure you that this is not something I "did" to the file from messing with things i shouldnt. I am certainly no master modder, but have done quite a bit of work and have published quite a few mods.. so i wouldt consider myself a rookie either. That being said, the reason i chose to post on this forum instead of Nexus for example, is that ive always seen TESA as a more advanced community and thought it would be a good place to look for answers to a more technical/difficult problem.

 

As far as the different save games, i dont know. I havent tested this on a bunch of different saves.. or any, for that matter. 

Basically it boils down to this; I have these two ESPs both in good working order. They could be given to anyone and installed and would adjust indexes properly according to their load order. everything is all good.

Then

I open the files in CK and resave them (probably have to create an ITM record or actually make some changes in order for it to let you resave them). Once you resave them in CK, they will no longer function as they did before. For the most part they will, but for items in my formlists used by my sorting scripts, it wont work. The indexes of the items listed in my formlists are no longer dynamic; they will be stuck with the mod index given to it by CK (in my case, CK gave these dragonborn items the 04 index, which now should have the 03 index based on tes5edit using a different/correct load order..but they do not, they still have the 04 index that they had while loaded in CK, so now tes5edit is looking in the wrong master for the base object)

Anyways, i appreciate your responses and your attempts at finding a resolution.

[AirtightSpring3]

For my mod I am working on there are a few errors that I can not figure out how to fix. The errors are.

 

PATHFINDING: NavMesh in cell 01AirtightTown (Tamriel[-25, -10]) should be refinalized, there are navmesh bounds missing.

 

Also various errors with triangles not being connected where they should be in Navmesh in certain cells, what is the best way for finding those certain Navmesh Triangles or should I just look individually.

 

I hope that these are not grave errors but if they are it looks like I will be redoing a fair amount of work. Any help would be greatly appreciated.

 

- AirtightSpring3

[michaelrw]

For my mod I am working on there are a few errors that I can not figure out how to fix. The errors are.

 

PATHFINDING: NavMesh in cell 01AirtightTown (Tamriel[-25, -10]) should be refinalized, there are navmesh bounds missing.

 

Also various errors with triangles not being connected where they should be in Navmesh in certain cells, what is the best way for finding those certain Navmesh Triangles or should I just look individually.

 

I hope that these are not grave errors but if they are it looks like I will be redoing a fair amount of work. Any help would be greatly appreciated.

 

- AirtightSpring3

From the navmeshing ive done, when i get those errors i usually just start over because i dont know where theyre referring to and what would be wrong with them (ive tried to find them and find mistakes but never been successful).  i would just save wherever youre at, open the mod in tes5edit, delete all of your navmesh records including any vanilla navmesh record overwrites if youre dealing with an exterior (dont delete the vanilla navmesh by accident (if youre dealing with a DLC cell - T5E wont even let you delete things from skyrim.esm by accident). Once youve cleared your mod of all navmesh records in the cell youre having trouble with, start from the beginning.  This is what ive had to do on more than one occasion.

maybe some of the more experienced guys will have something better

[AirtightSpring3]

From the navmeshing ive done, when i get those errors i usually just start over because i dont know where theyre referring to and what would be wrong with them (ive tried to find them and find mistakes but never been successful).  i would just save wherever youre at, open the mod in tes5edit, delete all of your navmesh records including any vanilla navmesh record overwrites if youre dealing with an exterior (dont delete the vanilla navmesh by accident (if youre dealing with a DLC cell - T5E wont even let you delete things from skyrim.esm by accident). Once youve cleared your mod of all navmesh records in the cell youre having trouble with, start from the beginning.  This is what ive had to do on more than one occasion.

maybe some of the more experienced guys will have something better

What if I had to completely delete the vanilla Navmesh to get the cell to work for what I had done to it. (Because I changed ground heights.)

[michaelrw]

That can be ok to do if its called for.

When i was talking about not being able to delete things, i was referring to the fact that you cant ever delete things from skyrim.esm. You can delete them in CK, but this only creates an override record of what you deleted and adds a *deleted* record flag. so when the game loads, it will read your mod and see the *deleted* flag, and treat the navmesh as if it were truly deleted from Skyrim.esm. If youre going to delete things from the vanilla game using CK, you should make sure to clean the mod with tes5edit and run the "undelete and disable refrences" command (part of standard cleaning). This will remove all of *deleted* flags from the override records you created when you deleted them in CK. tes5edit will replace all of these flags with a *disabled* flag (and if the its a world object it will be moved out of view from the game world so the player can not see it). 

Changing ground heights is fine. I had to do the same thing for a house mod i did. It was a pain to get the navmesh working. You can either edit the vanilla navmesh or "delete" it and regenerate your own. I ended up removing the vanilla and regenerating my own. Experiment with the different types of navmesh generation and the settings used for each type. eventually i found a method (dont remember which one) that worked really well for my situation, and all i had to do was a few minor edits to get it all put together. The other big task when you do it this was is making sure the navmesh is properly attached to all surrounding cells. i found a video on youtube the first time i did this. you should be able to find it if you look. but basically all navmesh has to be connected properly and the joining/shared vertices must sit directly on the cell border between your cell and the other one. Of course, you may already know all about this.

[AirtightSpring3]

Thank you for your help, I will have to download TES5Edit and follow your steps before I attempt any more work.

[AirtightSpring3]

That can be ok to do if its called for.

When i was talking about not being able to delete things, i was referring to the fact that you cant ever delete things from skyrim.esm. You can delete them in CK, but this only creates an override record of what you deleted and adds a *deleted* record flag. so when the game loads, it will read your mod and see the *deleted* flag, and treat the navmesh as if it were truly deleted from Skyrim.esm. If youre going to delete things from the vanilla game using CK, you should make sure to clean the mod with tes5edit and run the "undelete and disable refrences" command (part of standard cleaning). This will remove all of *deleted* flags from the override records you created when you deleted them in CK. tes5edit will replace all of these flags with a *disabled* flag (and if the its a world object it will be moved out of view from the game world so the player can not see it). 

Changing ground heights is fine. I had to do the same thing for a house mod i did. It was a pain to get the navmesh working. You can either edit the vanilla navmesh or "delete" it and regenerate your own. I ended up removing the vanilla and regenerating my own. Experiment with the different types of navmesh generation and the settings used for each type. eventually i found a method (dont remember which one) that worked really well for my situation, and all i had to do was a few minor edits to get it all put together. The other big task when you do it this was is making sure the navmesh is properly attached to all surrounding cells. i found a video on youtube the first time i did this. you should be able to find it if you look. but basically all navmesh has to be connected properly and the joining/shared vertices must sit directly on the cell border between your cell and the other one. Of course, you may already know all about this.

I went into TES5Edit and cleaned my mod through a tutorial on the creatoin kit site, now those Navmesh edits that I did are gone (what I wanted.) But when opening in creation kit I still get the warning because the old skyrim Navmesh was not restored, (which is what I wanted because the cells with the warnings are not actuallly used in the mod, but were in another one of my mods I opened with mine to transfer some of my work...) Should I redo the Navmesh of these cells by hand and then finalize them and see if the errors go away or is there a way to restore the old skyrim Navmesh?

 

Edit: I fixed the problem by deleting all Navmesh records in Tes5Edit with in my mod for the area that was having problems, I then went back in creation kit, refinalized the original one's which were restored, and then reopened the mod and all errors relating to my mod were gone. Only errors left now are one's that pop up because of a Bethesda error in Dawnguard. Thanks for your help, I can no continue work without worrying about those bad edits of mine.

Edited August 28, 2013 by AirtightSpring3

[michaelrw]

I went into TES5Edit and cleaned my mod through a tutorial on the creatoin kit site, now those Navmesh edits that I did are gone (what I wanted.) But when opening in creation kit I still get the warning because the old skyrim Navmesh was not restored, (which is what I wanted because the cells with the warnings are not actuallly used in the mod, but were in another one of my mods I opened with mine to transfer some of my work...) Should I redo the Navmesh of these cells by hand and then finalize them and see if the errors go away or is there a way to restore the old skyrim Navmesh?

 

Edit: I fixed the problem by deleting all Navmesh records in Tes5Edit with in my mod for the area that was having problems, I then went back in creation kit, refinalized the original one's which were restored, and then reopened the mod and all errors relating to my mod were gone. Only errors left now are one's that pop up because of a Bethesda error in Dawnguard. Thanks for your help, I can no continue work without worrying about those bad edits of mine.

 

youre welcome buddy, i know how it is. im the same way. glad you got it worked out though