Improving FPS of meshes -- general question

[syscrusher]

I've set up a test cell with *just* my house mesh in it, and only ambient lighting. Inside that cell, I'm reliably getting 160~200 FPS on my relatively modest video (mobile NVIDIA). In the real house, I'm getting 10~20 FPS. By comparison, walking around outdoors in a wilderness cell I see 60~70 FPS on this hardware.

Updated FPS: At one "worst-case" point in the house, I was getting 7~11 FPS.

Now, here's what I've found so far:

1. I had accidentally used practicals on 13 wall sconces that just didn't need to be. I bumped up the ambient from 30 grey to 60 grey and replaced all 13 of those with the fake variants. Instant improvement in the typical (formerly 10~20) case to about 16~24. No brainer...that's a keeper.

2. I replaced all my fireplace flames with nonflickering variants, and reduced their radii to cut down on overlaps. Another noticeable jump.

3. I found one true "problem child", a simple phantom light to simulate a desktop candle. I thought I had pasted in a 175 radius; it was 1750! Blasted thing was overlapping almost the entire house! Also, I had one light I'd pasted in for testing that was a 512 radius, and had forgotten to remove later. Got rid of both of those.

4. Replaced all flicker sources, including phantoms, with non-flickering, and reduced some radii.

5. I eliminated a nirnroot with its sound effect from the conservatory. Gained me about 1~2 FPS in that area.

Result: My worst case was 7~11 FPS, now 16~22 FPS. Still not acceptable, but at least *tolerable*. My typical case, evident throughout all but one small part of the house, has gone from 10~20 FPS up to 22~38 FPS, tending toward the high end of that when I am not moving the camera rapidly.

The improvement is actually larger than what you would conclude from these numbers; in my "before" numbers, several of the fireplaces (which are player-controllable) were off, while in the "after" numbers every light in the house is on.

All of these numbers (before and after) are with four NPCs moving around in the cell.

I experimented with eliminating the flora from inside the house. It was measurable, but not earthshaking, and I've decided to keep the flora because this is a case where the coolness of the feature outweighs the performance hit.

I'm a long way from being happy with this, and a long way from being done, but this is certainly a huge improvement.

Next steps:

1. I still have a few overlapping light sources. In several cases, these are because ceiling-hung chandeliers are passing through the floor to the level above. I'll replace these with static chandeliers and phantom lights, the latter being lower in the Z axis so they don't flow through the ceiling so much.

2. There are a few places where I have deliberate overlap, because I'm layering daylight with interior light. However, the daylight will be scripted to follow the clock, so I'll reverse-script the manmade lights to be on when the daylight is off. Also, I will reduce the amount of clutter that "just happens" to lie in the overlapping regions.

3. I'm going to take a good hard look at my clutter over the next few days, and will probably make some clutter combos in Blender for things that involve a lot of small objects like groups of potion bottles.

Thanks again for everyone's help; I'm making progress, and have a fairly clear path forward.

Syscrusher

[syscrusher]

One cautionary note to others like me who are relatively new to this: I assumed that since a bunch of the vanilla lights have "Flicker" in their editor names, that the others (especially the phantom lights) were steady. WRONG! There are a bunch of lights that flicker and don't have that in their base object names. I cloned those objects and made my own custom non-flickers so I wouldn't change the objects for the whole game, just my mod.

Syscrusher

[Khettienna]

You may want to consider allowing All Natural to handle your indoor day/night stuff for you via patch, or at least using the same method of creating your own exterior weather type and setting the cell lighting to behave as an exterior. This is better for performance than adding new lights, because it's a single directional light rather than multiple radial lights, and it replaces the ambient lighting of the cell. It also has the benefit of being a heck of a lot less work.

I have found indoor day/night lighting transitions to be much like LOD. Those who care about it a lot probably already have TES4LODGen, and therefore packaging LOD files is somewhat inane. Likewise, people who really care about day/night lighting probably already have All Natural. Using its indoor weather stuff is silly easy, and there are even instructions for it in the readme.

That's just been my experience, though - if you're happy with what you've got, by all means, keep with it!

[syscrusher]

You may want to consider allowing All Natural to handle your indoor day/night stuff for you via patch, or at least using the same method of creating your own exterior weather type and setting the cell lighting to behave as an exterior. This is better for performance than adding new lights, because it's a single directional light rather than multiple radial lights, and it replaces the ambient lighting of the cell. It also has the benefit of being a heck of a lot less work.

I have found indoor day/night lighting transitions to be much like LOD. Those who care about it a lot probably already have TES4LODGen, and therefore packaging LOD files is somewhat inane. Likewise, people who really care about day/night lighting probably already have All Natural. Using its indoor weather stuff is silly easy, and there are even instructions for it in the readme.

That's just been my experience, though - if you're happy with what you've got, by all means, keep with it!

"Happy with what I've got" is a non-issue, because I had only gotten as far as placing a few phantom lights as placeholders; I haven't done the scripting work behind them yet.

Ergo, I am easily able to conclude as follows: "Excellent advice, enthusiastically adopted, and thank you."

[syscrusher]

"Happy with what I've got" is a non-issue, because I had only gotten as far as placing a few phantom lights as placeholders; I haven't done the scripting work behind them yet.

Ergo, I am easily able to conclude as follows: "Excellent advice, enthusiastically adopted, and thank you."

Incidentally, I'm already running All Natural on my own system (and loving it!) because a certain someone who shall remain nameless because I don't want to embarrass KHETTIENNA was kind enough to recommend that excellent mod.

Sys

[syscrusher]

I have found another factor that seems to make a large difference: The vanilla mesh I imported as a starting point had two-sided collision meshes, and I kept that as a default when I reworked the model. I dove into the re-collisioning today with the intent of getting far enough to do some testing, and I've found two things:

1. The collision mesh really does make a large difference in my situation, and it's worth it to re-collision the entire mesh. I'm taking a very low-poly approach, forgoing detail on the collision in areas where the PC won't be able to tell the difference during normal play. For instance, there are some railings that visually have gaps between the vertical slats. In a dungeon, these would matter because characters might shoot ranged attacks through them. In a player home, I've decided they Just Don't Matter [tm], and I'm collisioning those railings as a single rectangular object for a reduction of around 90% on the railing's poly count. My original collision had about 3400 polygons; my goal is to reduce that by at least 60% in the new mesh. I'm hoping to be under 1500 polygons.

2. The dual-sided collision mesh works perfectly fine in-game, but it really does seem to eat up a lot more CPU. Other things being equal, with my new partial mesh (which has about 400 polygons so far, and is done enough to be walkable with care), I observed a change in the most-cluttered area of about 35% performance improvement in game. Lesson learned: Two-sided meshes are fine when you need an object of zero thickness, but they are a bad idea for normal collisioning such as floors where the ceiling below is not on the same height plane (that is, the intervening space has finite nonzero thickness).

[AndalayBay]

In terms of reducing the poly count on collision meshes for architecture, we did the same thing. Originally our modeller simply created a "standard" mesh, which had a high number of vertices in the roof. We decided that a much simpler collision mesh would work just fine for the roof. It reduced the total poly count by almost half.

[Arthmoor]

I'm curious then to know how many other vanilla meshes might be suffering the same problem. And is this something PyFFI will correct?

[Artisanix]

You mean collision?

Well, PYFFI or any automated tool can't correct badly made collisions properly. As the best collisions are still those made "manually" by a human, who decides which portion should be detailed and accurate and which can have a simpler shape (that all depends on context in which model will be used).

As for such meshes there are a few, some ayleid roots come to mind which have collisions exactly the same as models - and that really puts some strain on performance if you use them in great numbers.

[Arthmoor]

I was referring to being able to fix the two-sided collision problem syscrucser was talking about. Where there is one mesh doing it, there are probably others.

[washington]

I reckon that it wouldn't be a problem to do it by automatic command, but the real problem would be to check huge amount of vanilla meshes to find out which ones to include into the process.

There is also something similar I have noticed. The IC palace tower has much details (the mesh itself, I don't know about the collision) which is not really needed because player never sees it from close distance. I think that if remodelled, the poly count could be drastically lowered.

[Artisanix]

I was referring to being able to fix the two-sided collision problem syscrucser was talking about. Where there is one mesh doing it, there are probably others.

The point is that there may be no such problem as "two-sided collision". Because from my point of view, with all the respect, syscrusher rather complicated things than explained them in his post ;]

First he needs to explain what exactly the "dual-sided collision" means :] Because if he refers to just a simple shape like plane, then well, there are legions of these in meshes - every regular wall has them, every interior floor and ceiling, etc. Besides, in his post in pt.2 he mentioned some 35% performance improvement and made a conclusion that those "two-sided meshes" are responsible for this :] Yet he uses collision that has 400 polys to test this and original one had 3400 polys - so to me it looks like that gain in FPS was because of much less detailed collision overall than an any influence of those "two-sided meshes" ;] Really this case needs to be explained better as you may suspect the specific problem where there's no any ;]

[syscrusher]

You mean collision?

Well, PYFFI or any automated tool can't correct badly made collisions properly. As the best collisions are still those made "manually" by a human, who decides which portion should be detailed and accurate and which can have a simpler shape (that all depends on context in which model will be used).

As for such meshes there are a few, some ayleid roots come to mind which have collisions exactly the same as models - and that really puts some strain on performance if you use them in great numbers.

[syscrusher]

The point is that there may be no such problem as "two-sided collision". Because from my point of view, with all the respect, syscrusher rather complicated things than explained them in his post ;]

First he needs to explain what exactly the "dual-sided collision" means :] Because if he refers to just a simple shape like plane, then well, there are legions of these in meshes - every regular wall has them, every interior floor and ceiling, etc. Besides, in his post in pt.2 he mentioned some 35% performance improvement and made a conclusion that those "two-sided meshes" are responsible for this :] Yet he uses collision that has 400 polys to test this and original one had 3400 polys - so to me it looks like that gain in FPS was because of much less detailed collision overall than an any influence of those "two-sided meshes" ;] Really this case needs to be explained better as you may suspect the specific problem where there's no any ;]

[syscrusher]

I also have discovered that some of my potion bottle clutter objects, imported from a modder's resource, are static (as advertised) but have really complex collision geometry. Since they are inside a sealed glass display cabinet that doesn't open, I'm going to take the time to make collision-free versions of them. I don't know how much it will help, but it certainly won't hurt.

[Artisanix]

syscrusher -

Thank you very much for your info, but... it's something missing there I think :] You haven't explained what exactly one-sided or two-sided collision means :]

Well, I'm asking because I work with various collisions for years now, and read a lot of things about them in the past but never ever I've encountered such terms anywhere. This is the first time when I can read about one- or two-sided collisions. So without an understanding of the key element your posts about it aren't as valuable as they may be ;] So, please post two pictures - one with one sided collision and another with two-sided, or even better (if you want) - upload those two nifs with different collisions somewhere, so others could check them and learn something in the process :]

[washington]

Huh, I thought it was pretty straightforward... Two-sided collision has 'Double sided' button in Blender pressed and one sided doesn't. Two sided plane would block the player from both side while one sided would block player only on the side where the normal points. Am I right?

I always (If I remember) turn off double sided, unless I need it for some reason.

[Arthmoor]

That's basically what I took it to mean too, but being one of those people Blender hates I wouldn't have had any idea how to even think about fixing it

[Artisanix]

err, guys, what you're saying is true but for normal meshes.

If double sided button is pressed for particular mesh (plane for instance) then it "tells" the engine to draw texture on both sides of each surface and it adds NiStencilProperty to the nif file on export. That is perfectly clear.

The point is though, that collisions are just different animals. That's why they are usually represented and treated as wireframes. Because such things like direction of normals, double sided feature, or other tricks that are related to normal meshes don't work for them (and are just omitted). Every single plane of collision in game just blocks from both sides, and it doesn't matter whether the mesh you had previously to make this collision from, had normals pointed inside or outside, solid or smooth faces, or double sided property - the exported data of such collision is always the same (just a very simple "list" of vertices and triangles and nothing more.

That's why I fail to see (so far) how this can improve performance by such relatively huge amounts. So I can only suspect that syscrusher has something different in mind ;] - well, I really do hope for some nice discovery here (some kind of breakthrough) that could allow us to improve performance of collision more - unlucky not enough specific data from syscrusher so far, so I would like to see those meshes he uses and inspect the subject by myself to eventually confirm such good results he describes ;]

[washington]

Are you sure? I think that it is also relevant for collision. For example, I created interior mesh by flipping normals on exterior and I forgot to do so on collision mesh. While player couldn't go through, arrows passed right through the wall. When I flipped normals on collision mesh too, arrows stuck in the wall as planned.

I haven't tested or experimented on that and honestly I don't see practical advantages of that even if it does affect on performance.

[Artisanix]

I wasn't so precise there I admit ;] I put everything to the same bucket to make post shorter, but it doesn't change that much anyway.

Of course the direction of normals is important for arrows or particles (and is recorded in collision data), and it changes nothing for the player (he still is blocked from both sides). But the double sided feature does exactly nothing - the collision is exported and works as it has only one side enabled.

[washington]

Now you got me intrigued and I had to test.

Same ugly, hairy collision mesh with 'Double sided' on has the size of 179 kb, while that same mesh single sided has only 28 kb. That must make a difference in performance too.

My experiment: normal cube, subdivide multi fractal 20 on both pop ups, Logic settings triangle mesh and HAV_MAT_METAL.. Try it yourself.

http://www.mediafire.com/?hoau4tz7bbhfgww

[washington]

And here's something more interesting too!!! Pyffiing the meshes got them both to be 179 kb !

[Artisanix]

nice, but the size of both files you put into that 7z is exactly the same and the content as well ;]

edit:

see you noticed this :]

anyway I was just thinking... Blender uses PyFFI by default on every export, so there will be no any difference in size and content between them. But what about 3DS users? washington, you exported this from 3ds or Blender? :] Maybe try not to PyFFI the meshes from 3DS and test it instead...

[washington]

nice, but the size of both files you put into that 7z is exactly the same and the content as well ;]

edit:

see you noticed this :]

anyway I was just thinking... Blender uses PyFFI by default on every export, so there will be no any difference in size and content between them. But what about 3DS users? washington, you exported this from 3ds or Blender? :] Maybe try not to PyFFI the meshes from 3DS and test it instead...

From Blender. I pyf'd it later so I probably packed pyf'd meshes by mistake. Try reproducing if you have chance.

I tried some other meshes (but with mesh and collision, all low poly), smaller ones, but I got same sizes this time.

Edited May 10, 2011 by washington