7 posts / 0 new
Last post
Ormin
Ormin's picture
Contributor
Offline
Last seen: 1 year 8 months ago
Joined: 02/03/2012 - 13:17
Karma: 114
[TUTORIAL] Finding Invalid Meshes with tools

Hi guys,

First and foremost - I am not certain whether people here found any way of searching for those crashing meshes simple way already. I just discovered a method which can be used to identify those meshes. Mind that it's quite time consuming, but accurate, also involves using a lot of low-level tools. I am going to try to explain the guide as simple as it's possible, so everyone ( or almost everyone ) can make use of it, more people can do it - the better.

1. Tools Required

2. Configuration of the Tools

2a. Configuring PrGrep

Make sure you configure your PrGrep to search for binary files ( the box on the right to the ,,containing" field ) and searching in subdirectories and other options. Set "directory" path to your destination mesh directory ( in this case i've chosen Meshes\tes4, because I wanted to search for invalid meshes from Skyblivion ). A correct configuration looks like this:

( http://i42.tinypic.com/auduew.png full resolution )

2b. Configuring Olly Debugger

Nothing to be configured, just make sure you run it as an administrator.

2c. Configuring the Game

In Launcher, tick all the .esp / .esm you want to try ( in my case I had Skyrim, Skywind, Skyblivion and Skyblivion Beauty/LOD ticked ). Make sure your game runs in windowed mode ( otherwise debugging might get really hard )

3. Starting the game from the debugger.

Start the Olly Debugger. When the big window appears, drag on TES5.exe from Skyrim's folder. Wait, until it ends the analyze ( you can see it being analyzed in the bottom bar below the window ). Hit the f9, it should say "Running" in the lower right box. In case it stops cause of one reason or another, for now, just keep pressing f9 until Skyrim menu starts ( in the another window that is ).

Now, start whatever you would like to do. Run a game, run a COW command from a terminal or whatever you wish to find a crashing mesh. I just did COW Tes4Tamriel 5,5.

When the game is about to crash ( the invalid mesh is being hit by the game engine ), the debugger will stop and you will see something like this ( or similar, but from what I've observed, the crashes are always in similar places ):

( http://i42.tinypic.com/n12p28.png full resolution )

the black address on the left shows the place the crash happened. It's the int3 command.. see the next command ( jmp xxx) ? Click on it and hit enter. You should land in a place similar to this:

( http://i40.tinypic.com/fjkl91.png full resolution )

The thing here now is to see the three ,,mov xx,byte ptr ds:[REGISTER+something]. IN my screenshot it is EBX ( you can see it three times in a row ). Now , search for it on the right panel ( on my screen - EBX =156D9A1A )

Now, in the left down corner, hit the big space ( where zeros on my screenshot are ) and press CTRL+G. Write the offset you've readed prior here, and hit enter. The window here should change and there should be a lot of nonsense characters and random bytes. Copy , let's say, first 8 bytes from this:

( http://i42.tinypic.com/6ss2tj.png full resolution )

That's it - you actually copied a small small part from a mesh which was causing the problem. Now, back to PrGrep, in the ,,Containing" Field paste what you just copied. Remove the paddings ( unnecessary post-spaces etc so its like AA BB CC and nothing else ) and hit search. If you've done everything right, eventually the full path of model ( with an ascii representation of what you've been searching ) will be shown in the window.

4. Fixing [optional]

Crashes which I am describing here are based on the MOPP collision bug which affected quite a few meshes from both skywind and skyblivion. To fix it you can remove the appropiate branch using nifskope or just add a new collision by hand.

That's it!

 

 

Edited by: grumpycat on 06/04/2013 - 22:27
zilav
zilav's picture
Contributor
Offline
Last seen: 9 months 1 week ago
Joined: 11/26/2012 - 12:51
Karma: 1338
Thanks, but I highly doubt

Thanks, but I highly doubt there are people interested in Skywind/Skyblivion and don't afraid of assembly language (except you and me of course) :)

Deleting collision is not a fix for problem, we need a proper collisions. There are guides available on how to, for example search for "collider meshes" here http://www.darkcreations.org/forums/topic/4249-100-custom-tr...

But it involves a lot of manual work, and considering there are probably hundreds of meshes without proper collision in Skyblivion, I don't know how we can fix them.

Those bad meshes don't always crash the game, but they don't collide with anything either and sun shines through them. For example look at shadows here

https://www.youtube.com/watch?feature=player_detailpage&v=3J...

https://www.youtube.com/watch?v=46VzRbIuoLQ&feature=player_d...

Ice
Ice's picture
Member
Offline
Last seen: 2 years 10 months ago
Joined: 12/11/2012 - 15:30
Karma: 1036
So while I'm learning

So while I'm learning Payprus, this seems over my head, but really excites me at the same time.
From what I gather, Olly debugger/your setup is allowing you to see the game's code as you play. That's allowing you to search the code to find not only the bad mesh but the bad part of the bad mesh, that caused the game to crash.

Is this sort of thing used to create SKSE functions? Is this how they find variables to write getsomething() commands?

On another note: every mesh crash I've ever run into with my own custom made meshes was the result of a trishape not having a name in Nifskope. In Nifskope it looks like it has a name but then when you click to rename it, it's blank. Are we sure the problem has to do with collision, and doesn't just need to be renamed?

Ormin
Ormin's picture
Contributor
Offline
Last seen: 1 year 8 months ago
Joined: 02/03/2012 - 13:17
Karma: 114
@zilavThat's why i tried to

@zilav

That's why i tried to make the thing as simple as possible, so even someone who didn't see assembly at all is able to find the invalid meshes. That's also why I've done screenshots and explained it step-by-step without getting into what is happening. Yeah, I know collision deletion is not a fix, I just pointed to WHAT should be fixed :))

@Ice

You got the basics - yeah, indeed that's the case, the place you are highlighting here are exactly the bytes causing the game to CTD ( or actually, it's not a crash but an infinite loop ). If I recall correctly what SKSE is, then you are correct on this matter, too - reading assembly can provide a lot of valuable info, but mind it that it's already doing so-called Reverse Engineering and if i recall correctly it's disallowed by EULA - therefore I do not condone it :P

 

 

Drakonman
Drakonman's picture
Member
Offline
Last seen: 2 years 7 months ago
Joined: 02/08/2013 - 02:12
Karma: 188
Hey, this is simple enough.

Hey, this is simple enough. Its not hard people. its time  consuming on the grounds of having to go cell to cell in game and finding the crashes. Making a Collision model is easy enough. Iv already fixed a few models and noticed the crashes around ruins have dropped significantly.  In all honesty most the models are fine its just the dungeon tile sets that are crapping out for some reason. 

zilav
zilav's picture
Contributor
Offline
Last seen: 9 months 1 week ago
Joined: 11/26/2012 - 12:51
Karma: 1338
All plants meshes are bad,

All plants meshes are bad, they have collision reusing the same high poly mesh and causes severe FPS hit in wilderness, crash the game or crash when you approach some of them.

I don't know how to say properly, but they should have low poly collision so you can activate them, but at the same time you must be able to walk through them. Considering plants and mushrooms are everywhere, they are the major issue currently.

zilav
zilav's picture
Contributor
Offline
Last seen: 9 months 1 week ago
Joined: 11/26/2012 - 12:51
Karma: 1338
Just leaving a link about

Just leaving a link about making proper collisions (better leave it here instead of adding to private favorites)

http://forums.bethsoft.com/topic/1464441-quick-questions-qui...