49 posts / 0 new
Last post
RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Building a Oblivion to Skyrim conversion tool [outdated]

Hey everyone, great to see the excitement and abition still going. Even with Skyrim out now, Morroblivion is still a blast.

Porting Oblivion into Skyrim isnt strictly impossible, but it may as well be. There are some things we'll need first (and it could be monumentally difficult). The different game engines makes something like this MUCH harder than coming up with the original Morrowblivion converter.

However, ALL HOPE ISNT LOST. If a conversion program is to be made, there's one way I can see it happening, but it'll take time, and programmers (my friends and I are can program, but we're... novices when compared to a task like this). A rough way to think about it would be in the following easier-said-than-done-steps.

1)We'll need to know the internal file structure's for oblivions esp's (that is, we need to know how to read from and write to esp files). Fortunatley, this has been done already, there are a number of programs made by modders that can open up esp's, tinker with them, and re-save without a problem. Unfortunatley, there dosnt seem to be much information that I can find on how to actually do it, but we know its possible.

2) We'll need to be able to do the same thing with Skyrim's mod-files. The construction set is comming, and its only a matter of time before the file structure is figgured out. We're lucky that bethesda gives us so much access to thier files and modding tools, if this were a different engine, decoding the filestructure would be even more of a pain in the ass. But this is something we'll have to wait for, unless some tallented programmers come our way, and we'll still have to wait for the construction set to even begin.

3) Write a program that reads data from an oblivion esp, and inserts it into a Skyrim one. This part will take a tone of work, there's an endless amount of things that can go wrong, and it's a huge task (and its madness). It should be possible, because for MOST things in Oblivion, there will be an equivalent in Skyrim. (e.g, NPC, worldspace, dialog, item, activator). But we're going to need to know the file structures inside out! We'll need to interpret every piece of data about every element in the oblivion esp, and understand where we need to put it in the Skyrim one. How that data needs to be rearanged, what data needs to be left behind because Skyrim dosnt have a place for it, and what data needs to be fabricated with a placeholder because Skyrim has something new that wasnt in oblivion.

Its possible guys, but i think we'd almost be better off doing it manually, and either way could take years. Although seeing as we have a lot more activity on the site now days, it could be possible. To do it manually, we'd start with the world space, maybe try to take the heightmap from oblivion and generate it in skyrim. Then... the rest is just incredibly daunting.

I think everybody should just enjoy our achievenents in Morrowblivion and the new and shiny Skyrim for now. Once the construction set is out for a while, maybe there'll be some developemts that'll make this task possible, for now though, anything we start is going to run into a solid wall very fast, and the project will probably stagnate for a few years anyway like Morrowblivion did all those years ago.

I'd love to see it happen though. And I'm sure we arent the only mod community thinking about doing this, someone else might surprise us with a way to begin in the future!

Edited by: grumpycat on 03/25/2013 - 15:20
ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Maybe the other way,put

Maybe the other way,put skyrim into the oblivion engine,there is already a place for skyrim on the oblivion map and it is just a short walk across the border,no sea.

But either way the file structures would have to be understood,and headers converted.

And then all content from skyrim would have be converted to oblivion counterparts,otherwise we would be doing time for real without a lockpick,:)

 

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Its going to be hard work no matter what we do.

I was thinking it might be possible to do things in half measures. I might be able to get a program going that will copy all the variables of an instance (e.g, an NPC) thats open in the TES:4 construction set, and then punches them into the fields of a new instance in the Skyrim editor. I know its a long way around, and there are still thousands of instances to go through, but its something thats in my programming skill range (I might be able to learn how to work with the file types, but I've never had to decode an unknown structure before, someone momre experienced would have to help me).

I could probably automate it to create the original objects but not place the instances in a world-space just yet. I'd have to leave Dialog and AI packages behind. I guess it could run down the list of objects, and when an object references another object, e.g, an NPC is wearing a pair of shoes and some pants, I'll go and create the referenced object (the shoes and pants) and then come back to the top one (the NPC). or maybe just create a placeholder that gets overwritten when my program gets to the shoes/pants in the object list. Its still a lot of work though and I'll have to poke around the editors with Cheat-Engine or something similar to see if the automated option is possible. We'll just have to see whats possible when the Construction Set gets here.

Clint
Clint's picture
Member
Offline
Last seen: 3 years 4 months ago
Joined: 03/08/2011 - 04:00
Karma: 985
Morrowind in TES:V vs Skyrim in TES:IV

I agree with you, Raditz, that sounds like a plan.

I'd rather see Morrowind in the TES:V engine than oppose to Skyrim in the TES:IV engine. I've only gotten a little time to play Skyrim, but it has some awesome things that Oblivion doesn't have like realtime shadows and better character models, better combat, etc.

But I also agree with Ironman at this point, however. We don't even have a CS yet, so right now it's just speculation.

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
I have managed to get

I have managed to get syrim.esm converted to TES4 and then changed it to a .esp dependent on Oblivion.esm and it loads into the oblivion CS.

So I quess the next step [if I take skyrim into the oblion engine] is getting the main .bsa`s done too.But like others say the skyrim interface is good.

Personally I like the feel and look of the oblivion interface,skyrim seems a bit repetitive in the graphics department to me.

 About the story line,if after finishing oblivion the player could be jumped into the future [graphic or story mod needed] then it would cover up the time difference,but once into the skyrim world there would have to be no going back.

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
@ ironman

Wow, that was crazy fast, what exactly is working with this? I suppose the bsa's are where the meat is though. Still can you link some screenshots of what youve managed to play around with?

Also I've been tinkering witht he TES:4 construction set, there are definitly SOME things that I can rip from it while its running, but theres no garuntee ill be able to get them into the Skyrim editor. Also, like I said, its only some things, stuff thats in 'dialog box's' (in the programming user inerface sense, not dialog conversations from the editer, yet) maybe ill be able to dig depper into the other variables but im new to this sort of thing.

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Hang in there guys I

Hang in there guys I definately can`t put any screen shots here yet as at this time I am struggling with the nifs and the combat sound files are I think special format extentions, as are a lot of the files in the game.

Imo there ain`t going to be a snowballs chance in hell of putting together any form of converter for this baby.

At this stage converting the esm`s is easy but getting them and the bsa`s actually working in oblivion is going to be a real hard road.

And cutting to the chase the guy that eventually does it would have to be real brave to release any of it imo.

But getting the oblivion world into skyrim like Brutal said might be easier,but then everything in oblivion & maybe morroblivion would have be compatable with the skyrim engine and Creator Kit when [if] we get it.

 But morroblivion started with speculation,and look at it now...imo it rocks!.

 Just had a look at Cheat-Engine,it has loads of usefull assemble language tools and dis-assemblers.:)

But doing it by reverse engineering is ok only for the use of the person who does it,but it could never be released unless  that person liked bars [the steel one`s].

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Yeah I think this is gonna be

Yeah I think this is gonna be a tough cookie. And I'm not keen too keen on disassembly. And on top of that I'm almost no good at lower level things anyway. The things I mannaged to pull from the construction set were all things you could coppy/paste by hand, I've just made a way to do that faster, and write it to notepad...just wait for it...

Yep, Im such a bum, I've just writen a keyboard macro that tabs through everything, ctrl+v's it, alt+tabs out to notepad to paste it and then goes back to tab to the next thing it can copy. Although it would take a while to go through everything, not to mention it'd be unreliable, but I'm just going through some dumb raw (and not so naughty) ideas hopeing they'll spark something more sophisticated.

Hashmi1
Hashmi1's picture
Member
Offline
Last seen: 1 year 9 months ago
Joined: 03/10/2010 - 04:00
Karma: 970
@ Raditz: I am taking some

@ Raditz: I am taking some programming courses as well ... so hopefully I can help you out when the time comes ;)

I think that working at file-structure level would be better than using a keystroke macro ... 


I've worked with professional Macros quite a bit ... and:

- Macros are fairly slow: It took the ESPconverter 5-15 minutes on an average CPU to convert all of Morroblivion. On the other hand when I tried a macro to just randomise the faces of vvardenfell via the CS ... it took like 8-10 hours. 

- They need to be constantly monitored: otherwise small, inevitable fluctuations in the enviroment can mis-align an operation ... and a single mis-aligned operation arising somewhere in the middle of a batch can waste the whole remining batch (and the time alloted to it).

 - The oblivion CS is very prone to crashing randomly and frequently. When I used a macro on the CS much of the time wasted was on re-starting the CS and re-loading the esms.

Also we don't need to decode the file formats ... the community will eventually do that for us (Skyrim equivalents of tools like TES4EDIT) and we can modify their source code to do our work.

... 

Anyways this is all to early to discuss. The Creation Kit needs to be released and then the community will slowly learn about the new NIF format, the new animation format (havok behaviour), the new esp\esm format etc. Then they will slowly develop the tools like importerers\exportes for blender etc. These tools will slowly get debugged and the modding knowledge\techniques will slowly pile up. 

Probably for an year or two ... Nothing is happening, so might as well finish Morroblivion in the meantime.

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Yeah I'm itchin to get at

Yeah I'm itchin to get at that creation kit, for random mods and potentially for this. Maybe someone can help me understand the current esp format so I can at least get my head around it in the meantime.

http://www.uesp.net/wiki/Tes4Mod:Mod_File_Format

and

http://www.uesp.net/wiki/Tes4Mod:BSA_File_Format

I get whats going on in a general sense, I'm just not sure where to start. Also yeah the Macro's are clunky, just trying to scope out some ideas.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Yep, the CS and community

Yep, the CS and community tools will come with time. Plently of Morrowblivion and Skyrim to play in the meantime.

EDIT
Also Ive got my dirty lil hands on some saucey-code for reading TES4 esm/esp files. For the people who can program, whats your flavour? Iv'e got C++ and Java. I'm sifting through the Java atm, trying to pinpoint a few things.

empty
empty's picture
Member
Offline
Last seen: 5 years 8 months ago
Joined: 09/05/2011 - 05:24
Karma: 14
I have been having a look

I have been having a look through the ESM structure of Skyrim with Skyrim Viewer (Lojack's early work before we get some dev happening on Wyre Smash).

Skyrim is MUCH closer to Oblivion than Morrowind was to Oblivion. Skyrim uses a heavily modified FNV engine and still has a lot of FNV commands and structures in place (including PipboyRadio, VATS related things). It is not as everyone first speculated, a completely new engine.

Also it uses the same CELL and WEAP structures so far as I can tell and many of the GMST/GLOBs are all the same as Oblivion, almost.

Basicly, once Wyre Smash is out and stable we will have a Python API to interface with just like we did with Wyre Bash. I am planning on creating a converter as soon as any tool reaches a point of maturity and I know Lojack will get serious work done quickly however. I know Lojack loves Python and thankfully I have extensive industry experience with Python, also I have confirmation that Elminster will be getting TES5Edit done soon too, it's just a matter of time before the records are understood.

Once that work is done it's a relatively simple job of writing a converter really (as long as it is open source), the tough part is reading and writing to the formats and understanding the formats. If the CK comes out soon, we should have a good enough understanding of the new stuff shortly.

I am just wondering how this community will feel about testing and in part moving to Skyrim? I'm sure work on Morroblivion will continue and it can be converted if the tool is solid.

EnvyDeveloper
EnvyDeveloper's picture
Member
Offline
Last seen: 1 year 4 months ago
Joined: 05/02/2011 - 05:00
Karma: 505
ST is porting over their
ST is porting over their assets to Skyrim so it shouldn't be so difficult to port over the Nifs. I'm sure the quest system in oblivion has remained the same in skyrim. The reason why morroblivion was so difficult to set up was because there was a generation leap which was really huge. Of course it was difficult to port over morrowind to oblivion. An auto port converter is VERY likely to happen.
RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Nice to see this building

Nice to see this building momentum, I've been looking through an esm/esp reader/writer in java, but it dosnt seem complete. Ive also got some C libraries from the guy that made OB-Edit, which I havnt looked at in detail yet but they seem more promising.

It seems like Empty is ready to be on the ball when the community tools get going, which is great coz I'm still fumbling through Oblivions headers and flags and probably wont be able to accomplish much. Ill try to make whatever small progress I can on my own in the meantime, incase I can be of use later. (Python is over my head, I've only used it inside a few 3D modelling packages for simple stuff)

empty
empty's picture
Member
Offline
Last seen: 5 years 8 months ago
Joined: 09/05/2011 - 05:24
Karma: 14
Python is easier than C/C++

Python is easier than C/C++ RADITZ.

High-level language that is dynamicly typed, no memory management to worry about, simple threading, powerful string manipulation, type mixing is easy and elegant syntax. Also if you are familiar with OO, Python's classes are by far the best intuitive and simple to use.

Give it a shot, it's a powerful language that is quickly becoming the standard interpreted language and has applications from desktop to web to low-level automation. Check out Django if you have a web project and wxPython if you want to do a desktop app.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Yeah Ive heard its pretty

Yeah Ive heard its pretty good, I just havnt had a real chance to delve into it. I really hate C, maybe because its a language im slowly teaching myself (because all my programming friends use it), where as I have some uni education for Java. Speaking of which, my final Java exam for the year is tomorrow, so Id better stop stalking the forums for the night!

EnvyDeveloper
EnvyDeveloper's picture
Member
Offline
Last seen: 1 year 4 months ago
Joined: 05/02/2011 - 05:00
Karma: 505
How did the exam go, Raditz?
How did the exam go, Raditz?
RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Easiest exam ive ever done,

Easiest exam ive ever done, thankfully. I was worried it was going to be a killer because the subject has a high failure rate, but most people finished in half the time. Probably would have been much harder if wasnt exactly the same as the practise exam though.

Anyway, for Skyrim and conversion related things, what are the various bits that will need to be covered? Too early to bother thinking about it but I wanted something on-topic to contribute. So, theres:

-All the in-game objects (NPC's, Items, scenery etc)---Terrain---Sound files---Models/Animations---Dialog---TES Scripts---AI Packages---Pathgrids....uh weather?. Some of that stuff can probably be left behind, TES script should be similar to Oblivions and the community will make some way to convert the NIF's that we can hopefully plug into our own program, or just use independantly. Annyway I dont suspect that theres much more to discuss untill one of the things we need comes along, then we can get something underway if all goes well.

empty
empty's picture
Member
Offline
Last seen: 5 years 8 months ago
Joined: 09/05/2011 - 05:24
Karma: 14
Quote:if wasnt exactly the

Quote:

if wasnt exactly the same as the practise exam though.

How I love lazy convenors. If not the practise exam, then last years exam sample.

Seems people are already easily porting over NIFs to Skyrim without much trouble. So my worries about that being an issue are gone. All thats left to do is wait for the CK and something like Wyre Smash to come out and I can get started on a converter. So it's looking more and more likely but we won't know for sure the full implications till we get Wyre Smash and TES5Edit.

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Ive taken a look at Python,

Ive taken a look at Python, it seems like quite a 'loose' language compared to what im used to.

I've never done much involving files, if I were to write a program that reads a record from an esp, I'd open the esp in binary mode and scan for the records header, then read forward a certain abount of bytes for the first flag (assuming its a flag), assign the value to a variable in a class, then read forward a set amount of bytes again for the next flag or variable? Then do that untill I reach the end of the record. Of course id need to know how many bytes to read and what the flags represent ect.

Is that roughly how i'd go about it? I'd like to try something small just as a learning exercise.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Creation kit coming in January

Creation kit coming in January according to Bethesdas blog, looks like steam will support mod distribution or something. We'll finally be able to see how far we can push some of our ideas once we get it!

http://www.bethblog.com/2011/12/01/skyrim-what-were-working-on/

Also I'm attempting a simple program to read TES4 records form esp/esm files, I dont intend for it to lead on to a converter to TES5, as Wry Bash can already handle both formats, just doing it as a little experiment to see if it helps me understand things better. If i have any luck ill explain my findings.

Ice
Ice's picture
Member
Offline
Last seen: 2 years 7 months ago
Joined: 12/11/2012 - 15:30
Karma: 1036
Hey, you don't think it'd be

Hey, you don't think it'd be at all possible to do what they did for nifskope to the construction kit, do you?

I think it'd be near impossible, but it would be really nice if someone could come up with a code that let the oblivion construction kit act as a makeshift creation kit. (I'd be happy with any barebones functions it could achieve!) It sounds like that is sort of what you're getting at, but some of what you just said went clear over my head, lol.

Keep us posted.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
That would be way too

That would be way too complicated for me, plus by the time it was made we'd have the construction set already =p All i'm doing is just taking a simple oblivion esp and seeing if I can get a program that can read part of the data correctly. So pretty much if a 'Potion' record had a price of 38 and a weight of 0.5, I want my program to be able to find where the potion is stored in the esp amongst other records like Scrolls/Weapons etc, then find where price and weight are stored.

Its just a self teaching excercise, Ive never done file inputs and outputs in detail before so ill see what I can learn. So far ive managed to open the file and display all the raw data, which isnt a big deal I guess, but I can kind of see how the file is put together. There are a lot of random chunks of data that ill have to research how to interpret, not sure how far ill get.

I have seen some esp files for skyrim on Skyrim-Nexus oddly enough, so someone must be able to make simple mod files, no idea how they mannaged it.

Ice
Ice's picture
Member
Offline
Last seen: 2 years 7 months ago
Joined: 12/11/2012 - 15:30
Karma: 1036
Dang man, that's so cool to

Dang man, that's so cool to know what you know!

Okay, that makes sense about the cs. Figured I'd ask just in case.

I guess the people making esps are using FOMM and tweaking it so it can look at Skyrim, but its all code based, and it seems like too much work to do anything of real use, other than add an item here or there. Couldn't find the link to the discussion though.

Let me know when your able to read the matrix. lol jk

But ya, keep us updated! :D

Mooncat
Mooncat's picture
Member
Offline
Last seen: 5 years 6 months ago
Joined: 04/05/2011 - 07:53
Karma: 226
My understanding is that

My understanding is that they're basically doing what you're talking about. The most advanced mods I've seen so far add new effects to or modify perks.  There's also ScriptDragon, which runs alongside Skyrim and lets modders code things in C++. I think (not sure) that it works by letting programmers edit the game's memory directly, so it's not exactly useful for what we're trying to do, but it's still pretty cool.

Ice
Ice's picture
Member
Offline
Last seen: 2 years 7 months ago
Joined: 12/11/2012 - 15:30
Karma: 1036
Huh. Thanks Mooncat, that's

Huh. Thanks Mooncat, that's interesting. :)

So I've always used OBSE / SKSE. What's the difference betwee script dragon and those? Are they compatable?

I've heard of people running bat files through oblivion/skyrim. Man, the fact that a game is built to incorporate that sort of thing is just so cool.

Mooncat
Mooncat's picture
Member
Offline
Last seen: 5 years 6 months ago
Joined: 04/05/2011 - 07:53
Karma: 226
I think Script Extender hooks

I think Script Extender hooks into the .dlls to actually modify how Skyrim works at runtime. Script Dragon doesn't change the game itself, just makes changes to the memory used by Skyrim.  I could be wrong on this; I haven't actually used either, I've only read about them.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Hey guys, I've got some good

Hey guys, I've got some good news! Continuing on from my last post about Oblivions file structure, I can now open an esm/esp as a binary file, and have made the miracle step from being able to see a rough format to knowing how to interpret it 100%

In my spare time over the next week or so, Ill see if I can get a simple program going to search for the first potion record in a file and return the data, which will hopefully match up with the data for that potion shown in the construction set.

If I can get that to work I'll make a seperate thread and explain how to open a file and how to understand what all the data means =) I'm surprised I mannaged to accomplish what I have so far, seeing as ive had no experience (or knowledge, even) of file Inputs/Outputs untill now. There are some records (Combat Style, Effect Shader, Idle Animations, Race, Region, Road, Water Type) That I dont have any documentation for, so I wont be able to read those, but the only one that might be an issue is Region, the rest will already be handles in Skyrim so I can ignoor them all going well. One other thing is that some data is compressed, I dont know what I'll do when that pops up, but that'll be later down the road.

Just out of curiosity, who here has programming knowledge and how much? If your comfortable with Object Oriented stuff then hooray!

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
I have programmed in Object

I have programmed in Object Oriented C++/C,but these day they seem to be using the more Virtual aproach with piont & click Virtual C++ etc.

But assuming the job can be done,it will still mean that there will not be any quests [main or otherwise] in the ported code,and do we really want to have to rewrite everything.?

 If the scripting format is the same in the skyrim CK as it was in the oblivion CS then we might stand a chance of porting the scripts,but a lot of the dialog etc would still have to changed depending on regions and story lines.

Just my 2 cents.

 

Ice
Ice's picture
Member
Offline
Last seen: 2 years 7 months ago
Joined: 12/11/2012 - 15:30
Karma: 1036
 Sweet Raditz! And I

 

Sweet Raditz! And I basically know nothing of real coding. years ago I went through 50 pages of C++ for dummies, and I have a fair grasp of CS scripting, but nothing of real coding... I always like reading what people are doing and trying to understand it, though.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
I guess we'll wait on the

I guess we'll wait on the Creation Kit to see whats different in those key areas. I hope we get lucky and the in-editor scripts will be mostly the same. Moving from Morrowind to Oblivion was able to be done with a lot of scripts in tact, anything that dosnt compile in the new editor would have to be done by hand (lets pray that dosnt mean all of it). I'm a bit worried how world spaces and cells are gonna work out, we'll decide on what to do when we have the editor i suppose.

Clint
Clint's picture
Member
Offline
Last seen: 3 years 4 months ago
Joined: 03/08/2011 - 04:00
Karma: 985
I've taken in intro to C++

I've taken an intro to C++ class, but that class was a joke. We stopped right when we were getting to making objects and classes :~ Good news is though, I'm taking a full fledged C++ class next fall semester, and also a more in depth one following fall that, so I should be able to be of some help then.

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Cool! Robert!,a few tips if

Cool! Robert!,a few tips if you don`t mind.?

Oriented programing [programing is spelled with one "m" in the coding world because TV programming uses 2.[info only].

I find that formatting is the most important thing and also keeping it all neat and readable,indenting really matters and lots of remark lines [notes to yourself and others reading the code].

Always make variable names [static and global] readable for easy back checking and make sure that all functions return with an error if needed and that the variables,strings etc within each function are local as that way the function is self contained and easy to check.

Just keep checking the code [program] is running to plan before ending a session,then add more each time untill the whole thing is cool enough to compile or build into an assembly lanquage final exe.

Hope you don`t mind an old man running off at the mouth.:)

 

Clint
Clint's picture
Member
Offline
Last seen: 3 years 4 months ago
Joined: 03/08/2011 - 04:00
Karma: 985
Ah, I don't mind at all

Ah, I don't mind at all anyone speaking their mind. You probably have experience with Fortan and Pascal?

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Nope! no fortran or

Nope! no fortran or pascal,but I quess they are basically the same as C/C++ except I think they were a compiled language a bit like the old microsoft DOS basic.

Years ago we just used raw machine code binary tools to program directly,but then one mistake and you had to sift through reams of 1s & Os to find the fault,then we used hex editors which actually I find usefull even today.

But imo a compiled program will alway run slower than an properly assembled flatform independent one.

 You proberly know this already,but the difference between a compiled program and an assembled binary one is that when a compiled program is used the main functions etc have be on the system as they are read every time they are needed from the files [slows it all down].

But a machine coded exe resides in memory and is read from there in 1s & Os [real fast but less forgiving,:)].

My fallout 3 keeps going into slow mode every time any action involving the BOS and certain robots etc are on screen,I think that is because the program is trying to read compiled scripts and getting it`s panties in a bind.

 

 

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Ironman, if you have any idea

Ironman, if you have any idea of how to deal with large files in memory it might come in handy, I'm not sure if it'll be an issue though, just thinking ahead. I'm inexperienced so I dont know if executing code that runs through a >200mb file will cause problems or just 'stop responding' untill the process is done. Or just flat out stop responding. I guess theres only 1 way to find out though, so I'll start roughing out a program now and if it works on smaller esm's we'll see if it can muscle through some larger ones.

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Radltz,Imo you should have np

Radltz,

Imo you should have np running through >200MB,but you will need to write some buffering code into the program,and you might need to have a lot of RAM in your system,but 4GB RAM should keep it going.

The problem with some games CTDs and slow downs is because system RAM just locks up and virtual memory then needs to do it all.But thats just my 2 cents worth.

 

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Yeah ive tested some things

Yeah ive tested some things with Oblivion.esm and it seems to be fine. So far my little program returns the data expected when useing a test esp I created, but Oblivion.esm isnt co-operating... ill keep digging though.

EDIT: Possibly figgured it out 2 minutes later. I was searching for 'GRUP' (record header) then reading and decoding the next 16 bytes, which is the expected follow up length. Problem I was getting was the 1st instance of GRUP in Oblivion.esm occurs in a place where it isnt acting as a header for a record, so my decoding just returned nonsense. Hopefully thats all the problem was and ill be able to move past it, and the real records will behave themselves!

ironman12345
ironman12345's picture
Member
Offline
Last seen: 5 years 1 month ago
Joined: 08/29/2011 - 18:29
Karma: 1557
Are you just dumping the data

Are you just dumping the data or can you edit.?

If you know where that GRUP is located in the code after or during the program run,could you edit it to point to it`s proper GRUP if you can find it.?Or make it a dummy GRUP just to satisfy the code.

 

 

Mooncat
Mooncat's picture
Member
Offline
Last seen: 5 years 6 months ago
Joined: 04/05/2011 - 07:53
Karma: 226
Just found this

Just found this beauty:

http://forums.bethsoft.com/index.php?/topic/1312070-rel-skye...

Should help give some insight into how Skyrim's plugin files are set up.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Good find Mooncat, I'll check

Good find Mooncat, I'll check out the source and hopefully there'll be something usefull to be learnt. Projects like these will be even more usefull if we ever get round to tinkering with skyrim.

@Ironman, I'm just grabbing data for now. I could probably edit values for some existing records, but the public documentation on the file structure has some frustrating holes in it. If I can figgure out how to write a propper version of a mod file's main header it would be a huge step. If I were able to do that accuratley, I think I'd be able to create mods involving the records that have full documentation, without going in the Construction set at all.

EDIT: Seems like some of the parts in the main header that I'm having trouble with are not required, if thats true then I may be able to write an esp/esm that dosnt include them, and still have it function like a proper mod. There are still parts in other records that I may not be so lucky with, however.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Seeing as this thread has

Seeing as this thread has bumped to the top of the list again I'll just update everyone with my progress since last time, which is absolutely zero due to christmas, I'll get back to buisness a few days after new years, all going well. Maybe we'll have the creation kit to play with by then too :)

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Also I jot off my ass and

Also I jot off my ass and hammered out a rough user interface because it was getting tough to make any progress with just the console. So far the program makes a list of top level groups (which have a GRUP type of 0 ). Double clicking on anything in the list adds its info to the display in the middle, for groups this is how many bytes into the file the group starts, how many bytes long the group is (including its records and any sub-groups), its type (Top, Child etc) and its date stamp. I'm not sure how to interpret the date stamp, but its not important for now. The program can read any group or record, but I havnt worked that functionality into the interface just yet.

Here's a screenshot of it in progress, hopefully it resizes properly and dosnt break the forum

Ormin
Ormin's picture
Contributor
Offline
Last seen: 1 year 6 months ago
Joined: 02/03/2012 - 13:17
Karma: 114
I just registered to make

I just registered to make this comment, whew.

I am an experienced assembler reverser and would like to contribute to the project. I'd like to ask about some help, specifically about the Oblivion's .esm / .esp 's , so I could compare it with Skyrim's .esms' and .esps' and have a contribution in all of this.

Any help appreciated. Thanks :)

 

thermador
thermador's picture
Administrator
Offline
Last seen: 2 weeks 2 days ago
Joined: 12/16/2010 - 04:00
Karma: 1397
Donor
What would you like to know

What would you like to know exactly?

Part of building an Oblivion to Skyrim converter is knowing how to read Oblivion's ESP and ESM format.

The best tool for reading Oblivion's format is probably TES4Edit .  Apparently the author will release SOME of the source code, as revealed here, but not the GUI.

Then there is TES4Gecko, which I am not sure if they release a source code for, but you could ask.  There is also TESsnip, which is pretty much abandoned, but you might be able to find the source somewhere.  Then there is OBedit / TES4Lib, which is very old and abandoned and also TES4Plugin, the precursor and source code to TES4Gecko.  Apparently the source for TES4Plugin used to be freely available, but all the links I can find are broken (they referred to Bethesda's old forum).

You may also find this information useful, particularly the section on toolmakers.

For reading Skyrim files, there is TESVSnip, which is amazingly open source... I don't know what other projects are in the works for reading/writing Skyrim files.

Ormin
Ormin's picture
Contributor
Offline
Last seen: 1 year 6 months ago
Joined: 02/03/2012 - 13:17
Karma: 114
 Thanks, that will be helpful

 

Thanks, that will be helpful :) Will tell ya if I'll manage to find out something


@Edit

TES4Gecko is java, not hard to dump it's source code. I am already looking at it's reading schemes, allowing me to learn something about it :) Then I'll try to convert it to skyrim working one :)

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Ill try to throw together a

Ill try to throw together a tutorial for what ive learnt about the esm structure based on the documentation avaliable online and how I went about accessing it. Bear in mind that I could have made a wrong turn at any stage, everything I attempted with this project was new to me =) Hopefully that be some help to Ormin and anyone else who wants to take a crack at the files. Ill report back later today, all going well.

RADlTZ
RADlTZ's picture
Member
Offline
Last seen: 7 years 3 weeks ago
Joined: 11/21/2010 - 22:23
Karma: 74
Here's something I hope will

Here's something I hope will help those with programming knowledge get a bit of an idea of what we're working with. --> http://i42.tinypic.com/288p40k.png If theres any feedback I'll be happy to explain what I can in more detail, or explain something I didnt go over in the pic.

Ormin, Oblivion and Skyrim esm/esp formats are quite similar, check out

http://www.uesp.net/wiki/Tes4Mod:Mod_File_Format for Oblivion

http://www.uesp.net/wiki/Tes5Mod:Mod_File_Format for Skyrim

The Skyrim page has filled out a tone since I last checked it.

Ormin
Ormin's picture
Contributor
Offline
Last seen: 1 year 6 months ago
Joined: 02/03/2012 - 13:17
Karma: 114
Now we should focus on

Now we should focus on converting Morroblivion objects into Skyrim Objects, so we could put all the landmass in.

 

 

Topic locked