CHRONO CROSS FILE EXPLORATION THREAD

[FaustWolf]

Awright, we've got an entry for character battle models now. Check it on this encyclopedia page: http://www.chronocompendium.com/Term/Chrono_Cross_File_Structure.html

Feedback is welcome, especially error corrections and stylistic suggestions.


Now for some more Youtube vids showing screwed up character models. I'm going subsection-by-subsection within the first section (arg, that's confusing to type) of Serge's character battle model. I shall now use the nomenclature Section 1-1, Section 1-2, etc., to refer to the subsections within...the first section.

Section 1-1: Offset Range 0x48 ~ 357 (Subheader from 0x20 ~ 0x47 remains unchanged)
I don't actually have a vid for this yet, but I hope to in the future. A study of the first subsection is going to be tedious, because I can't simply overwrite Serge's data with another character's without the game freezing upon trying to load the model. My emulator tells me it's being forced to execute "illegal opcodes" as illustrated here: http://img159.imageshack.us/img159/9781/illegalopyv6.gif

I've previously changed a few bytes at a time within this data range without such consequences. I'll go through byte-by-byte sometime and make sure I'm keeping the "structure" intact, i.e., columns of 00 bytes may have gotten shifted this time around and maybe that's what's causing the trouble. Advice is welcome, 'cause I don't know what the heck's going on with this yet.

EDIT: The emulator is reporting an illegal opcode starting with value "2C" in one instance. Halkun, could this be a textured quad opcode!? Maybe I can find it in the model data after all, and somewhere within this section...


Section 1-2: Offset Range 0x358 ~ 28E7
http://www.youtube.com/watch?v=is5YdHgvJHg

Now, THIS is intensely interesting to me. Once again I've taken a section of Kid's model data and written it over Serge's, producing this mess of "shadow spikes," as I like to call them. Can anyone determine from this vid what type of data is being affected?

Also, I'd like to note that when Section 1-2 is messed with in such a way, I can no longer get Serge to perform a "fierce" (i.e., 3-stamina point) regular attack, nor can I get him to do his "Dash&Slash" special without the game crashing. He can do his other regular attacks and can cast other spells. So does this mean I've touched upon some animation data, or is the emulator just flipping out because it can't handle these animations with all the shadow spikes sticking out of Serge, I wonder?


Section 1-3: Offset Range 0x28E8 ~ 4247
http://www.youtube.com/watch?v=yrJQ5r0rSaE

Once again, data from the equivalent subsection of Kid's battle model has been pasted over Serge's. I think I see one "shadow spike" here, which is odd -- I would have thought all those were accounted for back in Section 1-2. But other than that, anyone know what kind of model data has been affected? We had taken to calling this subsection the "vertex pool." Do the results seem to jive with that label?


Section 1-4: Offset Range 0x4248 ~ 4EF7
http://www.youtube.com/watch?v=QIPrAZ2Vyas

This time I lifted the data from Guile's battle model and pasted it over the equivalent subsection in Serge's model. BUT - NO CHANGE!? I've re-checked time and again, and I've surely successfully overwritten Section 1, Subsection 4 of Serge's model with data from Guile's. Earlier Luminaire and I were hypothesizing that this data may correspond to color values or perhaps shader values. I can't see any difference between this and Serge's unaltered battle model, which I link to below:
http://www.youtube.com/watch?v=snlWXhjPYM8

The data change was significant, yet no visible alteration of the model. What gives?


That's enough for now, I think. I'd like to get feedback on the results for Sections 1-2 and 1-3 especially. For those who live in the US, or any other country that happens to worship turkeys this Thursday, have a happy Turkey Day!
 

[Cyberman]

Once again, data from the equivalent subsection of Kid's battle model has been pasted over Serge's. I think I see one "shadow spike" here, which is odd -- I would have thought all those were accounted for back in Section 1-2. But other than that, anyone know what kind of model data has been affected? We had taken to calling this subsection the "vertex pool." Do the results seem to jive with that label?

It looks like it is so. The thing to keep in mind is the ordering of the vertex information is VERY specific with the Ps1 it is LEFT handed IE A B C are counter clock wise and the normal projects in the direction of the thumb from your left hand. (Make a fist partially relax your left hand and poke your thumb straight up and you get the idea right away.) That is only the triangle. HOWEVER the Quadric is a bit different since internally it is split into 2 triangles by the hardware. The ordering for a quad will be A B D C instead because of this.
See diagram I drew about 4 years ago here

Code: [Select]

Triangle Quad
B--C    B--D
|  /    |  |
| /     |  |
|/      |  |
A       A--CWhat you have done is got the vertex references out of order and you notice a number of triangles missing (because they are invisible from the direction of there normal surface likely).

Your actual vertex data is NOT changing. Remember they aren't Vertex's they are references to vertexs (IE points in the model). Change the references your vertexs don't move just there ordering.  So this confirms they aren't vertex but references to vertexs.

Section 1-2 is likely the actual vertex information. Namely you killed the dimensional characteristics of the model when you messed with it.  The next thing to find is how this information is held together. I'm sure there is information on what blobs of polygon data are what.  (IE Section 1-3 is possibly your triangle data and 1-4 is likely your quadric attempt to add something into the fourth column in 1-3 in some of the data and see if anything at all happens if nothing then it's .. triangle data only and 4 is likely quadric).

Cyb

[FaustWolf]

More wrinkles in my brain, Cyb! Now, in section 1-3 (vertex pool references), the data are arranged like so when viewed with a hex editor:

xx xx xx ** xx xx 01 yy - xx xx xx ** xx xx 02 yy
xx xx xx ** xx xx 03 yy - xx xx xx ** xx xx 04 yy

...with yy starting at 00 and progressing to 01 and 02 until the data reaches section 1-4. The ** are either 00 or FF. Here's a snip of the actual data (you've probably seen it already in this thread):
http://img230.imageshack.us/img230/9238/successionpatternth7.gif

So I'm wondering what the best way is to treat the ** and yy columns when I change the values. I could just change the values to something arbitrary, like make them all 0x50 I guess.

Section 1-4 is arranged like so:
xx xx xx 00 xx xx xx 00 - xx xx xx 00 xx xx xx 00
xx xx xx 00 xx xx xx 00 - xx xx xx 00 xx xx xx 00

And a true representation: http://img516.imageshack.us/img516/6649/section14samplenw8.gif
...so every fourth byte is 00 there at least, if that means anything.


EDIT: I've just done two experiments with Section 1-4 -- first writing some 0xCC bytes over every fourth column, and secondly wiping Section 1-4 entirely (replacing all data with 0x00 byes). NO CHANGE. Zip. Zilch. Zero. Serge is completely intact in appearance and animations. What could it mean? Is Section 1-4 "extraneous"?

[Cyberman]

No clue. I did forget about the vertex numbering (the brain leaks you know). I think we can assume that Serge et al are likely made entirely of triangles (that would be simple to do).  Dumb question did you view the textures on Serge? IE did they move any when you messed with section 4?

Cyb

[yaz0r]

Hey there,
a few years ago, when my nick was yazoo (it changed a bit since as you may have noticed), I wrote some of the tools you've been using to hack into CC files.
It's been a long time for me, but maybe I can give you a few pointers on where to look and how things work in the game.

Also, some of you may have come to realise that since my work on CC, I wrote a bunch of model viewer for FFX/FFX-2/KH/KH2/FFXII/SOTC/... so maybe I can also help you on that side.

yaz0r

[FaustWolf]

OH.MY.GOD!! 

This is another joyous day for the Compendium. Yaz0r, I had no idea you and Yazoo were the same Entity! First, let me give you a big THANKS on behalf of videogame fans and asset enthusiasts everywhere for the work you've done, both on CC and outside of CC.

There are SO many questions I'd like to ask you. I'll start with these:

1.) First, I guess I'd like your input on whether we're going about this the right way, from what you've seen of our investigations -- is there any more "powerful" way to figure out how a model format is set up other than identifying headers/pointer tables and then messing with the data to see what happens? What basic process(es) do you typically use?

2.) If our goal is to extract models and view them outside of the game, would it be best/easier/more efficient to write a standalone model viewer for CC, or to write a plugin for 3DS Max/Blender/Lightwave, etc.?

3.) What programming language is most, er, "effective" for writing model viewers? I believe you wrote at least one later version of your FFX model viewer in DirectX (if DirectX is even a language -- please forgive me for my inexperience in that regard). I'm so incredibly interested in videogame models that I'd be willing to plunk down some time and learn a programming language over the next few years just for the ability to do what you do.

Thanks for walking in here yaz0r, and thanks for any advice you can give this newbie hacker!

@Cyberman: No change in Serge's texture when I filled Section 1-4 with 00 bytes -- it appears to be assembled properly on the model at least. It's extremely odd.

[yaz0r]

First, let me give you a big THANKS on behalf of videogame fans and asset enthusiasts everywhere for the work you've done, both on CC and outside of CC.

I have a few stuff that I decided not to release, like the SOTC model viewer, but I hope it will make it to the public hands someday...

Here goes for the questions:
1.) I can't realy say before I look into the model files myself. I'll have to redump the data files tonight when I get home. I don't have much expertise on PS1 games so I can't really tell, but I suspect that looking how the PS1 handle vertice format and things like that would be interesting.
I don't really have any predefined process when looking into a game, but most of the time, I try to have simple objects work before I try to get animated objets working. This is mostly a question of getting the basic geometric data figured out before starting the skinning.

2.) Writting a model viewer from scratch can be a tricky task. One solution would be to write a tool that would export the data to a more universal format. I suggest you look into the collada format as it's open and widely accepted.

3.) My model viewer is written in C++, and use DirectX for 3d rendering. While C++ is nearly a must-have for that kind of tool, the question of DirectX versus OpenGL is mostly a question of taste. I ditched the OpenGL version mostly to stop supporting old hardware as I wanted my modelviewer to be a testbed for more advanced graphic features. But this is behond the scope of a model viewer.
In term of learning curve, I figure OpenGL is simpler the get started. I suggest you take a look at nehe's tutorial (just google nehe). But keep in mind that learnign programming isn't something you do overnight...

[Luminaire85]

In regards to 2, I think it will be better (and probably easier) to import the data into 3D modeling environments than writing a standalone viewer. Blender makes this very easy since it provides facilities for writing custom scripts in Python. This way all we have to worry about is finding the data and adjusting it to the right format, and once the data is imported we can export it in other formats.

In regards to 3, like yaz0r I do all of my 3D/game work in C++. However, I use OpenGL (via the Simple Directmedia Layer (SDL)) because I try to write only cross-platform code.

---------

Section 1-2 has been bugging me for a while now because I think it contains important information but I have had no luck in deciphering the hex code. That section is 9616 bytes long, or 601 16-byte sequences, but the hex code doesn't seem to follow that sort of pattern. I am starting to wonder (perhaps incorrectly) if it is compressed data.

[FaustWolf]

Yaz0r, I've attached Serge's battle model and Kid's battle model to this post, so hopefully that might save you some work. There are also "ModelMines" consisting of the battle models and all the textures applied to the battle and weapon models, ripped straight from Disc 1, so you can take a glance at how the models and associated weapons are arranged on the CD. I think your dumping utility might extract these things as separate OUTs in the MISC folders.

For the battle models w/o accompanying textures, everything we know so far is recorded here:
http://www.chronocompendium.com/Term/Chrono_Cross_File_Structure.html#Character_Battle_Models

Thanks for all the advice on coding and model viewing, guys! Now I know which things I'll need to look into to make this sort of thing a long-term hobby.

Luminaire, I think it was Section 1-3 that you managed to view in Blender earlier - have you tried loading Section 1-2 as raw data? Cyb believes it may be related to vertices as well, and I was wondering if anything would show up.

[attachment deleted by admin]

[ZeaLitY]

Yeah, big welcome from all of us. Exploring Cross with your tools has been an adventure, and we can't wait until we've documented everything.

Other than file sharing, our other big "goal" for the short-term is figuring out how Chrono Cross instructs the room GFX in the .raw files to set up. Then, Ramsus or someone else can write a program to assemble the GFX files like that and combine them with the ACT files that your tools provide, giving us instant background images for the encyclopedia.

Anyhow, thanks for stopping by! It's a great time for Cross fans now.

[yaz0r]

Other than file sharing, our other big "goal" for the short-term is figuring out how Chrono Cross instructs the room GFX in the .raw files to set up. Then, Ramsus or someone else can write a program to assemble the GFX files like that and combine them with the ACT files that your tools provide, giving us instant background images for the encyclopedia.

Regarding that part, I know someone who made a tool to do something similar with Parasite Eve. I think it's really close to the way CC store it's background so I'll be asking him for pointers..

[Luminaire85]

I attached a picture of the Section 1-2 data, read as if each 6-byte set represented a vertex. I chose that because I thought I saw a vague pattern to that end in the hex, but as you can see, the picture does not look anything like Serge.

[attachment deleted by admin]

[FaustWolf]

Thanks for checking that out Luminaire. I'm going to change Sections 1-2 and 1-3 simultaneously this time (from Serge's data to Kid's) and see what happens. It may be that there's some direct relationship between these subsections, which may become apparent when we view the results. I'll report back later and post a vid if it's anything worth seeing.

EDIT: Well, this is weird.

I overwrote Sections 1-2 and 1-3, additionally changing the subsection start values in Serge's first section subheader to match Kid's. Nothing important enough to make a new Youtube vid over (the changes aren't extraordinary), but I did notice something that may be useful. Specifically, I believe the "shadow spikes" are in fact stretched textures. BUT - there's a complicating factor. See for yourselves (these are JPEGs to preserve quality - 56k users beware):

http://img514.imageshack.us/img514/6243/sergetexhs8.jpg
A section of Serge's texture apparently stretched away from his model upon a "shadow spike."

http://img410.imageshack.us/img410/9002/sergefacetexqf5.jpg
Another.

http://img410.imageshack.us/img410/8236/glenntexxr8.jpg
This is where things get really odd. This is part of Glenn's texture jutting out of Serge's battle model! However, notice that the texture is discolored because it seems to be filtered through Serge's CLUT.

Note that Glenn's texture isn't jutting out of Serge's model when Glenn isn't in the party; another character's texture takes its place. Sooo -- is it possible that messing up Section 1-2 causes the emulator to barf up some of what's in VRAM?

Just an oddity for people's consideration. It probably doesn't help us whatsoever, but wanted to report it.

[Cyberman]

It's not odd.
The UV coordinates AND the texture page are all used to generate the data. So if things are over written correctly you will get strange texture information in the polygons being shown.
Further more I think this just reinforces that this is potentially UV data (so 6 bytes per UV (1 byte inside each page) if you dump the video memory It's possible to view what page is where. Just moving the page information shifts what data is being used, with a specific palette. 

Read up on it in PSX.PDF.  Part of the puzzle is really in there.

The GPU opcodes appear to be assembled and sent via DMA to the GPU from the model data you have thus far.  How they are assembled currently is a bit of a mystery. To me sections 1-2 and 1-3 are likely vertex indexs and UV information (If I was following all the nomenclature being used).

You may want to look through some of the data as signed short ints (2 byte integers -32768 to 32767) 1 byte unsigned (0-255) word etc. See what you might be looking at. The fact one could take some of the data and construct an outline of serge from it is a definite hint it is THE vertex data.  The data that showed a snow flake pattern is likely vertex information.  What is left is UV Information and texture page information. 

Cyb

[FaustWolf]

Thanks Cyb! So judging from Halkun's PSX doc, each GPU opcode for textured polygons has associated texture page values that tell where in the frame buffer (VRAM?) the model's texture is supposed to be pulled from, is that right? If so, then it would appear that I tripped over some texture page data, since part of another character's texture is apparently being applied to Serge's model in the final pic I posted just above.

Which means, the GPU opcodes for UV maps may be in Section 1-2 after all, maybe. Hmm. I'm going to have a closer look at that subsection then.

...Which actually leads me to another question. When I look at Section 1-2 of Serge's model data, I'll be comparing everything with tables that appear as follows in Halkun's doc:


The numbers running along the top of the table, from 0 through 31 -- do these represent bits? Hence, the 0x34 opcode is 8 bits, or one byte, "wide." Therefore, the "CLUT ID" element for each vertex would be 2 bytes "wide." It's probably a silly question, but I wanted to make sure I'm understanding these tables correctly before I delve in for another look.