CHRONO CROSS FILE EXPLORATION THREAD

[Luminaire85]

That's awesome that you've found model data.

A cursory search of Sourceforge gives CamStudio and Taksi, among others. The former seems to be more mature, while the latter explicitly mentions being able to capture DirectX/OpenGL. I don't really know if either of these are good, but they might be worth a try.

[FaustWolf]

CamStudio looks good. Thanks Luminaire! I'll get some video clips upped tonight on Youtube, along with some other helpful visuals to give everyone a good idea of the effects of the model data transfer.

[ZeaLitY]

Wow, what an exciting development! I'm not on the forefront of the model work, but...what's the anatomy of a file once you remove the textures? Are all the different-purpose portions and components preceded by a header, like the TIMs in the battle field OUTs I examined earlier?

Yeah, the hacking idea is enticing. Can't wait for tonight. For a rough count of how many individual models we might be dealing with...

http://www.chronocompendium.com/Term/Monsters_%28Chrono_Cross%29.html: 152
http://www.chronocompendium.com/Term/Characters_%28Chrono_Cross%29.html: 106

258 plus a few more for general NPCs and other stuff like fish. But speaking of that, fish and other details of battle models (for instance, the Lizard Rock battle field contains fish) might be contained in its actual /BF file instead of MISC/ with all the character models.

[FaustWolf]

Thanks for the battle model count, Zeality. That will come in handy when I begin in-depth mapping of the battle model data. As far as the file structure of models once textures are removed, I haven't got any info that's more in-depth than the skeletal map I posted earlier (for "ModelMine1"). Now, however, all the "Uknown Data" can be referred to as "Model Data."

Here's tonight's update. The vid quality is less than I would have liked, but hopefully we can find some insight within its grainy-ness. No sound either; I'll report later as to whether the "Chimera" uses Kid's or Serge's sounds in battle, because I've forgotten.

The first video is just a demonstration of how the characters Serge and Kid look in battle. Folks already familiar with Chrono Cross battle models can probably skip to the second vid, but this provides a useful reference:
http://www.youtube.com/watch?v=uxts0pisnI0

Now the fun begins. As boring as it will be at times, please watch this in its entirety to get a full impression of what's going on with the model data. The final 30 seconds are especially important:
http://www.youtube.com/watch?v=YSHIT-thL5Q

I apologize for the expletive in the second video's file name. Youtube's web address generator has a filthy mouth.

As you can see, the "Chimera" is Serge's model data with Kid's battle texture. Serge's standing, running, and spell-casting animations have carried over, but not his physical attack animations for some reason. But a complicating factor is the significant difference in file size between Serge's and Kid's model data. Specifically (and surprisingly), Kid's model takes up 8184 bytes more in the game CD than Serge's! It could be that Kid's animations are more complex than Serge's or something, and therefore need more space.

I'll illustrate graphically what I did exactly. Below is a representation of Serge's battle model data (sans skin & eye blink textures). In ModelMine1, the unskinned model lasts from hex offsets 00009800 ~ 00014848:


The following is a representation of Kid's unskinned battle model data. I will release her battle data set - ModelMine2 - tomorrow evening, once I jot down some datamapping notes on it. Then we can find potential model file headers.


Now, with Serge's model data written over Kid's:


So some of Kid's model data is still left over. At some point I'll wipe the rest of the "red" (Kid) part out with 00 bytes and see how that changes things.

At least this shows that we've got battle models. Now, the question is how to go about deciphering the model format.

Frivolous Edit: Youtube says these vids are related to Area 51 and Illuminati videos. But I assure you all, the Chimera vid is the real deal.

[Molotov]

So -- does anyone know of any really good open source/freeware/shareware video capturing programs I could use?

I'll be honest: this topic confuses me and that's part of the reason I don't follow it all that closely. The other reason is perhaps that nobody has found sequence breaks because of this stuff.

Anyway, if you're using ePSXe or something with plugins, use Pete's Soft GPU 1.17 for video capture. No sound, but it can record without any watermarks. That way, you can record to Huffyuv and compress it to something good later. Your choice though, obviously.

If I'm saying something not even remotely related, I'm sorry. Blame the lack of attention.

[Blackcaped_imp]

OMG!! Omnipresent shadow attacking in the second video!!! Damn Japanese people LOL!!!

[Cyberman]

At least this shows that we've got battle models. Now, the question is how to go about deciphering the model format.

Frivolous Edit: Youtube says these vids are related to Area 51 and Illuminati videos. But I assure you all, the Chimera vid is the real deal.

Well FF7's stuff was a bit easier however I suggest you look up skeletal animation information and begin learning how it works. This way you will understand what data you are looking for and how to use it.
You need vertexes, you don't need uv coordinates since those are mixed in with the GPU data.  If the animation moves the bones and the vertexes are 'weighted' to those bones then you need an idea of how they moved the vertexes with the bones (in simple terms I suppose).
Anyhow it will help you to learn about data like that which you look for!

Gamed Dev Net To your rescue however.
First Look at this article and then at this article. The first part of Article one is how FF7 handles things. The second part is similar to how CC likely handles things.

Cyb

[FaustWolf]

I wonder if the folks who deciphered Final Fantasy 8 and Final Fantasy 9's model formats left documentation anywhere as to how they went about it. That would be critically useful, seeing as how FF8 and FF9 both used texture models.

I think either Zande or Zidane 2 (maybe both) figured out how to view FF9 models. Cyb or Halkun, do you have any idea who figured out the FF8 format? Was it anyone at Qhimm?

Ooh, I just saw the articles you posted Cyb. Thanks!

@Molotov: Thanks for the vidcapture plugin tip. I'll try that if I'm ever in need of less-grainy videos (or vids with sound perhaps).

[Cyberman]

I wonder if the folks who deciphered Final Fantasy 8 and Final Fantasy 9's model formats left documentation anywhere as to how they went about it. That would be critically useful, seeing as how FF8 and FF9 both used texture models.

I think either Zande or Zidane 2 (maybe both) figured out how to view FF9 models. Cyb, do you have any idea who figured out the FF8 format? Was it anyone at Qhimm?

More specifically it was Qhimm who did it. Search is your friend. There are a few threads regarding that.
I suppose I must break down and actually compile PCSX and then beat on it's GTE engine. Since none has been willing to add the information to the wiki (I tried but all things noted I don't know anything about FF8's format although I didn't meddle with it some experimentally mostly).

My guess is CC is similar to FF8 and FF9 in how they are structured.

Now FF7 used the format they did (see article one part one) because it compressed well.  FF8 is the way it is because they realized they only needed certain things for each data set and disk.  Unlike FF7 which had a lot of redundant information (it was hastily put together to say the least).

My guess is they used bone deformation and weighting of selected vertex for transformation.  IE most of the parts were transformed normal relative to the bone they were associated with and ones that had more than one bone association were stretched accordingly. This is easy to say.  Well a hint about vertex information on the PS1 it's likely stored as 4 16 bit signed values in a row. So if you see a lot of 8 byte values with 0000 as the last word. Likely that's your vertex information.  To be honest you should start there (find that data first) then it's likely index sets are the next victim.

3d data has to be organized in some way to transform it for the polygon information.

A simple explanation:
On the PS1 3d data is set up in memory for the model. The GTE then transforms the 3d data as specified by the animation information. That data is then married to the GPU opcodes you were looking for.  The GPU opcodes contain the UV information with them so the only information the GTE needs to provide is the x0, y0 x1, y1 x2, y2 information after it performs the perspective transformation on the data (this is a GTE function if you look at the functions the GTE performs lighting and perspective transformation is one of it's duties).  The data then just needs to be sent to the GPU. This is repeated for each model in the display as well as the background information.  Granted they likely do depth sorting etc. on all that data as well.

Cyb

[FaustWolf]

Thank you very much for that guidance, Cyb! Vertex data first, then. And I'll read up on the documentation you provided.

I think Halkun had stated earlier back at the Qhimm forums that the model data may be a binary form of RSD or HRC format for ease-of-use by the PlayStation hardware, so I'll be on the lookout for characteristics of those as well.

I was under the impression Qhimm reverse-engineered the original battlefield model viewer for FF8 and improved upon it so that it viewed all the textures correctly and such. I didn't find anything about a release of the improved program though, and the only download link I can find for the original gives me a corrupted rar. But my guess would be that the FF9 model format is probably closer to Chrono Cross'.

[halkun]

I found the UV data, Qhimm put the model togeather. There have been 2 runs at creating a model viewer. Each time the models initally render "wrong" due to some kind of error. (The polys are all there, but not in the right place)

I'm not sure that CC uses weighted vertexes, that technology didn't show up till tekken tag, I think. Let me show you what weighted vertexes are....



This is a picture of a model I made. You see the colors in the dress? These are the "weights" that are assigned to the lower leg bone that is moved behind the model. The colors go from red to blue. The more red it is, the more influance the bone movement has on the surrounding vertexes. Because Kaori wears a dress here, I want to smoothly move the dress, but have parts "stick" with the bone.

When looking for the model data format, it is most likely going to be orginazed into a few sections with lookup tables in the headers. Psy-Q wrote out it's data in text format that was "compiled" into biniary data for the lib to understand in run-time. The big break for us was that FF7PC has the RSD files in an uncompiled text state so we could read them with standared text editors. we could then see how the models broke down.

Psy-q has the ability to save animation data per vertex, but not weights like my model above. If the model is in any way standard, there should be a kind of RSD discriptor at the beginning that defines how the model is put togeather.

Inside the RSD data it should point to other data types, depending on if the extended libiary was used.

Refrences inside RSD files [ReSource Data]
-PLY [PoLyGon data] (.p in FF7pc) defines the shape of an object in the model. (Head, Neck, Torso, l_arm)
-MAT [MATerial data] (TIMs) These reference the textures used on the particular PLY above
-GRP [GRouPing data] These discribe how various PLYs are grouped togeather. (Character Models vs. weapons mostly)

The other types are MSH [Mesh], PVT [Pivit], COD [coordinate], MOT [animation], and OGP [vertex groups]. I don't know much about these types as I've never ran across them. These are used on the per-vertex models and may need to be deciphered for CC.

Keep in mind an RSD file does not contain any data, but only points to other datasets, you may need to seperate out the refrenced data so you can get a clear idea of what section is where.

THE DATA WILL BE SECTIONED INTO DECRETE DATA BLOCKS AND NOT BE MIXED.

I have yet to see a model that didn't follow that rule above. Look for headers and lookup tabes that go upwards in value. That will lend a clue.

[Luminaire85]

I am thinking the first 32 bytes of the Serge battle model file are a header. Specifically it looks like a pointer table:

Offset	Value	Comments
0x0000	0x00000006	Number of sections of file?
0x0004	0x00000020	Pointer to start of section 1
0x0008	0x00004EF8	Pointer to start of section 2
0x000C	0x000050C8	Pointer to start of section 3
0x0010	0x00005228	Pointer to start of section 4
0x0014	0x0000A2CC	Pointer to start of section 5
0x0018	0x00000000	Pointer to start of section 6 (zeros because it's not present?)
0x001C	0x0000ABDC	The length of the file

Note that at offset 0xABDC is the start of a long string of zeros that runs until nearly the end of the data. I am thinking that this is not part of Serge's battle model and that we can ignore it for now.

If this is accurate (and please correct me if it is not), the next step is to identify the contents of each section. I would expect the vertex/normal/uv/face data to be in one of the larger sections (like the first one).

[FaustWolf]

Wow! Luminaire, Halkun, thank you both for your input!

Looks like a header table indeed. I'll get Kid's model data mapped and uploaded tonight; cross-referencing will probably confirm that the headers have the same structure, though the sections will have different offsets since Kid's model data takes up more space.

Halkun, did you and Qhimm actually view Chrono Cross models? Even if they're screwed up, that is totally awesome! Thank you both VERY much for your work on this! Is it possible to "decompile" binary model data back into text format?

Ooh, and Lupus Erectus provided some input at Qhimm that will be applicable here:

I had a look at the "modelmine" file, I just noticed only now.
The file is nicely structured a header and relative offsets to subsections, and each subsection has a similar header. The only exception seems to be the (probable) model data starting at 9800, this may be hardcoded in the game code. Same for 14800 which is a header containig pointers to the other subsections until the end of the file.
Section 00009800 ~ 00014800:
the first $20 bytes are a header. The first value may be the number of sections, the others are relative the offsets (relative to the beginning of current section, that is). Some of the sections are similarly split in subsections. I have no clue what they may be
Section 14800 - end of file:
header as above; the "unknown" sections between weapon textures are certainly weapon models.
for example, the first is at 15dc8. From there:
    0-7 unknown
    8-11 offset to vertices section
    12-15 probably offset to unknown section (at 15df0)
        this seems to be made of groups of 4 bytes. I nocited some textures have weid coloring,
        maybe models are textured AND color shaded.  They could be RGB values. (but the 4th byte?)
    16-19 probably offset to unknown section (at 15ddc)
The vertices section is a Cyber said above: groups of 4 2-bytes values, the last is 00 00. Vagrant Story is like this both for characters and weapons, but your model data here seems different.
Note how the first value also is often 00 00 or little more, that's because of the flatness of weapon models. However the last value sometimes is 01 00 or 02 00 

EDIT: So Luminaire, when relative offsets are given in a pointer table, the rule is to switch the byte order then? For example, the relative offsets in the table header in ModelMine1 read F8 4E; C8 50; etc.

I'm attaching a file that is ONLY Serge's battle model data (sans textures) to facilitate. It is exactly ABDC bytes long, so that seems to confirm your header table.

Without further knowledge of the model cracking process, I guess I'll go ahead and start screwing up data one section at a time and see what happens. That might tell us what aspects of the model each section is responsible for.

[attachment deleted by admin]

[Luminaire85]

The file is in little-endian format, meaning that the value 0x1A2B3C4D (corresponding to the decimal value of 439041101) is stored as 4D 3C 2B 1A in the file. Therefore the first four bytes of the header represent the number 0x00000006 (decimal value of 6) and not 0x06000000 (decimal value of 100663296). More details at Wikipedia.

Section 00009800 ~ 00014800:
the first $20 bytes are a header. The first value may be the number of sections, the others are relative the offsets (relative to the beginning of current section, that is). Some of the sections are similarly split in subsections. I have no clue what they may be

This is in good agreement with what I said, so that's a good sign.

[FaustWolf]

Thanks for the informative link Luminaire. I'm still at that point in my hacking career in which the words "Little Endian" bring up memories of childhood nursery rhymes.

The file I attached recently conflicts with what I had previously reported in the ModelMine1 map (where I said Serge's model data was 45127 bytes long -- or B047 bytes long). But in truth, the space between relative offsets ABDC and B047 is just a bunch of 00 buffer bytes, so I think we can treat the recent attachment as Serge's complete battle model.

Tonight I'll attach a ModelMine for Kid (battle and weapon models plus applied textures) and Kid's battle model without textures.