CHRONO CROSS FILE EXPLORATION THREAD

[ZeaLitY]

Ah crap, it'll have to be tomorrow. But I'm curious if the <---> data comes in varying sizes...like, if a block of data after the battle texture dwarfs the data after the weapons textures. We can establish a rhythm of what comes before what if it plays out like that.

[ZeaLitY]

Code: [Select]

Headers
                                    00 00 00 00 64 65 67 75              ....degu
04 00 22 01 00 00 00 00 64 65 67 31 04 00 22 01 00 00 00 00  ..".....deg1..".....
64 65 67 32 04 00 22 01 00 00 00 00 64 65 67 33 04 00 22 01  deg2..".....deg3..".
00 00 00 00 64 65 67 34 04 00 22 01 00 00 00 00 64 65 67 35  ....deg4..".....deg5
04 00 22 01 00 00 00 00 65 6e 6b 65 04 00 22 08 00 00 00 00  ..".....enke..".....
65 6e 6b 31 04 00 22 04 00 00 00 00 67 61 6b 65 04 00 22 01  enk1..".....gake..".
00 00 00 00 67 61 6b 31 04 00 22 01 00 00 00 00 67 61 6b 32  ....gak1..".....gak2
04 00 22 01 00 00 00 00 67 61 6b 33 04 00 22 01 00 00 00 00  ..".....gak3..".....
67 61 6b 34 04 00 22 01 00 00 00 00 67 61 6b 35 04 00 22 01  gak4..".....gak5..".
00 00 00 00 67 61 6b 36 04 00 22 01 00 00 00 00 67 61 6b 37  ....gak6..".....gak7
04 00 22 01 00 00 00 00 67 61 6b 38 04 00 22 01 00 00 00 00  ..".....gak8..".....
67 73 30 31 04 00 22 01 00 00 00 00 67 73 30 32 04 00 22 01  gs01..".....gs02..".
00 00 00 00 67 73 30 33 04 00 22 01 00 00 00 00 67 73 30 34  ....gs03..".....gs04
04 00 22 01 00 00 00 00 67 75 30 31 04 00 22 01 00 00 00 00  ..".....gu01..".....
67 75 30 32 04 00 22 01 00 00 00 00 67 75 30 33 04 00 22 01  gu02..".....gu03..".
00 00 00 00 67 75 30 34 04 00 22 01 00 00 00 00 6a 69 6d 65  ....gu04..".....jime
04 00 22 01 00 00 00 00 6a 69 6d 31 04 00 22 01 00 00 00 00  ..".....jim1..".....
6a 69 6d 32 04 00 22 01 00 00 00 00 6a 69 6d 33 04 00 22 01  jim2..".....jim3..".
00 00 00 00 6a 69 6d 34 04 00 22 01 00 00 00 00 6b 75 6d 6f  ....jim4..".....kumo
04 00 22 02 00 00 00 00 6b 61 6e 6b 04 00 22 04              ..".....kank..".
There are the headers. Each is 3 bytes long. I'll now make a chart of the corresponding textures and see if we can establish some logic. Each header ends with .."., but as you can see in the hex, the " is sometimes 01, 02, or 04. Nonetheless, this might give us something to search. I've attached the OUT file with the headers still in there (at the bottom) but with the TIMs removed.

Edit: I've also attached the corresponding textures and names. They follow a set pattern, which is encouraging. If someone wants to guess translations for those, it'd be very welcome. For instance, a simple search yields kumo = clouds, as expected. Anyway, the good thing is that the entire OUT file starts with our famous "DRP" header, so I'm assuming whats left is the battle area box model and texturing instructions.

[attachment deleted by admin]

[FaustWolf]

Now for my report on the potential battle model / weapon model data.

Firstly, I've attached a zip containing a chunk of data from the CD image that I have named "ModelMine1." It's just raw hex data, and its beginning corresponds to a fresh sector (2048 bytes/sector as always) in the CD image. The data fall in this order: A header; Serge's battle model texture; Serge's eyeball texture; unknown data; Serge's first weapon model texture; unknown data; etc. Also inside the zip is a copy of my datamap for "ModelMine1," which I reproduce below. The chunk seems to be 11 sectors long even, if that means anything.

Also included in the zip are Serge's battle model texture, eyeball texture, and the seven weapon texture TIMs that occur within the ModelMine.

DATAMAP: All offsets given in hexadecimal notation.

00000000 ~ 0000000F: 16 byte header
00000010 ~ 0000823B: Serge Battle Texture
0000823C ~ 00009467: Serge Eyeball Blink Texture (NOTE FOR COMPENDIUMITES: slight differences from "4" in CD\Serge)
00009468 ~ 000097FF: 00 Buffer Bytes
00009800 ~ 00014847: Unknown Data (NOTE FOR COMPENDIUMITES: slight header differences from "3" in CD\Serge, but way larger file size)
00014848 ~ 00015DC7: Serge Weapon Texture 1
00015DC8 ~ 00016BE7: Unknown Data
00016BE8 ~ 00018167: Serge Weapon Texture 2
00018168 ~ 00018C5F: Unknown Data
00018C60 ~ 0001A1DF: Serge Weapon Texture 3
0001A1E0 ~ 0001AF0F: Unknown Data
0001AF10 ~ 0001C48F: Serge Weapon Texture 4
0001C490 ~ 0001D2EF: Unkonown Data
0001D2F0 ~ 0001E86F: Serge Weapon Texture 5
0001E870 ~ 0001F687: Unkknown Data
0001F688 ~ 00020C07: Serge Weapon Texture 6
00020C08 ~ 00021897: Unknown Data
00021898 ~ 00022E17: Serge Weapon Texture 7
00022E18 ~ 00023FFF: Unknown Data

SUSPECTED MODEL FILE INFO: The offsets and file lengths may actually be one byte too "short" depending on whether I'm interpreting the hex code correctly. For example, the First "Weapon Model" listed below is described as 3615 bytes long. If it makes more sense, feel free to interpret the file size as 3616 bytes.

First "Model" : 45127 bytes long. Header begins "06 00 00 00 20 00 00 00"
First "Weapon Model" : 3615 bytes long. Header begins "01 00 00 80 48 00 5E 00"
Second "Weapon Model" : 2807 bytes long. Header begins "01 00 00 80 39 00 46 00"
Third "Weapon Model" : 3375 bytes long. Header begins "01 00 00 80 43 00 58 00"

The weapon "models" are consistent in their first 8 bytes, which may constitute a model file header; the first 8 bytes for the weapon "models" read:  01 00 00 80 xx 00 xx 00

My notes on the data were hastily scribed, so if anyone wants clarification on any point, please ask. Once again, the "ModelMine" for Serge's (suspected) battle and weapon models and all the associated textures are in the attached zip if anyone would like to investigate.

EDIT: Ah, and yes, to make a long story short Zeality, the "Battle Model" does dwarf the "weapon models." This is encouraging. The "weapon models" seem to vary slightly in size, which I suppose would be understandable considering Serge's weapons are shaped slightly differently, no?

Looks like you've probably found the 3D battlefield data as well, since it seems to follow a similar sandwiching pattern (model data sandwiched between textures I mean).

[attachment deleted by admin]

[FaustWolf]

Hopefully someone will be able to analyze what Zeality and I have found regarding potential model data and give us some feedback.

Moving on, I've found some very interesting ASCII blocks in Chrono Cross' SLUS file. First up...

Seems to be file paths. Is it possible that this is a signature left from the disc burning process, meaning there's MDE, FID, FND, and PRG files on the game CD? Anyone know what these file extensions even refer to?

Next up...

If I had to take a wild guess, I'd say these might be assembly language opcodes that tell the Playstation GPU how to handle model data, but keep in mind that "I R a NooB" at this sort of thing. Now, this isn't functional opcode data -- just ASCII evidence that the opcodes are in the SLUS. I expect I'd find these elsewhere in the SLUS when I investigate with a disassembler such as PS2Dis.

Anyone have any input on this?

[halkun]

If you need any help with PSX internals, I'm here ^_^. Think of me as a free guide to that one PSX doc that's on the internet. I hear that the guy who made at and myself are really close...

--->||<---

Like that even.

[FaustWolf]

Holy cow, Halkun's here! 

Yes, I'll definitely be asking you questions as time goes on. I'd better read your PSX doc so I know the right questions to ask.

I suppose for the time being, I'm wondering what the "kz_modelGetNormalPolygon()" and similar functions shown in the above pics ARE exactly. I'm seeing these things when I view the SLUS file in the PS2Dis disassembler. If it's part of the PsyQ library, I won't bring it up again since Sony means to keep a tight lid on that info. I'm just wondering if it'll help us view model data in any way.

And regarding the GPU quad opcode subject you brought up earlier over at Qhimm, where would that data occur in the typical PlayStation game CD? Only in the SLUS, near the model data, both, or am I completely clueless?

For anyone who doesn't know yet, Halkun's the most knowledgeable person on PlayStation architecture outside of Sony itself - and he probably knows even more than they do.

[halkun]

In FF8, I found the quad data right after the UV (texture) TIM. It was a partial opcode list that collected the UV information for later aggregation into the OT sequence.

Little help with vocabulary:
UV Map : Another way to say "Texture map" It's called UV because U and V are used to represent the 2D coordinates on the texture map.  X,Y, and Z are used for 3D coordinates.

OT Sequence : The Ordering Table Sequence : The big list of opcodes that get linked together to be sent to the GPU


The layout was something like this:

(quad opcode)
(blank)
(U point 1)
(V point 1)
(U point 2)
(V point 2)
(U point 3)
(V point 3)
(U point 4)
(V point 4)

That's what I remember.

The tris were the same way, but only three points as opposed to four.

[FaustWolf]

Ah, that helps immensely! So Halkun, is there quad opcode after every character model texture in FF8, regardless of whether they are field models or battle models? If so, I can load my FF8 game image into a hex editor, find a texture for Squall or something, and see what the opcodes look like in hex perhaps.

Though in FF8, like in Chrono Cross, PSicture and TIMViewer seem to detect a lot of "clone textures," some of which I suspect are present extraneously and not applied to any model at all. Or maybe I'm wrong?

But anyway, this may help us determine for sure whether Zeality and I actually *have* model data.

EDIT: I've just done some investigation of the data following Zell's battle model texture(s) in my disassembler (because viewing data through a disassembler makes me feel cool ), and it appears the data immediately following Zell's texture starts with the following assembly language instructions:

dsrav zero, zero, zero
sync
dsllv  zero, zero, zero

Halkun, does this provide any clue as to whether or not I'm looking at quad opcodes? Or, an alternative question, how is one able to tell that he/she is even looking at quad opcodes?

And I've got a few quick questions about your PSX doc, which I am still reading through (rather haphazardly, as time allows):

1.) In your GPU packet command list, "0x2c" constitutes a packet command for a textured 4-point polygon. What would that look like in hex? Just "2C"? And would that appear in a quad opcode?

2.) You differentiate between "monochrome" and "textured" polygons. Would I be correct in stating that Final Fantasy 7 uses monochrome polygons, whereas Final Fantasy 8 (and, by association, Chrono Cross) uses textured polygons? This is an obvious NooB question, but I just want to make sure.

[halkun]

1.) In your GPU packet command list, "0x2c" constitutes a packet command for a textured 4-point polygon. What would that look like in hex? Just "2C"? And would that appear in a quad opcode?

2.) You differentiate between "monochrome" and "textured" polygons. Would I be correct in stating that Final Fantasy 7 uses monochrome polygons, whereas Final Fantasy 8 (and, by association, Chrono Cross) uses textured polygons? This is an obvious NooB question, but I just want to make sure.

I'm going to answer #2 first, because that will answer #1.

First a little more vocabulary.

Triangle - A shape with three sides.
Quad - A shape with 4 sides
Vertex - a "corner' of a shape. A triangle has three vertexes (or vertices). A quad has four.


Let's pretend that I want to tell the GPU that I want to draw a red square (quad) where the vertex points were (0,0) (0,10) (10,0) (10,10). I will want to make the data something like this.

28 00 00 ff 00 00 00 00 00 00 00 0a 00 0a 00 00 00 0a 00 0a

Then send that off to the GPU.

It breaks down like this.
===============
28 = The GPU command for a Monochrome (single color) 4 point polygon

00 = blue value
00 = green value
ff = red value

00 00 = y0, vertex 0
00 00 = x0, vertex 0
00 00 = y1, vertex 1
00 0a = x1, vertex 1
00 0a = y2, vertex 2
00 00 = x2, vertex 2
00 0a = y3, vertex 3
00 0a = x3, vertex 3
===========

You see how it goes? Remember, the GPU only draws in 2D. The GTE is responsible for the projected 3D maths.

FF7 used gradated 3-point polygons (one color at each vertex). It's format is different.

Any other questions?

[FaustWolf]

THAT...is amazing.  Thanks Halkun!

I'll ask a series of questions as a followup to your post:

1.) So does

28 00 00 ff 00 00 00 00 00 00 00 0a 00 0a 00 00 00 0a 00 0a

constitute a monochrome quad opcode in hex then?

The structure will look very different when textures are wrapped around a model I expect(?), but at least I can look out for byte strings that start out with values like "24"; "2C"; and "34"; and "3C" which would give me an idea that I'm looking at data pertaining to textured triangles, textured quadrilaterals, gradated-textured triangles, or gradated-textured quadrilaterals respectively (reading from your PSX doc).

2.) Going back to your "28 00 00 ff ..." byte string example, would the "28" occur on a fresh line if I'm looking at game data with 16 bytes per line in a hex editor? Or could this instruction occur anywhere?

3.) Is there any way to tell, looking at a model texture, that the corresponding model is gradated-textured versus "plain" textured? I can see why the Final Fantasy 7 models are gradated triangles and not monochrome triangles now that you mention it.

4.) I take it that I should just put away the disassembler for now and look for opcodes in a hex viewer?

5.) One more question just for my curiosity: is color data sent to the GPU byte-order-reversed then? I'm used to thinking in terms of Red-Green-Blue, but the PSX GPU thinks in terms of Blue-Green-Red if I understand correctly.

[halkun]

THAT...is amazing.  Thanks Halkun!

I'll ask a series of questions as a followup to your post:

1.) So does

28 00 00 ff 00 00 00 00 00 00 00 0a 00 0a 00 00 00 0a 00 0a

constitute a monochrome quad opcode in hex then?

The structure will look very different when textures are wrapped around a model I expect(?), but at least I can look out for byte strings that start out with values like "24"; "2C"; and "34"; and "3C" which would give me an idea that I'm looking at data pertaining to textured triangles, textured quadrilaterals, gradated-textured triangles, or gradated-textured quadrilaterals respectively (reading from your PSX doc).

This sequence....

28 00 00 ff 00 00 00 00 00 00 00 0a 00 0a 00 00 00 0a 00 0a

That is a GPU *command*. The opcode is just the 28 at the beginning (operation code).

2.) Going back to your "28 00 00 ff ..." byte string example, would the "28" occur on a fresh line if I'm looking at game data with 16 bytes per line in a hex editor? Or could this instruction occur anywhere?

GPU data will only relly show up with UV maps and only in the model data. It will be too hidden in the game execuatable. Different GPU commands are different lengths. The commands are all broken down in my doc.

3.) Is there any way to tell, looking at a model texture, that the corresponding model is gradated-textured versus "plain" textured? I can see why the Final Fantasy 7 models are gradated triangles and not monochrome triangles now that you mention it.

Most textured polys are actually gradated. The colors under the texture is what's used for lighting. The opcode hunt is good for looking for the UV maps. (The lines that trace out the polygon boundries on the TIM file). These will have the 2D UV coordinates, but the polygon data will be blank or missing.The Poly data will be around that area though.

4.) I take it that I should just put away the disassembler for now and look for opcodes in a hex viewer?

You will not find code in the data files. Looking through the executable is kind of a waste of time. Hex viewing and seeing patterend is how you will find things. Load up an uncompessed data file as an image (using a RAW loader where you can define X,Y and colordepth) you will see patterens change where the data is a different kind. That helps a lot. Also, look for tables that go up. the tables point to offsets and that marks data boundries.

5.) One more question just for my curiosity: is color data sent to the GPU byte-order-reversed then? I'm used to thinking in terms of Red-Green-Blue, but the PSX GPU thinks in terms of Blue-Green-Red if I understand correctly.

The PSX is a MIPS computer, so it's reverse-edian in relation to a PC.

[Cyberman]

5.) One more question just for my curiosity: is color data sent to the GPU byte-order-reversed then? I'm used to thinking in terms of Red-Green-Blue, but the PSX GPU thinks in terms of Blue-Green-Red if I understand correctly.

The PSX is a MIPS computer, so it's reverse-edian in relation to a PC.

Actually halkun the R3k processor used on the PS1 thinks either way. MIPS has a patent on out of order little/big endian transfers that is quite annoying to many would be micro processor designers.
In simple terms it can read it either way. However all the byte ordering I've seen in Square soft's software thus far on the PS1 and up are little endian, which is the same as that of the PC.

Cyb

[FaustWolf]

Cyberman's here too! 

Thanks again to the Qhimm folks for their guidance and their willingness to migrate over here for this discussion. It is MUCHO appreciated.

Now that I know the byte values "3C" and "34" are the GPU opcodes pertaining to gradated-textured quadrilaterals (quads) and gradated-textured triangles respectively, I can sniff around the model texture data to make sure that what Zeality and I have are, indeed, models. This is because the textures should be accompanied by commands that tell the GPU how to "wrap" those textures around the models. Halkun and Cyb, please critique my understanding of this matter if I've misinterpreted.

By the way, awhile ago I replaced one of the Chrono Cross character's "overworld" models with a solid color and thereby had a chance to see the shading/lighting applied to the model: http://img134.imageshack.us/img134/815/kidoverworldyd3.jpg
Looking at this, would it appear that the shaded aqua-colored model does, indeed, use a gradated texture?

Looking back to that simple exercise, I wonder if we can "test" the supposed model data Zeality and I have found by altering the supposed model code in a hex viewer. Like, if we changed a few random bytes and a character's face gets all screwed up, we should be able to tell that we're fooling with model data, right? Secondly, by noticing the effect on the model, perhaps we can tell which "regions" of the model data pertain to bone structure, animation, etc, by trial and error. Then we could use this knowledge to help decode the model format Square used for this game. Does this seem like a plausible idea?

Ooh, that's tonight's project!


For anyone who wants to find out about GPU instructions and such, you can find Halkun's PSX doc at Zophar.net:
http://www.zophar.net/tech/psx.html

Zophar's version says it was last revised in 2000 (v. 1.0), Halkun. Is that the latest version you've released?

[Cyberman]

On Page 40 you will see the format of the data sent to the GPU for Triangles and Quads of the kind you are refering too.
Texture pages are 256x256 pixel sections on the display. the u and v values are points on that page and CLUT is the clut that is being used for it.  So I suggest you use that information and tweak the vertex locations on the page and find out.

Cyb

[FaustWolf]

Thanks for the tip Cyb!

Now's the time to start getting excited, folks:

http://img229.imageshack.us/img229/8201/chimerafu9.gif

This is what happens when I overwrite Kid's model geometry with Serge's model geometry in my test iso. It's sorta like Silence of the Lambs, only without the fava beans.

This means several things:

1.) We can now be 100% sure that we've located the true battle models. Otherwise, what I attempted wouldn't have had any effect.

2.) It will probably be just as easy to confirm that we've found the battlefield environment models. What we'll need to do is find two battlefields that have wildly different geometries, then do the copy&paste and see if one of them gets really messed up like this.

3.) We can rely on sheer trial and error if necessary to determine which byte strings in the model files correspond to geometry, animations, and whatever other data is incorporated into the larger model files. My gameplan is to employ what I call the "halfies" approach. We segregate a given model file into two parts, drastically change one of those parts, and see if, say, animations are screwed up in game. If so, we split that portion of the model in half, rinse and repeat until we've honed in on the animations.

But I worry that the "halfies" approach will screw up opcodes and such if they happen to be spread throughout the model files. Halkun, Cyb, Luminaire, jono, Vehek, Zeality, Akari, Ramsus, Gemini -- hell, everybody, is there a more powerful and efficient way of reverse-engineering model data?

I'd like to update this thread with some Youtube vids illustrating the chimera Serge/Kid so that everyone can see it in motion and view what's happened to Kid's animations. If other eyes besides mine look at it, we can collect insight from various hackers and modelers.

So -- does anyone know of any really good open source/freeware/shareware video capturing programs I could use?