CHRONO CROSS FILE EXPLORATION THREAD

[Luminaire85]

So, I guess the next question is: if the goal were to gather the knowledge necessary for writing a plugin for 3DS Max, Lightwave, Milkshape, or Blender, how in-depth do we need to go in each section of data? Should I start documenting which byte offset I changed to screw up Serge's bandana, for example?

As I see it, the first goal is to narrow down the range of byte locations that could hold, for example, vertex data. If you can say that bytes x to y in the file represent a vertex, that could be helpful in both figuring out how the vertex data is stored and where the rest of the vertices are. It's probably more important to know that the data represents a vertex than it is which vertex it represents, so I wouldn't bother trying to search for specific body parts. Furthermore, I would not expect us to have to use your method to identify every vertex in Serge's battle model.

[FaustWolf]

Sounds good to me, Luminaire! After a bit more investigation of section 1 of the model data, I'll move on to section 2 and we'll see what other surprises await.

This update comes past everyone's bedtime, but things are getting even more interesting.

First, I'll post links to two Youtube vids I upped to illustrate the effects on Serge's model (still altering bytes in the first section of his model data while holding the other sections constant). They're both very similar, so I'd recommend skipping to the second one, because that's more interesting.

http://www.youtube.com/watch?v=IZeUvSJgL-g
http://www.youtube.com/watch?v=B8C4xeZYreg

Both vids show that I've managed to gouge out Serge's left eyeball, leaving his face horribly mutilated. I suppose it's possible that I've somehow folded one or more of his facial vertices over his eye texture. Besides that, the other major features are the typical "shadow spikes" that stretch off into nowhere. Whereas there's major spikes jutting off Serge's back in the first video in this post, the vertex dislocations on his back are far less pronounced in the second vid.

The true significance of this post is that I divided the first section of model data into about 30 equal intervals so that I could spread changes evenly throughout the data. I tried at one point to change the value at relative offset 29 (on the first row of data just after the full model header), but that caused my emulator to scream bloody murder and crash when it tried to load Serge's battle model. Therefore, the earliest rows of data in the first section are critical in some way. Perhaps they constitute an internal header that applies to the vertices?

Also, perhaps very importantly, perhaps not, I've found a fascinating byte-value progression pattern that lasts from approximately relative offsets 28F0 ~ 4247 in Serge's battle model data. It's rather difficult to explain without a visual aid, so--

http://img230.imageshack.us/img230/9238/successionpatternth7.gif

The columns outlined in red progress in the following way.

xx xx xx xx xx xx 01 yy - xx xx xx xx xx xx 02 yy
xx xx xx xx xx xx 03 yy - xx xx xx xx xx xx 04 yy

And so on. The "yy" columns outlined in green evolve from 00 to 01 to 02 once the red-highlighted columns run their course from 01 to FF. At relative offset 4247 the data loses this pattern before the green-outlined columsn can progress fully to 03.

I'm not sure what it means though. Before I move on to examine the second section of Serge's model data, I'll see if I can find this pattern in Kid's or Guile's battle models.

I'm attaching my notation of which offsets I changed to yield the videos above. The first part of these notes corresponds to the first new video, the second part details a dud attempt due to game crashing, and the third part corresponds to the second new video.

Anyway, is it safe to assume that the first section of battle model data pertains to vertices, and vertices only?

EDIT: In my notation, "OByte" signifies the original byte value, while "NByte" signifies the new byte value to which I changed the offset.



[attachment deleted by admin]

[Luminaire85]

If by "vertices only" you mean just the spatial position of vertices in the model, then I doubt this is the only data in the first section. I don't think there are enough vertices on the Serge model to fill that space, and the hex data suggests there are multiple subsections within the first section anyway. More likely is that the first section contains all of the vertex, normal, and texture data, the face data (i.e. how to connect the vertices together), and maybe one or two other little things.

I had noticed that section you pointed out before, but now that you've posted that image the pattern in the red-boxed locations is obvious. This is very close to what was suggested would be the pattern for the vertex data, only instead of using 00 00 as padding bytes to make each vertex 8 bytes, some sort of counter bytes appear to be used instead. Also interesting in that section of hex code is how every second column has entries that are only either 00 or FF (excluding the boxed columns, of course).

[FaustWolf]

Hopefully this vid will help illuminate things:

http://www.youtube.com/watch?v=mlQctDrSJek

I overwrote that "special" section Luminaire speaks of in part one of the model data with its equivalent in Kid's model, leading to Serge's complete and utter discombobulation. It should be noted that Kid's "special" section is similar to Serge's but not identical in structure. Hers follows this pattern:

xx xx 01 yy xx xx xx xx - xx xx 02 yy xx xx xx xx
xx xx 03 yy xx xx xx xx - xx xx 04 yy xx xx xx xx

The "special" section in part one of Guile's model data does follow the same general pattern as Serge's does though. I'll write Guile's data over Serge's next and see what happens.

So Luminaire, we'll probably need to identify which parts of section one of the battle model data pertain to various aspects of the model. Have you seen a subheader for the first section at all? I imagine it probably occurs starting at relative offset 20 but I'm unsure how long the subheader would be. In any case, I'll take a closer look at that later.

[Luminaire85]

The header starting at 0x0020 is not a strict pointer table like that at 0x0000. However, there appears to be part of a pointer table within it:

Offset	Value	Comments
0x0020	0x00000002	?
0x0024	0x00000014	?
0x0028	0x00000208	?
0x002C	0x042B0205	?
0x0030	0x000000F9	?
0x0034	0x00000338	Pointer to start of ? section
0x0038	0x000028C8	Pointer to start of ? section
0x003C	0x00004228	Pointer to start of ? section
0x0040	0x00000011	?
0x0044	0x00000000	?

My guess is the header ends at that point, although I could be wrong.

The subsection that starts at 0x0358 (0x0338 + 0x0020 for the offset to the start of this section) has a sharp decrease in the density of zeros as the subsection that precedes it. Near 0x28E8 is the reverse: a sudden increase in the density of zeros after that point. And I believe 0x4248 is the end of the "special" section we've been discussing today. I don't see any significance to 0x0034 and 0x0228 yet, other than the former's presence in the table above.

Lots of questions marks to hopefully fill in above.

[FaustWolf]

Ah, very good! If the offsets are relative to the subheader, then the pointers to 28C8 and 4228 correspond to offsets 28E8 and 4248 in Serge's overall battle model (untextured). Aaaand, budda-bing, offsets 28E8 ~ 4247 do represent all the "special data" we've been talking about!

We'll have to think of a name for this "special" data, then you can fill that in in your section one subheader table Luminaire.

Thankfully, the "special" data doesn't seem to have its own sub-subheader because it starts right into the byte progression pattern.

EDIT: Oh, and I just found the following utility. I'm going to start tossing some Chrono Cross data at it when I get a chance:

http://www.backerstreet.com/rec/rec.htm

Looks like it *might* be useful. It works with PlayStation compiled data, but is says "MIPS Target only." Anyone know what that means, exactly?

EDIT: Doesn't seem that the REC decompiler is giving me anything useful. It only works on the SLUS file and doesn't give me anything I couldn't have found with a hex viewer and disassembler, but perhaps others more experienced than I would have expected that.

UPDATE: If the following 32 bytes constitute the section one subheader in Serge's model file...



The final three sections are definitely all pointers. The blocks marked "xxxx", "yyyy" and "zzzz" below...

| aaaa | bbbb | ????  | ????
| ????  | xxxx | yyyy | zzzz

Correspond to (xxxx) the beginning of a very dense hex string with no 00 bytes; (yyyy) the "special data"; and (zzzz) whatever comes between the "Special Data" and the end of the first section.

I will say that "aaaa" and "bbbb" are the same for Serge and Kid, but not Guile. So these do not provide any kind of common header lead-in.

[Luminaire85]

The final subsection of section 1 - from 0x4248 to 0x4EF7 - also has a common pattern of ii jj kk 00 for seemingly any byte values ii, jj, and kk. This section is 3248 bytes long, and if it contains 32-bit chunks, then there are 812 of them. Guess how many 64-bit chunks are in 0x28E8 to 0x4247: 812!

For reference, the second of the subsections - from 0x0358 to 0x28E7 - is 9616 bytes long and the first subsection - perhaps 0x0048 to 0x0357 - is 784 bytes long. (Note that what I called the second subsection is the first referenced in the header, and starts at 0x0358.)

Not sure what all of this means exactly. 812 seems like a lot of vertices for the Serge model, doesn't it?

Edit: Fixed typo.

[FaustWolf]

VERY good find, Luminaire! When you say that the final subsection lasts from 4248 to 43F7, do you mean 4EF7? Just checking.

I have no clue whether 812 vertices would be too much. However, we can compare to Kid's and Guile's models and see if theirs have similar numbers (which they probably do). On what basis can we assume that there's 32 bits (4 bytes?) per vertex in the final subsection? If there were 8 bytes per vertex in that section, it would bring the number of vertices down to 406, which might be more reasonable, perhaps. But I know no more than a stone when it comes to 3D modeling (I've gotta read those docs Halkun and Cyb provided yet).

Halkun, Cyb? How many vertices did Final Fantasy 7 character battle models typically have? I imagine Chrono Cross would have more because the models should be more detailed, but it could give us a rough ballpark figure on how many vertices the typical PlayStation character model has.

Anyway, the next step I'm going to take is to start fooling with each subsection of model data (meaning, each subsection within the first section) separately and see what happens. We'll document the probable data contained within each section based on the effects, then move on to the second full section of model data.

[Cyberman]

Also, perhaps very importantly, perhaps not, I've found a fascinating byte-value progression pattern that lasts from approximately relative offsets 28F0 ~ 4247 in Serge's battle model data. It's rather difficult to explain without a visual aid, so--

http://img230.imageshack.us/img230/9238/successionpatternth7.gif

The columns outlined in red progress in the following way.

xx xx xx xx xx xx 01 yy - xx xx xx xx xx xx 02 yy
xx xx xx xx xx xx 03 yy - xx xx xx xx xx xx 04 yy

And so on. The "yy" columns outlined in green evolve from 00 to 01 to 02 once the red-highlighted columns run their course from 01 to FF. At relative offset 4247 the data loses this pattern before the green-outlined columsn can progress fully to 03.

That appears to be vertex data IE the 3d locations Notice the numbers in the other columns are SIGNED SHORT INTS. IE D8 FF is FFD8 or -40. The incrementing numbers (since that 4th column can't have any real use but MUST exist due to the 32 bit alignment issues in the PSX to maintain performance), are likely them making better use of this memory.  Could be the vertex ID's (my guess at leasT). So you have found the VERTEX pool is my guess

Cyb

[FaustWolf]

Thanks Cyb! We can most likely start calling the final subsection "vertex data" then! Yay!

Oh! But before we consider the term "vertex data" safe for the final subsection, does the fact that the final subsection of Kid's model data progresses based on the following scheme...

xx xx 01 yy xx xx xx xx - xx xx 02 yy xx xx xx xx
xx xx 03 yy xx xx xx xx - xx xx 04 yy xx xx xx xx

...change anything? It's just like Serge's model, but her vertex data seems shifted left by four columns.

[Luminaire85]

Four columns is probably okay. If it was not a multiple of four, I'd be worried.

I was thinking the same thing about the format of the vertex data. But can we just take the values directly from the hex? When I do I get the attached picture. I also attached a .zip file containing a .blend file (for Blender) since it's hard to see the 3D position of the vertices in a single 2D image.

Of course, we still need to determine how the vertices are connected into faces.


[attachment deleted by admin]

[FaustWolf]

NICE work there, Luminaire!! OMG, I think I can actually see Serge in that cloud of vertex dots, sort of. Unfortunately my graphics card (ATI Express IIRC) detests Blender, so I won't be able to check this out personally until I have time to let my PC sit there and make crunching noises while it tries to load the model data.

Does this mean a plugin doesn't even need to be written to view the data? But I expect it will have to be done to view the full model with textures and all.

[Cyberman]

There should be a vertex list somewhere.  You found the GPU polygon commands correct? The list should be about the exact same number of entities as there are polygon opcodes in 4 word columns (Again a multiple of 32 bits). Triangle information likely terminates with a zeroed column. So I would look for data that has 0 spaces in the same ordered location as the triangle data does in the GPU opcode list.
Then you will have
Vertex Data
Polygon Vertex List
and GPU opcodes with UV locations in the textures.

Cyb

[FaustWolf]

I've found the GPU opcode commands in Halkun's doc, so that's a start. I still have to find them yet within the model data, however. Looks like that's next on my list of things to do then, because that'll answer some questions.

Luminaire, I don't know if you've seen Halkun's PSX doc yet. If not, you can download it directly here:
http://www.zophar.net/tech/files/psx.pdf

GPU opcode info starts on page 38 as viewed in Adobe Reader. On p. 39, I think the table labeled "0x2c textured 3 point polygon" should read "0x2c textured 4 point polygon." Am I correct in that, Halkun?

Halkun or Cyb, it is my current understanding that there should be one GPU opcode per "face," right? Since each GPU command specifies three or four vertices... So if we were checking out, say, a pyramid with a triangular base, there would be four GPU commands to look out for?

And, will the color bytes directly following a GPU opcode be 00 bytes if the GPU opcode pertains to textured polygons? Or would the color bytes be used for shading in that case?

EDIT: I thought I should mention that I'm seeing an anomoly in the vertex pool. I hope this doesn't change things too much. Basically, there's a point at which the byte progression scheme changes. Right in the middle of the data, it will change from:

xx xx xx xx xx xx 01 yy - xx xx xx xx xx xx 02 yy
xx xx xx xx xx xx 03 yy - xx xx xx xx xx xx 04 yy

to:

xx xx xx xx xx xx 01 yy - xx xx xx xx xx xx 01 yy
xx xx xx xx xx xx 02 yy - xx xx xx xx xx xx 02 yy

Thus, the byte progression slows down to half-pace, I guess you could say. Here's a graphical illustration:
http://img144.imageshack.us/img144/4627/subsection1anomolyyv2.gif

I'm not sure what significance it has, if any.

[FaustWolf]

Time for an update so large it needs a new post. To summarize our progress so far, I've more or less mapped everything we know about the header and first section subheader anatomy of the battle models, using Serge, Kid, and Guile as examples. Everything's listed as hexadecimal offsets.

In the hex viewer representations reproduced below, the byte strings are broken into four-byte chunks by red lines (for the overall model header) and blue lines (for the first section subheader). Relative offsets we've identified in the subheaders are followed by their true addresses in the model files (in parentheses). Anything we (or at least I) have not identified yet will receive the "???" treatment.

I always ignore the fifth line of hex code for Serge and Kid, but it becomes extremely important in the first section subheader of Guile's model.

First up is Serge's header and first section subheader:

Header: 00 ~ 1F
6 Sections           | Section1 0x20       | Section2 0x4EF8 | Section3 0x50C8
Section4 0x5228  | Section5 0xA2CC  | Section6 N/A      | EOF 0xABDC

Section1: 20 ~ 4EF7
SubHeader: 20 ~ 3F
Pointer to Gobbledegook  | PointerTable 0x14(34)       | ??? 0x0280                       | Gobbledegook 0x042B0205
??? 0xF9                         | SubSection2 0x0338(0358)| SubSection3 0x28C8(28E8)| SubSection4 0x4228(4248)
SubSection1: 40 ~ 357
SubSection2: 358 ~ 28E7
Vertex Pool, SubSection3: 28E8 ~ 4247
SubSection 4: 4248 ~ 4EF7


Next up is Kid's model header and first section subheader:

Header: 00 ~ 1F
6 sections           | Section1 0x20     | Section2 0x4DB4   | Section3 0x50C4
Section4 0x5224 | Section5 0xC6F0  | Section6 N/A         | EOF 0xD000

Section1: 20 ~ 4DB3
SubHeader: 20 ~ 3F
Pointer to Gobbledegook | PointerTable 0x14(34)       | ??? 0x0238                        | Gobbledegook 0x040701D9
??? 0x0226                    | SubSection2 0x0374(0394)| SubSection3 0x2C04(2C24)| SubSection4 0x4264(4284)
SubSection1: 40 ~ 393
SubSection2: 394 ~ 2C23
SubSection3: 2C24 ~ 4283
SubSection4: 4284 ~ 50C3


Guile's Battle Model Header and First Subsection Header:

Header: 00 ~ 1F
6 Sections           | Section1 0x20      | Section2 0x4C3C  | Section3 0x4F10
Section4 0x5070  | Section5 0xE41C  | Section6 N/A        | EOF 0xED2C

Section1: 20 ~ 4C3B
SubHeader: 20 ~ 4F
Pointer to Gobbledegook       | PointerTable 0x20(40)         | ??? 0xD8                           | ??? 0x016C
??? 0x01B8                          | ??? 0x025C                        | Gobbledegook 0x049A034B | ??? 0x02BE
SubSection2 0x033C(035C)   | SubSection3: 0x2A38(2A58)| SubSection4: 0x40D0(40F0)| ??? 0x0D
SubSection1: 50 ~ 35B
SubSection2: 35C ~ 2A57
Vertex Pool, Subsection3: 2A58 ~ 40EF
SubSection4: 40F0 ~ 4C3B

I'm still trying to find GPU opcodes within the model files, and so far I've come up dry searching through the first section of everyone's models. GPU opcodes wouldn't occur in the textures themselves, right? It should be in with the model data to my understanding.

EDIT: Luminaire85 has determined that the first 4-byte string in the first subsection header points to a section of data that I've termed "Gobbledegook" (solely for the reason that we don't know what it represents yet).