Chrono Cross engine reimplementation?

[Schala Zeal]

So I've been noting the various "open source engine implementations" for various retro games like Doom, Quake, ES3, etc and it made me consider that the Chrono Cross game discs essentially have the script, dialogue, model, texture etc data in readable formats. Of course, there remains various bits and pieces of unknown data, particularly in the ATIM texture format and 4 bytes of the LZSS compression format.

My C++ skills have vastly improved these past months, and so after studing the OpenMW source code, I'm eager to put it to the test and perhaps learn more in the process, particularly in the area of OpenGL or perhaps the new Vulkan API.

From what I've gathered, I think so far these libraries/dependencies will help:

- SDL 2: For video, audio management
- OpenSceneGraph: For its higher level OpenGL components
- Boost: Endianness management, cyclic buffers, dynamic bitset, etc

If successful, this engine may allow for modding the game in a way similar to Bethesda games or Doom. Perhaps a Radical Dreamers mod using CC data? Of course, a copy of Chrono Cross will be required on the end user's part. This is, after all, just an engine that makes use of it.

[kolt54321]

That would be useful! I mean, there's potential to even add more connections from CT to CC, and fix whatever complaints there are about pacing, too many characters, etc.

[Schala Zeal]

I've also decided to add a subproject: a plugin for Assimp to allow MDL/MESH files from CC to be imported. Assimp's an open source C/C++ library that imports/exports a dozen or two popular 3D file formats (Blender, FBX, OBJ, Direct X, Doom, Quake...)

[alfadorredux]

(Regarding LZSS testing blues)

The files you probably want to use to test de-LZSS are the dialogue subfiles, which generate (some) comprehensible text in the Cross relative alphabet--mapping the relative alphabet to ASCII is a simple task. So the sequence goes: deblob [room].script.cpt, take subfile 008 (where the first subfile is 000), de-lzss that, translate from relative alphabet to ASCII and check the contents. The final file should contain, if nothing else, the debug room strings, with "--------DUMMY END--------" near the end of the file. There will still be some binary garbage near the beginning, though--dialogue pointers, I think.

[Schala Zeal]

I see. Unfortunately Linux is going into confetti barf mode everytime I've been booting it up in the past 3 days, so I'm not sure how long a hiatus I'll be taking from this... -_- Nvidia can eat a dick.

[alfadorredux]

Why nVidia in particular? Did your setup get broken by a driver/kernel update? Or are you trying to use some distro that only supplies Nouveau and not the binary blob driver? I've found the latter to be reliable in general, but Nouveau is notoriously feature-poor and simply doesn't support a lot of cards. (After ten years of using Gentoo Linux as my primary OS, I've learned a lot of useless crap about how to set up a system . . .)

As for looking over the code at https://oku.edu.mie-u.ac.jp/~okumura/compression/lzss.c ...  I can try, but I've never been very fond of C (manual memory management and exposed pointer values give me hives, so I'll use almost anything else if I can get away with it), which means that I'm very, very rusty. Both on reading source and compiling C code outside the context of my distro's package manager. I'll give it a poke or two this weekend if I can find the time.

If you like, I can also de-lzss Cross files with my own code to give you samples for comparison--it does work, at least for the first buffer full (no guarantees on larger files, but it'll give you a starting point).

[Schala Zeal]

Binary blob driver. However, my cursor begins to get choppy and suddenly confetti barf.

[alfadorredux]

Hmmm. What distro are you using? Does this occur with all desktop environments?

[Schala Zeal]

Arch... and I'm on Plasma 5 desktop. It seems to have stopped after upgrading to Plasma 5.7

[alfadorredux]

Oh, KDE Plasma. That explains everything--they probably released something half-baked again.

[Schala Zeal]

Possibly. Any update on that LZSS issue? I linked to some C source code that implements it, but it mystifies me.

[alfadorredux]

Ugh. I warned you my C isn't that good, right? And that code was not written with easy readability in mind--there are a lot of places that require you to remember the order of operator precedence in C, which I do not. I'm pretty sure that the code as it stands can't be applied directly to Cross files without adding a line to skip the initial twelve bytes of the file, and possibly not even then.

The decode() function seems to start by initializing the buffer (Cross uses a 4096-byte buffer), then reads one bit and checks to see whether it's a 1 or a 0.

1 indicates that the next few bits (8, for Cross) contain a literal byte (so read the next eight bits from the compressed file and send them directly to the output).

0 should indicate a buffer backreference, but I can't make heads or tails of the code this bozo wrote there because he didn't actually say what most of his variables represent. However, in Cross, the next twelve bits after the 0 are a pointer indicating a byte in the buffer, and the four bits after that indicate the number of bytes to transcribe to the output. Every byte transcribed to the output also gets written to the buffer to support future backreferences, and when you hit the end of the buffer, you loop back around to the beginning. All this suggests that EI might be the pointer length in bits and EJ might be the length in bits of the number of bytes to copy out of the buffer, or at least the variables' suggested values match.

Lather-rinse-repeat until you run out of bits in the file.

I don't know if that's any help, but just looking at this thing is making my migraine even worse.

[Schala Zeal]

Well I do believe I told you if you don't know C, you could build it (or have me build it) and then run the result. As I stated before, the extracted file matches the decompressed length specified, but I am unsure beyond that if it came out correctly. I'm not asking you to examine the code itself (other than perhaps the constants up top). I know about the format, but implementing it seems to blow up in my face.

[alfadorredux]

The thing is, that code is not a known-good implementation of Cross LZSS decompression specifically, and I honestly don't see how my running it could be useful.

The attachment contains a raw dumped CC file with LZSS compression and a decompressed version of it using a (hopefully!) known-good de-LZSS implementation (mine, for Purple Cat Tools). If you can decompress and recompress that file and you get byte-perfect matches (or almost so, modulo a few ASCII w characters), then you're probably doing it right. If you don't have a good comparator for binary files, try VBinDiff: https://www.cjmweb.net/vbindiff/ (your distro may have packages). That's likely the best help I can offer you.

[Schala Zeal]

Appreciated. Yeah, unmodified, the code I showed you obviously won't decompress CC files. I needed to adjust the constant values accordingly.