Element Grid Growth Formula

[glennxserge]

Hi everyone, I'm poking around in some of the character stats trying to work at a few details about how the element grids function.  And I think I've mostly sorted out out the bitmasks that enable grids as well as how they are initially set when a character joins your party.  But I'm running into trouble understanding how the game determines which slot to open up at what level.  I'm not sure if it's set explicitly in some other data block or if there's a formula related to star level.

I'll use Serge's block as an example (from :MKL122788's character gamefaq)

Code: [Select]

[  22  ][  31  ][  39  ][      ][      ][      ][      ][      ]
[  19  ][  20  ][  24  ][  30  ][      ][      ][      ][      ]
[  12  ][  14  ][  16  ][  23  ][  36  ][      ][      ][      ]
[  07  ][  08  ][  09  ][  13  ][  21  ][  34  ][  46  ][  48  ]
[  00  ][  00  ][  00  ][  05  ][  11  ][  18  ][  26  ][  37  ]
[  01  ][  02  ][  03  ][  10  ][  15  ][  29  ][  35  ][  41  ]
[  04  ][  06  ][  25  ][  28  ][  33  ][  40  ][  43  ][  44  ]
[  17  ][  27  ][  32  ][  47  ][      ][      ][      ][      ]
At star level 0, there is one open slot in level 1, 2, and 3, on the fourth row, which is basically the center of the grid.  This is the case for every character, except Sprigg because she has no low level elements.  The rest of the slots are denoted with a number indicating which star level they will be unlocked.  I can't find anywhere that determines this growth pattern, and it's varies quite a bit character to character.

Referencing the stat data block for Serge at 0x13B721C8:

Code: [Select]

xx   00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
   - -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
00 - 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01 - 00 00 00 00 01 40 32 00 09 05 55 08 07 07 0A 0A
02 - 20 00 96 00 00 00 00 00 00 00 00 00 00 00 00 00
03 - FC 03 FC 03 FC 03 F8 03 F0 01 E0 01 E0 01 E0 01
04 - 00 40 00 40 00 40 00 40 69 40 00 40 00 40 00 40
05 - 00 40 00 40 00 40 00 40 55 40 00 40 00 40 00 40
06 - 00 40 00 40 00 40 00 40 00 40 A0 C0 00 40 00 40
07 - 00 40 00 40 00 40 00 40 00 40 00 40 00 40 00 40
08 - 00 40 00 40 A1 C0 00 40 00 40 00 40 00 40 00 40
09 - 00 40 00 40 A2 C0 00 40 00 40 00 40 00 40 00 40
0A - 00 00 7B 00 32 14 00 00 00 00 00 00 00 00 00 00
Line 03 is the bitmask series (2 bytes per grid level), that indicate the final grid shape
Line 04-09 are the specific element slot values (2 bytes per value, with the upper nibble of the high byte being a flag for slot behavior; bit 7 means occupied for stuff like a tech, bit 6 is default)

I don't see any other areas of the data block that would indicate which slots open at what level.  Does anyone know more about this? 

[glennxserge]

Figured out a couple more things that are worth documenting, that I found weren't identified in the modification pages for enemy data: https://www.chronocompendium.com/Term/2535.html.  The player characters actually share the exact same data block format, and a couple of the missing 'unknown' bytes from the enemy section are used for characters.

Relevant to this post, byte 0x03 (previously unknown) is some kind of offset for which grid element growth table to use.  There are 8 used growth types, with most of them being Pip (uses 3,4,5, and 6 for his various forms), and Harle uses 7.  Serge and friends use 0 for the most part, and Viper and associates all seem to use 1.  I'll get a complete list as I keep digging.  I need to see if I can find an explicit lookup table associated with that byte offset, but this does seem to be the way the game determines which slots become available for characters at specific star levels.  I'll keep checking.

The other unknown byte from that link, is 0x1F.  This is the character stamina attribute.  It's listed in the block in the full integer form, the game must divide by 10 to get the decimal point, but I've verified that this is the stat that's used to regenerate character stamina.  It makes sense this wouldn't be used for enemies since they use a different attribute to determine their turn order.

Alright, back to looking for the element tables...

[Grobycftw]

Sup Glennxserge! Long time!
I thought I shared this doc before somewhere but not 100% sure so I'll post it here

https://drive.google.com/file/d/157ShXf_YE0djzhVvIVZiWs6bjjpEei1W/view?usp=sharing

here's some info/test I did when looking how the element grid we're working. We, just like you couldn't figure out what determined which grid gets open at "x" lvl, but we did figure a few things out and just needed to do some more testing to hope finding an answer to this  
anyway hope this can help somehow, feel free to use/edit anything in there!

Im thinking maybe we should try to find some similar bytes in those blocks with characters that has similar Element Grid Growth, like Grobyc and Karsh I think? they have the same grid I believe, things like that would be worth being checked out!

[glennxserge]

Hey Grobyftw!  Thanks for sharing that, lots of good details in there 

Yeah, I tried finding some kind of lookup tables that represent the 7 specific growth patterns but I didn't have any luck.  Hard to search for something without knowing how the data is packed.  I should give this another look, after I read through your doc.

[ZeaLitY]

Updated that offset page in the encyclopedia. We can always get encyclopedia access going if you need edit access concurrent with some research/investigation; just let me know.

[glennxserge]

Thanks, ZeaLitY, I'm still pluggin' away when I have time.  I'll let you know if I learn anything new, but it's slow going.

[F-man]

I did a little research related on those grid lookup tables. The grid growth type 0, which is the most common one, seems to be 8x8 grid with 9 unknown values (rest can be gotten by combining Serge's and Guile's grids). So, I edited Serge's grid shape so that he could unlock all those unknown slots at some unknown star levels. Then I star-upped him to max stars, one star at time, to find which levels those would be. The answer ended up being never. Seems like those slots cannot be unlocked at any level as devs must have left them empty because they didn't use them. A very boring decission but might be useful information. I hope we can find those lookup tables.

[glennxserge]

That's a good test, F-man.  How are you incrementing the star level in game?  I'm taking a look at this again since I'm working through the character customization part of my tool.  I wasn't sure if the growth pattern was explicit, or something like: "at star level 1 an arbitrary first level slot opens", and then the grid layout of the character determined which slot matched that criteria.  So I did a similar test with star levels (but less complete than yours )

I tested using the grid bit masks in the player stat block to turn off the element slot that Serge learns at star level 1, and then fought the first boss.  The star level didn't yield any slots at all, which seems to imply that element growth is specifically hard coded somewhere.  I guess it's good to know that's the case, but it would be nice to have a better answer for where that information is stored. 

Oh, and one other thing that I discovered.  The stat blocks for characters only seems to apply at the point they are recruited.  So for Serge, making edits to his grid shape only worked if I started from a save state right as he's waking up at the beginning of the game.  There's probably more nuance here, but that's the only reliable way I've been able to test it.

[glennxserge]

I managed to run the full test on all star levels from 1-99 with all unknown slots checked too.  Ended up with the same result as you, F-man.  So it looks like growth table 0 (Serge, Kid, Guile, etc..) are bound by this leveling chart, regardless of their specific element grid availability:

Code: [Select]

[  22  ][  31  ][  39  ][      ]       ][      ][      ][  78  ]
[  19  ][  20  ][  24  ][  30  ][      ][      ][  69  ][  72  ]
[  12  ][  14  ][  16  ][  23  ][  36  ][  60  ][  63  ][  66  ]
[  07  ][  08  ][  09  ][  13  ][  21  ][  34  ][  46  ][  48  ]
[  00  ][  00  ][  00  ][  05  ][  11  ][  18  ][  26  ][  37  ]
[  01  ][  02  ][  03  ][  10  ][  15  ][  29  ][  35  ][  41  ]
[  04  ][  06  ][  25  ][  28  ][  33  ][  40  ][  43  ][  44  ]
[  17  ][  27  ][  32  ][  47  ][      ][      ][      ][  75  ]

Unfortunately, that means, the next test is chasing pointers and disc reads to figure out where that table lives.  Will let you know if I find anything.

[glennxserge]

   Made some progress tracking down the lookup table.  I was able to trace execution through the star level up routine and find where it's pulling data from.  The game reads in a block of data from 0x13B6B370 on disc 1 (sector mm,ss,ff == 31,16,71), and stores it in memory at 0x80101448.  I don't understand how this data is formatted, it seems obfuscated in a way.  But I did verify it's responsible for which slots open up at specific levels.  If anyone wants to step through it in a debugger, the instruction execution for using this table starts here (after a battle if a star is earned): 801EF130

   The best summary I can make is that the game iterates over this data for the number of current star levels in 16 byte strides and uses the table data with some bit manipulation to determine which slot opens.  It then ORs the value onto another table at 0x80071C7C. 
   This second table is the current state of unlocked slots for all 45 characters in the game.  It's stored as a 2 byte pair of grid bitmasks.  As the star level increases, this table will eventually match the grid bitmasks defined in the character stat blocks
 
   Once this second table has been updated, the routine finishes by iterating across each 2 byte pair, shifting and masking for each slot, and checks the low bit to see if the slot is unlocked.  If that is the case, then the game clears the seventh bit of the character stat block's low byte pair (0x4000 becomes 0x0000), which makes the slot usable.

I still need to dig into that first table to try and understand how it's formatted.  But at least now we know where the data lives on disc.

[glennxserge]

Ah, scratch that.  The data on disc isn't being used in this case, which is really strange because it matches exactly what's in ram.  But, that just means I need to keep tracing this upstream in the game execution.  I think I'm close to finding the answer

[glennxserge]

Well, another year has gone by, and I finally had some time to sit down and plug away on this problem again.  Not solved yet, but I'm much closer now.

There's a bunch of weird MIPS stuff in the post battle routine that took a while to sort through, but I was able to unpack the element growth tables and verify that you can alter this table and unlock a slot anywhere in the element grid at any level, provided the character's stat block has the right mask to allow it. 

The growth tables are explicitly defined as 100 consecutive 8 byte strides.  Each 8 bytes represents one star level, with the character's growth stat at 0x03 determining which of the 8 bytes is used for their grid.  I've compiled the standard 8x8 element grid with the table values that correspond to unlocking that slot as a reference, which I'll list below.

Code: [Select]

    Element Grid
1  2  3  4  5  6  7  8
-----------------------
02 0C 16 20 2A 34 3E 48
03 0D 17 21 2B 35 3F 49
04 0E 18 22 2C 36 40 4A
05 0F 19 23 2D 37 41 4B
06 10 1A 24 2E 38 42 4C
07 11 1B 25 2F 39 43 4D
08 12 1C 26 30 3A 44 4E
09 13 1D 27 31 3B 45 4FThese values are actually pretty easily derived, despite the game's obtuseness in how it generates them.  Given a row and column, the slot values is:
column * 10 + row + 2  //Decimal values, indexes start at 0

The table that the game generates ends up at 0x80101448 through some sort of DMA transfer, but I'm unfamiliar with doing traces of DMA requests through emulation.  There's probably some way to find where it's coming from.  That'll be my next test.  In the meantime, here's the full, 100 star level entry, that represents the element growth for all characters:

Code: [Select]

☆LV Growth Pattern Index
----------------------------------
0 FF FF FF FF FF FF FF FF   <--- 8 FF's mean the level up routine skips entirely
1 07 07 07 FF 08 08 08 07   <--- a single FF means that one growth index skips
2 11 11 11 FF 12 12 12 11
3 1B 1B 1B FF 1C FF 1C 1B
4 08 05 FF FF FF FF 26 05
5 24 0F 24 FF 09 FF 30 0F
6 12 19 FF 07 07 07 07 19
7 05 24 FF 11 11 11 11 24
8 0F FF 25 1B 1B 1B 1B FF
9 19 08 FF 25 25 25 25 08
10 25 25 FF 24 24 24 24 25
11 2E 2E 2E 05 05 05 05 2E
12 04 FF FF 2E 2E 2E 2E FF
13 23 23 FF 0F 0F 0F 0F 23
14 0E 04 FF 19 19 19 19 04
15 2F 2D 2F 2F 2F 2F 2F 2D
16 18 38 FF 04 04 04 04 38
17 09 FF FF 23 23 23 23 FF
18 38 0E 38 0E 0E 0E 0E 0E
19 03 2F FF 2D 2D 2D 2D 2F
20 0D 12 FF 18 18 18 18 12
21 2D 18 39 38 38 38 38 18
22 02 39 FF FF FF FF FF 39
23 22 FF FF 03 03 03 03 FF
24 17 1C 43 0D 0D 0D 0D 1C
25 1C 42 FF 39 39 39 39 43
26 42 03 42 22 22 22 22 03
27 13 37 05 17 17 17 17 37
28 26 22 08 FF 13 1C 42 22
29 39 26 0F FF 37 26 37 26
30 21 FF 19 FF 02 FF FF FF
31 0C 0D 12 FF 42 30 2C 0D
32 1D 41 23 FF FF 37 21 42
33 30 30 2D FF 0C 3A 41 30
34 37 17 37 FF 26 43 FF 17
35 43 02 41 FF 2C FF 36 02
36 2C FF 04 FF FF 02 2B FF
37 4C 4C 4C FF 43 42 43 4C
38 FF 3A 0E FF 30 2C 4C 3A
39 16 2C 1C FF 4C FF 3A 2C
40 3A 43 26 FF FF 4C FF 41
41 4D 21 18 FF 21 17 4B 21
42 FF 44 30 FF FF FF FF 44
43 44 0C 3A FF 4D 41 4D 0C
44 4E 36 4D FF FF FF 40 36
45 FF 4D 22 FF FF 36 35 4D
46 41 2B 2C FF FF 4D 4A 2B
47 27 4E 44 FF FF FF FF 4E
48 4B 40 FF FF FF FF FF 40
49 FF 4B FF FF FF FF FF 4B
50 FF 35 FF FF FF FF FF 35
51 FF 4A FF FF FF FF FF 4A
52 FF FF FF FF FF FF FF FF
53 FF FF FF FF FF FF FF FF
54 FF FF FF FF FF FF FF FF
55 FF FF FF FF FF FF FF FF
56 FF FF FF FF FF FF FF FF
57 FF FF FF FF FF FF FF FF
58 FF FF FF FF FF FF FF FF
59 FF FF FF FF FF FF FF FF
60 36 FF FF FF FF FF FF FF
61 FF FF FF FF FF FF FF FF
62 FF FF FF FF FF FF FF FF
63 40 FF FF FF FF FF FF FF
64 FF FF FF FF FF FF FF FF
65 FF FF FF FF FF FF FF FF
66 4A FF FF FF FF FF FF FF
67 FF FF FF FF FF FF FF FF
68 FF FF FF FF FF FF FF FF
69 3F FF FF FF FF FF FF FF
70 FF FF FF FF FF FF FF FF
71 FF FF FF FF FF FF FF FF
72 49 FF FF FF FF FF FF FF
73 FF FF FF FF FF FF FF FF
74 FF FF FF FF FF FF FF FF
75 4F FF FF FF FF FF FF FF
76 FF FF FF FF FF FF FF FF
77 FF FF FF FF FF FF FF FF
78 48 FF FF FF FF FF FF FF
79 FF FF FF FF FF FF FF FF
80 FF FF FF FF FF FF FF FF
81 FF FF FF FF FF FF FF FF
82 FF FF FF FF FF FF FF FF
83 FF FF FF FF FF FF FF FF
84 FF FF FF FF FF FF FF FF
85 FF FF FF FF FF FF FF FF
86 FF FF FF FF FF FF FF FF
87 FF FF FF FF FF FF FF FF
88 FF FF FF FF FF FF FF FF
89 FF FF FF FF FF FF FF FF
90 FF FF FF FF FF FF FF FF
91 FF FF FF FF FF FF FF FF
92 FF FF FF FF FF FF FF FF
93 FF FF FF FF FF FF FF FF
94 FF FF FF FF FF FF FF FF
95 FF FF FF FF FF FF FF FF
96 FF FF FF FF FF FF FF FF
97 FF FF FF FF FF FF FF FF
98 FF FF FF FF FF FF FF FF
99 FF FF FF FF FF FF FF FF
So for example, if you wanted Serge to unlock the most upper right slot in his grid at star level 1, after defeating the Komodo dragon boss, you'd change the first byte on line 2 from 0x07 to 0x48.  Like this:

Code: [Select]

1 48 07 07 FF 08 08 08 07
That's the short of it, but if anyone knows how the DMA stuff works, I'd love a pointer.  no$psx does spit out some debug TTY messages for when the DMA transfers happen and which interrupts get set, I just don't know how to interpret it yet.  Getting a little further here will help me figure out where the table comes from, and hopefully mean we can more easily modify it. 

Edit: Oh and I've compiled this info in a google doc that has a little better formatting.  It can be found here-> Growth Table

[glennxserge]

OK, I've been able to walk this growth table back to the source on disc.  It lives 0x0016A750 on disc one, and is a lzss compressed file.  Which is why it's been so hard to trace this through a debugger.  The decompression routine made it very hard to find.  The file header indicates the uncompressed size should be 800 bytes, which matches the growth table size.  I was able to pinpoint a literal in that file that aligned with star level 2's slot and was able to change it to unlock any position on the grid.  With proper decompression/compression insertion I think it would be trivial to adjust these. 

At this point, I guess I've answered the question I had which is 'how does element level growth work?'; turns out it's defined in a lookup table.  I was hoping for something simpler edit in place for modding, but I'm satisfied with having an answer.

[Grobycftw]

Amazing discovery Glennxserge!! Props for figuring it out and giving all the information for it!
I will give it a try once I can get back to modding

Thanks for sharing!   

[glennxserge]

Thanks, Grobycftw!  Appreciate the extra help along the way.  With element growth out of the way, I was thinking of helping with the fieldscript unknowns.  I'll probably have more questions for you, but I'll leave them in the other thread.