TotK - restore destructible rocks

[naikrovek]

I am a weirdo and I very much enjoy destroying those one-time destructible rock found in caves.

Without starting a new save, I want to find all of those things and destroy them all again.

I can't read JavaScript very well, so I don't know if your data structure can see those rocks or modify them, but if it can, please offer a button which restores those things, and another to remove them all as if they have all been destroyed.

Many thanks.

[DeftTitan]

I haven't found any entry in the master editor which restores rocks. I even created a save game prior to destroying a wall, and one right afterwards, and compared them for differences. there were differences, but I couldn't undo the rock wall damage with the editor. there may be a way but i'm not sure how it's tracked.

[naikrovek]

I did the same test and I couldn't find the things in the master editor, either. The master editor only shows stuff whose "hashes" are known. Those are listed in the .csv in the root of the TotK folder of this repo. find the correct hash for the start of the destructible rock data, or its address in the save file, and the format can be worked out and the values can be shown and edited in the master editor.

Maybe I'll get it stuck into my head one day that I'll figure this out, and work on it. Until then, I'll keep waiting.

[marcrobledo]

You won't find these in the Master Editor, as they are not variables.

In TOTK, things like specific enemies defeated, Addison signposts, obtained depths' pristine weapons and probably even destroyable rocks are stored in the MetaData block, which is usually stored at the end of the savegame file.
It is a sortered array that contains all Uint64 ids for those bits mentioned above. Its hash is a3db7114 in the savegame file and its value is a pointer to that data structure.

I wish I had time to work on an editor for this :-(

[DeftTitan]

What you said was helpful. I'm going to crack into the existing code for the addison posts and pristine weapons on the load page and see if I can figure out how to parse the metadata block.
I am interested in being able to mark certain treasure chests as opened, etc.

[marcrobledo]

The idea would be to do an editor that allows to add or remove things to the metadata. This map has most of the objects and their ids documented.

[naikrovek]

what does "hash" mean in this context? It's not a location in the save file, is it a magic word to look for in the file?

[naikrovek]

what are the IDs for the one-time-destructible rocks? I can't seem to figure that out. MergedActorD100 seems to be it, but i'm not confident, and I wonder if this name is not used only for this purpose. ¯\_(ツ)_/¯

[naikrovek]

Thank you for this editor, by the way, @marcrobledo. This is fun.

[marcrobledo]

Try to search AutoGenerateDestructibleActor* OR WallCrack OR RockBroken here.

[naikrovek]

I found the hash you mentioned via a hex editor, but I'm not sure where to go next. I've reverse-engineered file formats before, so I'm not completely useless here.

Try to search AutoGenerateDestructibleActor* OR WallCrack OR RockBroken here.

Are you saying that checking (or unchecking) these in the master editor will turn these rocks back on?

You won't find these in the Master Editor, as they are not variables.

In TOTK, things like specific enemies defeated, Addison signposts, obtained depths' pristine weapons and probably even destroyable rocks are stored in the MetaData block, which is usually stored at the end of the savegame file. It is a sortered array that contains all Uint64 ids for those bits mentioned above. Its hash is a3db7114 in the savegame file and its value is a pointer to that data structure.

I found the hash in a hex editor, as mentioned, but I don't know the width of the value that comes after it. Endianness messes me up after I don't think about it for a while. I found 14 71 DB A3, anyway.

My hex editor shows the next 8 bytes as 10 00 11 00 00 04 00 00 and after making the same endianness stuff that's 04 00 00 11 00 11 which is not a valid location in the save file.

Can you give me a short clue about what I'm doing wrong? I don't need a tutorial or anything, just some idea about how to read the value after the 14 71 DB A3 magic number.

Edit: I figured some of it out. The pointer is 32-bit and in this case it's 0x00110010. I don't know if the next 4 bytes are a length or not. I would specify a length here, if the length isn't predefined, and I don't know if the TotK save file ever changes size. I'm thinking it doesn't, but I don't know.

[marcrobledo]

I didn't say the MetaData contains the bits you are looking for. I said probably ;-)

The next four bytes after the MetaData hash is the offset to the MetaData. MetaData is just a list of Uid64 (specific killed enemies, Addison fixed posts, etc). You can see the Uid64 for every game actor in the map I posted.

But as I said, I can't confirm that MetaData contains the broken rocks. Could be it's just a boolean you can edit in the Master Editor. Have you tried to edit those AutoGenerateDestructibleActor* in the Master Editor? One easy way to find where the bytes we need to edit are is to save in front of a destroyable rock. Then destroy it and make another save, and compare with a binary file comparator.

Chances are, they are just in a boolean array. If that's the case, then I could easily add a button to restore/destroy all rocks.

[naikrovek]

That's enough of a clue for me. I should be able to tell you if the rock data is there or not with a bit of time.

[naikrovek]

oh, yes I've tried unchecking all the boxes in the master editor for AutoGenerateDestructibleActor* but it made no difference in anything I could see. The rocks I left unbroken were still there, and rocks that I knew I had destroyed did not reappear.

I did use the browser JS console to do the unchecking of all those checkboxes, though, maybe that prevented it from actually making the changes. I didn't think of that until just now.

[marcrobledo]

I did use the browser JS console to do the unchecking of all those checkboxes, though, maybe that prevented it from actually making the changes. I didn't think of that until just now.

Yeah, that's probably what happened.

[naikrovek]

I did use the browser JS console to do the unchecking of all those checkboxes, though, maybe that prevented it from actually making the changes. I didn't think of that until just now.

Yeah, that's probably what happened.

ok so do I need to click several thousand checkboxes, or is there a better way? 🙃

[marcrobledo]

I did use the browser JS console to do the unchecking of all those checkboxes, though, maybe that prevented it from actually making the changes. I didn't think of that until just now.

Yeah, that's probably what happened.

ok so do I need to click several thousand checkboxes, or is there a better way? 🙃

You could use jQuery (well, Cash in this case) for that:
$('#table input[type=checkbox]:checked').trigger('change').prop('checked', false);

[naikrovek]

OK, I found a single node in the object map, then I found it in the editor. That single node in the map is 1024 checkboxes in your editor. It took 5+ seconds to check each box, if I searched only for AutoGenerateDesctructibleActor* but narrowing it to the exact object sped things up dramatically.

I checked every unchecked box which was a part of that object, and it made no difference in-game. I also unchecked all 1024 of these, which also made no difference in-game.

I'm doing something wrong, or this isn't how these rocks get restored.

Later, after work, I will load up a new save file and see which of the 1024 checkboxes are checked and unchecked for this same OID and see what it shows.

[marcrobledo]

$('#table input[type=checkbox]:checked').trigger('change').prop('checked', false);

That didn't work as the events were not attached with Cash, so the trigger doesn't do anything. Try this:

$('#table input[type=checkbox]:checked').prop('checked', false).each(function(i, elem){
	var offset = this.offset;
	offset += Math.floor(this.arrayIndex / 8);
	var fullByte = tempFile.readU8(offset);
	var bitMask = 1 << (this.arrayIndex % 8);
	if (this.checked)
		fullByte |= bitMask;
	else
		fullByte &= ((~bitMask & 0xff) >>> 0);
	tempFile.writeU8(offset, fullByte);
});

or this if you want to check them all:

$('#table input[type=checkbox]:not(:checked)').prop('checked', true).each(function(i, elem){
	var offset = this.offset;
	offset += Math.floor(this.arrayIndex / 8);
	var fullByte = tempFile.readU8(offset);
	var bitMask = 1 << (this.arrayIndex % 8);
	if (this.checked)
		fullByte |= bitMask;
	else
		fullByte &= ((~bitMask & 0xff) >>> 0);
	tempFile.writeU8(offset, fullByte);
});

I'm doing something wrong, or this isn't how these rocks get restored.

Never said this was going to work ;-)

Still, I don't know why are you taking this route. It would be much easier to do a binary comparation between two savegame files with a bunch of rocks destroyed in-game.

[naikrovek]

I'll get there. I'll be able to see what changed, but knowing that won't help me understand the data format so I can change them all.

I can't find the few object IDs I looked up for these things in the save file at all, which confuses me, as well. Could be LE to BE conversion, or vice versa, I don't know, yet.

[naikrovek]

These 1024 checkboxes per AutoGenerateDestructibleActor*... these are exposed in your editor as checkboxes, but are they really two int64 numbers?

I don't know why are you taking this route.

Because you mentioned that this data was (at least partially) exposed in the Master Editor. Maybe what is needed is already there. You probably already know, but I don't. I'm trying to save you time by testing things (and failing horribly, lol).

I've never seen a file format with this many magic numbers and pointers to data structures. This must be a game save file convention or something. Most file formats that I've looked into have been plain TLV formats with a single magic number. This save file format is filled with magic numbers, called "hashes" here, followed by pointers or offsets to the real data, which is very odd, to me. These things seem like obfuscation, to me, because these things are certainly getting in my way of understanding how this file is laid out.

If I spoke JavaScript I would have a much better time understanding how this is working, but I don't. Like I said, I'll get there.

[marcrobledo]

Internally, all AutoGenerateDestructibleActor* variables in TOTK are bit arrays. Since they are bigger than four bytes, they need to be stored somewhere else ouside the hash table. So, instead of storing the bit array itself next to the hash key, it stores a pointer that points to the bit array which is stored after the hash table. All array types and other special types like coordinates are stored that way in TOTK.

Anyway, those AutoGenerateDestructibleActor* bit arrays specifically are 1024 long (1024 / 8 = 128 bytes) for some reason. Could be it's just a filler for all rock generators, even if they generate less than 1024 rocks.
Chances are a single bit belongs to a single rock, but this needs confirmation of course! So, every checkbox in the editor might belong to a bit (not a byte!).

As of the file format... Data in BOTW and TOTK is saved in a hash table :-) It's not obfuscation, since they could have just encrypted everything for security purposes and they didn't, it's all plain data. Indeed, TOTK savegame file is much simpler than BOTW's.