Adds save data compression with custom algorithm.
Original SugarCube had one too (LZString), but it was disabled years ago because of performance reasons.
Differences between this algorithm and SugarCube original compression:
This one's compression rate is down to ~60% of the original size. Compression and decompression time is ~3-7 milliseconds.
Naturally, compressed saves are not compatible with older DoL versions. A
Error: the passage "This save is compressed and is not compatible with old versions of Degrees of Lewdity. If you want to load this save in an older game build, use exporting." does not exist
message is displayed when attempting to load them.
Exported saves are not compressed with my algorithm - they are compressed with SugarCube's LZString (always have been). Therefore, they can be loaded in old versions.
For maximum safety, the compressed save is validated by decompressing it and comparing to the original data; if it's different, an error message is displayed and the save is written uncompressed. (That validation step effectively doubles the compression time, and can be disabled in the code).
All unique keys and values are put into a dictionary (an array).
Then these keys and values are replaced with indices in that array - repeated values take less space.
Objects are converted into arrays with a special marker - this shaves off the "
s, saving ~10% of space.
And dictionary can be pre-initialized with DoL variable names - so they are never written in the save data, saving another ~20%.
The dictionary (contained in game/00-framework-tools/03-compression/dictionaries.js) should contain most common strings and numbers that appear as keys and values in the save data. I've initialized it with variable keys and sub-keys I had in my save. If we add common values here (like NPC and clothing names and states) we can shave off extra 5-10%.
Without the dictionary, compression is to ~80% of the original size.
[]
[0, ...values]
(values are compressed)[1, key1, value1, key2, value2, ...]
(keys and values are compressed)compressed:1
- indicator that this is a compressed objectvalues
- the dictionary, skipping the well-known valuesdata
- the compressed object itselfLive demo (of a little outdated version) is available here: https://jsfiddle.net/aimozg/Lmnwtv2y/. It doesn't have the DoL dictionary.
Compressing input = {0:true, 1:false, 2:null}
with wellKnownvalues = [0, 1, true]
.
The result is
{
"compressed":1,
"values": [false, 2, null],
"data": [1, 0, 2, 1, 3, 4, 5]}
}