I'm working on an entirely new system for saving and syncing conquest games. It's currently in beta, and it's not certain that it will make it into the game yet, but I thought you might enjoy a dive into the why and how of it. So yeah, this is a pretty technical post you can just skip if you're not into programming. tl;dr is: I'm doing things to make the game go faster. There's two closely linked things here: saving the game, and calculating checksums to make sure that multiplayer games haven't desynced. Desyncing is what happens when the realities of two players in an MP game diverge, which is of course bad. Being able to detect it helps me fix bugs that cause the game to desync. Long-term, I hope to also add in functionality for desyncs to heal automatically. Currently, both save and sync are very simple: the entire game world - map, empires, ships, landscapes - is converted to the JSON data format, basically one long blob of text. When saving, that blob of text is written as a file to disk. When syncing, the game calculates a checksum of the text and compares it to other players' checksums to make sure they're in sync. But that's a lot of data, especially in the later stages on a large map. Dozens of megabytes. And of course the game world can't change while it's being saved, and so when the game syncs or autosaves, there's a noticeable pause. Sometimes, a very noticeable pause that causes your multiplayer game to steadily fall behind, because it can't keep up. The new save system does two things to fix this, both a bit fiddly:
Lazy Saving
Most of the game world doesn't actually change between saves. A ship that's just flying around without engaging in combat doesn't change. The landscapes of peaceful towns, with nothing crashing into them, also don't change. So re-creating all the JSON data for them isn't actually necessary. By splitting the save game into a bunch of smaller files, one for each landscape and ship, the game can avoid doing most of the work of saving. The hard part is that it has to know when a landscape or ship did change, which means that these things now need to keep track of a version number. That version number needs to be increased whenever a change happens. If not, the save will be partially outdated. Conveniently, I realised that during development and testing, the game can produce the JSON for things that are supposedly unchanged, and compare it with the information that's in the file on disk to see if it really hasn't changed. This lets me root out cases where the version isn't getting updated.
A Compact Format
JSON is a convenient format, but it's also very bulky. It's text-based and each data field is individually labelled, which makes it human-readable but verbose. Completely changing the data format of the game would be a huge effort. Instead, I created a more compact representation of JSON. (There's probably a lot of compact JSON representations out there, but this one is mine.) There's a bunch of minor gains to be made. For example, the number 9845329 takes seven bytes to store in JSON but only four if you store it as an integer number. A small number like 109 fits into one byte. The major gain is not having to repeat all the names of the fields. For example, each crew member on a ship stores a "weaponReload" value. On a ship with 100 crew, this means the letters "weaponReload" are repeated 100 times, which is 1200 bytes just for that. But in a compact format, you can just use "weaponReload" once and then refer back to that text using a back-reference that uses far fewer bytes. Because most of a JSON file's size is made up of field names, the total size of this new format is a fraction of the old one. A back-reference here means that instead of spelling out the text you keep track of all the bits of text you used previously, and then refer to them by number when you need them again. For example the following sequence of words: fruitbat, fruitbat, llama, fruitbat, iguana, squirrel, llama can be abbreviated to: fruitbat, 1, llama, 1, iguana, squirrel, 3 Where the numbers represent the position in the list where a word was first encountered. What makes changing formats hard is that it needs to have 1:1 fidelity to JSON. You should be able to convert any JSON into this format and back again without losing or distorting any information. But wait, why do we care about a compact format? We want a fast format! Well, conveniently those are very similar goals. Having to glue together far fewer bytes in less complicated ways makes this format faster. And having to write fewer bytes to disk also makes saving faster.
Results
So by combining these two techniques, saving and syncing will take two orders of magnitude less time. It's more work, but it's also necessary. You might be wondering why I'm messing around with save formats instead of working on the diplomacy update, but the way I see it, I can't do a big update and tell everyone to play multiplayer games when there's problems like multi-second sync pauses still happening.
Airships: Conquer the Skies
David Stark
David Stark
2018-08-16
Action Indie Strategy Singleplayer Multiplayer
Game News Posts 306
🎹🖱️Keyboard + Mouse
Overwhelmingly Positive
(5250 reviews)
http://www.zarkonnen.com/airships/
https://store.steampowered.com/app/342560 
The Game includes VR Support
Airships Linux 53 [139.2 M]Airships Linux 64 [545.79 M]
Airships: Conquer the Skies - Soundtrack
Airships: Heroes and Villains
In the game, ships are viewed side-on, and their modules are operated by individual crew members. During combat, players give high-level commands to a small fleet, positioning their ships, ramming and boarding others. Ships and terrain are fully destructible: they can catch fire, explode, break apart, and fall. Players can also compete against one another in Internet and LAN matches.
The ships are highly detailed, teeming with sailors moving around at their individual tasks, like an ant farm or a cut-away drawing. The player's choices in ship layout are crucial, and an important part of the game is exploring the design space of different airships and their matching tactics.
- OS: Ubuntu/Debian/Mint
- Processor: 1.8 Ghz+Memory: 256 MB RAM
- Memory: 256 MB RAM
- Graphics: 1 GB VRAM+
- Storage: 1 GB available spaceAdditional Notes: Not currently compatible with Intel HD graphics controllers. May run on other Linux distros. but no guarantees.
- OS: Ubuntu/Debian/Mint
- Processor: 2.2Ghz+ Dual-coreMemory: 2 GB RAM
- Memory: 2 GB RAM
- Graphics: 2 GB VRAM+Network: Broadband Internet connection
- Storage: 1 GB available spaceAdditional Notes: Not currently compatible with Intel HD graphics controllers. May run on other Linux distros. but no guarantees.
[ 5166 ]
[ 1903 ]