Friday News 2 - Solar Systems, Network History Some helpful notes ECS: Entity Component System. In the context of Lavender, this is unity's new ECS system based on fixed memory data first design. Its closer to how old MMOs were made rather than modern object oriented games. Instance: A single world that you the player run around in, which is different from the server as a whole. Think MMO again, shards, instances. The gameplay part where the physical players and props live in. Version numbers: Ive always disliked fake version numbers products use like, 0.4, 1.3.22. Early on we switched from this over to just displaying the raw revision of the projects master branch. Unfortunately weve shifted the project from one repository to another and the numbers reset, which is pretty confusing. The project started on revision r875, had thousands of versions up to r4565, then we ported to a new repo starting over at r1. We are currently at r428 as of writing. Eventually we want to change our version numbering, Were just not sure what yet. Abstract : June Network is one of my favorite topics when it comes to game design. Its both high level and low level. Hand crafting packets and bit compressing where needed. While also having to manage the complexity and abstracting this away enough to actually get the magic synced up between two game worlds. In this space, there isn't really one single right way to do networking either. Pretty much everything is a trade off when it comes to networking. Mesh vs Hub spoke, hybrid mesh, host transitions on disconnect, lobbys, it goes on and on. Cheating is a major concern, which has become very apparent these last few years. There's just so much room for experimentation and learning when it comes to it. I wanted to give an overview of the networking within lavender, how much its changed over the years, due to the game changing. May 2018 - Lavender prototype is added to our SVN Repo, cloud projects added to TFS
When we first started the project our decisions were based around a totally different game. Based solely around VR social interactions, with complex interaction (cars, planes, guns, scripting), This version of the game had much different networking needs. Not to mention we were planning on a different monetization model at the time, subscription based for upgraded accounts (extra slots, data caps removed). Since the users just needed to get in and see each other quickly, we planned on using a cloud provider, like unity multiplayer (now dead and a new version replacing it) or photon - whatever it was at the time. As you know from our current version, this didn't pan out and never even really made it into the game, only a few internal dev builds had access to spinning up cloud servers. October 2018 - Cloud based simple servers killed (Photon, Unity Multiplayer) With cloud based servers now dead and gone, we had to come up with a new game design and networking. Single server, multiple instances, basically reusing the concepts and tech we built for the cloud. But cleaning it up and handing it to users. This is when the concept of Galaxy, Solar System and worlds were created. Since we were shifting to a server that does more, why not shift the focus of the game a bit to reflect the change from the cloud. So we did, deciding to plan around a garrys mod like sandbox experience, but retain all social features from before. The solar system plan at this time was to create a process manager, with a web api for talking to our API and a local one, for direct communication with the game instances running on it. This process would have to both manage the users requests, which it would then forward to the cloud. Along with managing the CDN requests and keeping a central cache per solar system install (dont want to download the same map for every single spawned instance). November 2018 r1134 - ECS added to project Around this time, unity releases an early teaser for a project theyve been working on called ECS. During some time off I made a prototype on it, creating a Terraria clone. It was fantastic, and such a different way of thinking about game design solutions. I could already tell that if we ported our networking over to this, we could remove most of the crappy parts of networking while still being as easy to work with. There were other reasons for bringing ECS in, but this post is focused on networking. December 2018 r1795 Ported and reworking core networking to ECS. Armed with this knowledge we discussed this with the community in the discord (which was much smaller). They all agreed I should do it, to take my time and do it right. This by far is one of the best decisions during the project, Ive personally learned so much because of this. Thank you. The next year is spent struggling to learn, build and maintain this new system. It worked roughly like this. A single unity scene is loaded, the map you selected. And a single ECS world exists containing a bunch of entities with flat data. Stuff like, heres a player made up of [PlayerID] [PlayerEntity] [SkeletonBuffer]. Players bodys were all processed in multi threaded jobs that uncompresses, unpacks and blends (interpolated) their poses. To perform any action or logic on the server, you would create an entity with data on it representing your request. Example: [SpawnProp] which has a position, playerid who requested it and a galaxy id to actually download the prop. The spawning prop system would then look at all requests and process them, checking if the user was at their prop limit, or disallowed from spawning props on this server. This inversion of logic all revolving around looking at the presence of entities is how it was all built.
ECS Networking finally working after the initial rewrite in 2018. But things were still far from ideal, to local host a server (be both a client playing the game and hosting for you and other people) we had to come up with a way to isolate these two worlds. The simplest solution would have been to spawn two ECS worlds, which was barely doable at the time. The real problem came with unity scenes and global physics. There wasnt a way to isolate either of these and a player would self collide with the server version of himself, or a prop might fly around doing the same. There ended up being lots of complexity from the server and client being the same world, same scene. Modern times We were still not even remotely close to having the complex physics and scripting interaction we decided on ages ago. Fast forward to a month ago, Unity based split worlds became possible. I'm not doing well at this time and ask for advice. Can I take the time to rip it all up again? I hadnt made as much progress at the game for months at this point, I had spent all this time reducing and increasing the networking complexity of the single world system, waiting on solar systems, to provide motivation and direction for what the game needed. We decided since this would both simplify the entire networking stack, along with allow us to finally kill the solar system bogeyman, I should go for it. Entire stack was upended, there are now multiple worlds per server, each being a unity Scene and a ECS world pair. Networking stack is split into 2 layers and a global manager, * Network Drivers, manage connections, raw data flow in and hand it off to the next layer * Solar Server|Client The high level logic that manages the drivers even flow converting their data into entities and placing within the right instance. * SolarSystem Creates either a Server or Client and tells the server if running to create world instances if asked. And as of this most recent build, this is done, with support for Steam friends, direct IP and local hosting all going though the solar system and solar clients. This is a big deal internally, as it means we can shift our focus from networking, to interaction and gameplay scripting.
Two very happy cube players joining through direct IP, and the other through steam relay servers. The solar systems are now working in their most basic version but still have a few more things they will need before the next release. Solar Systems need to register their presence to a galaxy api somewhere to display and find users easier, users need to be able to see all servers running out there to find friends to play with. They also need the old web api from years ago to be brought back, so you can create worlds from a web api for power users.
This was from early 2018, feels like forever ago. In a future blog post Im gonna go through the hall of shame and it's always next week and go though the UIs history leading up to the new UIs proper reveal. That post will have to wait until the main style is settled on and most of the UI being finished. We would love to hear from you, consider joining our discord, or opening a new post on the steam community. If there are any suggestions for a future blog post topic we would love to hear it. Thanks for your support - Lavender Team