Hello all, and welcome to the November Multiplayer update! It has not gone unnoticed that our last update was fairly light, and that we skipped our October update. I would love to take some time to break down some of the reasons why and share as much visibility and clarity on our ongoing Multiplayer work.
I'd like to make reference to something Judd discussed a bit more in-depth in our July Road to 1.0 Update: "I cannot overstate how important it is to EHG that multiplayer is executed correctly. A bad initial multiplayer experience can harm the game for all players and jeopardize the success and reputation of the studio long term, much more than holding it until its ready. We want multiplayer to be enjoyable when we first release it without confusion or game-breaking bugs, even in early access. Several games with many more resources have failed to stick the landing on multiplayer, so were making sure to learn everything we can from those mistakes and not to repeat them. While we still may make some mistakes, well be able to react and iterate on them quickly." Through further testing of our scalability of multiplayer services over the last few months, and in the light of other recent major game launches, we recognized some core areas of our current Multiplayer experience that we would have to re-address. We wanted these to have a stable gameplay experience before we roll out even a multiplayer alpha test. We realized that some of the services we were relying on were things we should ideally build and own ourselves if we want to be able to reduce live problems as much as possible, as well as fix things quickly whenever we need to. To put it as clearly as I can - we've experienced some setbacks and delays because we made the choice to improve a number of our services, which we believe is the right decision for the long-term health of Multiplayer. **Our intended Community Testing program for Multiplayer has moved to Q1 2022.** The result here is that we have chosen to migrate a few of our features, where some of the other features we've now built ourselves. This has meant our previously intended Community Testing phase of multiplayer has been delayed until we're confident in the Alpha version of a number of these changes. Let me break it down visually with a quick run-down: The core multiplayer gameplay of Last Epoch is functioning very well - we playtest every week without any issues and are making a huge number of gameplay changes to consider the impact of the multiplayer version of Last Epoch. Let me cover some of this visually:

Party

Our multiplayer party system has been working well and we've been using it for about 8 months now. We needed to migrate it, however, as the party system relies on our account system which we are moving. We have recently just completed this and are testing it.

Friends

Very similar to the party system, it works online the same as the one you have now, but we also have to migrate it. This has been completed and is in testing.

Gameplay

Everything is looking great here and nothing is impacted for online gameplay. We have a lot of work to go optimizing it and we'll have to make some changes to how the game saves data since we are moving the character saving system. We are still implementing the Monolith in an online format but we regularly play the rest of Epoch together online, and have continued to work on systems such as latency compensation, as well as other elements we need to make the experience feel great.

Characters

The service we were relying on for character saves was fairly reliable at the scale we were testing at, but as we have continued to grow our playtests it increasingly became clear that this wouldn't be sufficient if we had a large influx of players. Migrating this to a more robust service has been a priority and we've been able to make gains here quickly by moving to an industry standard platform.

Deployment

This is our ability to deploy patches quickly to a global audience - nobody wants to wait 7 days for a patch. We have actually been really happy with how fast we can deploy the game online - we were at least as quick as other similar games. However, because we migrated our server hosting solutions, we have to modify a lot of the scripts and tools we use to deploy the game both internally and externally. This work is ongoing but it is something we have a huge amount of experience with at this point, we are confident it will be quick by the time we move to CT testing.

Users / Game Access

This is the first of our main set of changes. We consider this of extreme importance as the integrity of the online experience will be critical to have a competitive ladder, as well as ensuring we can maintain a great community by ensuring bad actors cannot access the game again. We are actively moving towards migrating our account system to work within an industry standard provider - work here is still ongoing and we will want this before releasing out a MP Alpha to testers.

Chat

Another area where we experienced some issues even in the single player game over the course of this year - there was more than one occasion in which we had to disable our chat. We recognized it would not scale in a way that would make us happy. We also had a number of issues this year with chat moderation, which our current chat system has not made easy to deal with. This is now actively in development using a third-party chat provider that should safely support us scaling from our current numbers up to launch. We expect this to be in when we also have our new user system for community testing.

Login

The second of our main set of changes - an inability to login is one of the worst things that can happen for us when we go live. We know we need this critical piece of infrastructure needs to keep up with however many people decide to play at once. We're feeling good that our new provider is up to the job and should easily scale at the numbers we anticipate. We're keeping an eye on the need for a login queue, but have no immediate plans to implement one at the moment. We expect this to be in when we also have our new user system for community testing.

Game Server / Matchmaking

This is the last of our main set of changes. Adapting a general purpose matchmaker to our scene allocation needs proved to be quite challenging. We need to be able to quickly allocate game servers to handle the wide array of scenes players will be in, while keeping our server load manageable. We have successfully moved to a new platform, are regularly testing with it, and no longer have to rely on a complex matchmaker. We are really happy with our changes here but it took a few months.

Final Thoughts

One thing I'd also like to touch on a bit is our regular release of the single player version of the game. Previously we were a much smaller team, so it was often the case that we had to divide our attention between continuing to develop, test, release, and fix our single player game. Our 3 month content process is really critical for us to continue to fund the ongoing develop of Last Epoch, as well as make big changes for our current players, so this had a lot of impact on our ability to remain consistent. It is not common to build out a multiplayer version of a game while consistently releasing a single player version - but considering that the development of Last Epoch has been financed by early access, it's a challenge we've had to rise up to. With the growth of our team, it takes time to onboard new developers into our processes. We are starting to cross this threshold with new developers that have joined the team over the last year, and are adding even more developers to resolve this conflict. Over the course of the next period of time, we are looking to merge our single player and multiplayer processes so we can put the entirety of our team on the multiplayer release of Last Epoch.
I hope this information has helped clear up the air about all the work we're putting into the development of Last Epoch multiplayer to ensure that when it releases, we do it right. We have learned a lot and are continuing to learn as we develop the platform we need to succeed now and far into the future. I know that the best time to release multiplayer is yesterday, but moving forward we're going to be a lot happier with some of the things we've rebuilt. As a final thing, we're still hiring for all kinds of roles! Check out our postings at https://eleventhhour.games/careers for more information.

[ 2021-11-30 20:48:01 CET ] [ Original post ]