Hi everyone! Lisy here, producer at LoG. The highest pillar of our production process here at LoG is stability. Build stability takes precedence and priority over every single decision we make at LoG. And, yep! You’re not imagining things, the February update of Armello’s Road to 2.0 is *very* unstable. We came in to see this this morning…
And then, upon further investigation, this...
Since releasing the February update we’ve seen a sharp increase of NMA’s (instances when the game does not crash, but “No More Actions” are possible) and Crashes affecting around 16% of games played. This is something we’re taking very seriously and obviously the entire team is truly devastated about it. With stability being LoG’s highest pillar, we have decided to temporarily roll back Armello to v1.9.2. What you need to know (The TL:DR):

• We have rolled the February update back to v1.9.2 ‘From Below’.
• We’ll be working to re-release the February update over the upcoming week. We’ve shifted Karl (lead developer) and Tom (tech director) onto this and only this and QA internal resources will be tripled until the update is out again.
• A community testing round will be held when the build is ready.
• The Lunar Dice event is currently suspended, but will return within the next 3 days.
• As a result of this, Lunar Dice assets have been removed from the game.
• They will remain in your inventories and return as soon as the event is live again.
• Upon its return, we will extend the Lunar Dice event to account for any down time (and then some!).
• As a gesture of goodwill, we will also activate double drops for the entirety of the Lunar Dice Event.

Why did all this happen? Okay, so strangely, here’s where the good news is... the February update had some very extensive AI changes hidden in the build. Major AI improvements were to be one of our 2.0 key features that we would announced later this year. Surprise! :/ We’ve been working on the AI overhaul for a while now and the February update was the first build that had these changes merged in. While these features are not publicly visible or ready to be made “active” in the build, the code was still affected by changes made to support these new features, and apparently waiting to wreak havoc on the live build. As part of our release process we have full analytics on test builds, three in-house QA folks manually testing the build, automated testing seeking out errors, crashes and NMAs (by playing four multiplayer games at 10x speed all day and night, non-stop!) before builds are approved for release. We intermittently run community testing, and yes, in hindsight it may have proven valuable for this release. However, time and resources leading up to the release of the February update did not permit us to coordinate this option. Additionally, the particular crashes and NMAs occurring in the February update slipped through all above QA measures and realistically, the brute force “testing” provided when the game is live (approx. 10,000 games played per day) is the only thing that would have alerted us to these issues. What are we doing to prevent this in future releases:

• Conduct a post-mortem of our QA and production processes to understand what learnings there are for future releases.
• Analysing each NMA/crash and refining our manual and automated test cases to be able to identify these particular bugs.
• Redesigning our community testing methods to enable a greater diversity of machines, players and network environments, so that we can employ community testing rounds for every release with maximum efficiency and effectiveness.
• Armello will never be updated again. (lol, jk. It’s gonna be updated a LOT.)

Unfortunately, we cannot promise that we will completely eliminate any chance of this happening again. Gamedev is chaotic and Armello is a hyper complex game. BUT what we can promise is that we’ll always respond to these situations as fast as possible, with your best interests in mind, to get you back into Armello with minimal delay. Thank you all for sending through your bug reports and forum posts, they truly are invaluable and we really appreciate everyone’s patience during this release. We’ll get the February update and Lunar Dice event back online ASAP. Hopefully this transparency into our processes can give you a little insight into our efforts in making this happen. If you do have any concerns or questions, feel free to reach out to admin[at]leagueofgeeks.com! We will be posting more information as soon as we’re able to! Happy gaming folks! See you in the February update (again, soon!) <3 Lisy and the LoG Team

[ 2018-02-16 01:37:39 CET ] [ Original post ]