Hello, Travelers! Its officially the start of Patch week, with only four more days to go until 0.9! In todays blog post, were going to be getting a bit technical. Performance and Optimization has been a major goal for us over the last year and something weve made a lot of improvements regarding. We also know this is a very important topic for many users, so we want to talk about all the things weve been working on. In short form, Performance coming to patch 0.9 has improved considerably, and well continue to improve it! If youre interested in the extended version with how were accomplishing that? Well, lets dive into that. Over the years, our team has been busy building a content-rich and fun game. With regular updates, we provided a lot of new content often. While we always had performance in mind and tried to keep performance in check or improve it over time, we were never as focused on improving performance as we have been and continue to do so. Many things affect the performance of the game, and today we would like to share more about the visual effects optimization improvements we have made for patch 0.9.0 and beyond. Visual effects (VFX) are present in every corner of the game. If you are casting fireball, throwing magical hammers, getting hit by void enemies, seeing flickering torches in the dark dungeons, or just standing in the rain, there are almost always several visual effects playing on your monitor. Over the years, weve added thousands of visual effects to the game, and as we are now also adding multiplayer functionality, we understand we need to have the performance in a good place, and VFX can and does affect the performance in moment-to-moment gameplay. We have been going through every single visual effect we have in the game, benchmarking it, and making sure it is up to our standards both from visual and performance perspectives.

Runtime Instantiation and Pooling

Every time an ability is used by the player or an enemy, the game has to create and initialize a visual effect associated with that ability in addition to other effects, such as hit effects. This Instantiation and initialization can be quite performance costly. Multiple things can cause players to experience more and more stuttering, such as the number of enemies increasing, players using skills more often, or those skills spawning more projectiles and effects. The solution to this problem is Pooling. The game creates all the effects we expect that will appear in the scene in advance, as such, the majority of the instantiation and initialization costs will be paid during the loading, and these effects will exist in the scene - in the pool but are disabled. When the game needs to display the requested visual effect, instead of creating a new one, writing it into memory, and initializing it, it will just take one from the pool and play it instead. After the visual effect is played, it will return to the pool and will be reused later. This solution reduces stuttering caused by VFX creation considerably. For patch 0.9.0, all player abilities VFX will be pooled. In addition to player abilities VFX, we are also pooling the most frequent visual effects - such as hit effects or ailments. We will also be continuing to add more visual effects to the pool.

Shaders, Overdraw, and Rendering Paths

Another aspect we are looking into is optimizing our shaders. A shader is an underlying code that decides the color of the pixel appearing on your monitor. Pretty much every pixel is calculated on the shader level, no matter if we are talking about UI elements, terrain, models, or visual effects. For VFX, we are using a wide variety of shaders to achieve different visuals - a smoke effect is using a different shader than liquid or blood, for example. Some of these shaders can be quite simple, but sometimes calculations to achieve a very specific look can be incredibly complex. Using our new internal tools, we identified several performance issues caused by our previous shader configurations. Since 0.8.5, we have been working on adjusting and even re-creating our shaders to improve from these issues resulting in a smoother and more performant experience.

Overdraw

Overdraw is when a pixel on the screen has to be drawn multiple times to achieve the final result - ideally, you want the color of the pixel calculated just once, but it is not always possible. This is mostly caused by transparent materials and effects. If you have a nice semi-transparent smoke effect, the engine needs to know what is behind the smoke to calculate the final color of the pixel, so in this case, everything that is behind the smoke needs to be calculated at least twice. If multiple particles or particle systems are using transparent materials, you can see how this will start to stack up. Overdraw is not an easy issue to fix, as well some level of overdraw is typical in modern games, but there are some options to mitigate the impact of overdraw or reduce the amount of it. to achieve this, we developed internal tools to measure the amount of overdraw caused by our visual effects. We have been going through all of our visual effects, searching for patterns or systemic issues we could fix by adjusting the shader or manually fixing the biggest offenders. We identified specific shaders which do not benefit from being transparent. Instead, we are converting them to something called alpha-test. Alpha test shader pixels are either fully visible or fully invisible. This allows the game engine to calculate the pixel color just once. This primarily applies to liquid/goo/blood effects with minimal or no visual impact. For cases where reworking the shader is not an option, we have been going through each element of the visual effects, adjusting them to reduce overall overdraw: sometimes switching elements to be alpha-tested instead, sometimes redesigning certain elements such as distortion, and sometimes redesigning the visual effect completely. Over time, we developed guidelines that help us minimize the performance impact of visual effects, and we are applying lessons we learned across the game. To demonstrate the impact of reducing overdraw, the Ancient Dragon encounter is a good example. In the current version, Ancient Dragon attacks can cause significant frame drops - in this example, frames drop to mid-40s.
After reducing overdraw for its attacks, the framerate curve is much smoother and hovers around 100, and there are still further optimizations planned. You will also notice higher average fps even before attacks, which is a result of other optimizations that we have been working on.

Rendering Paths and Visual Effects

Rendering Path is a term not used commonly, but it is essential for the overall optimization of the game. A rendering path is a series of operations the engine uses to gather and calculate geometry, lighting, and shading. There are two common Rendering Paths used in modern games - Forward and Deferred. Both rendering paths have their advantages and disadvantages, and it depends on the game and art direction to determine which is the ideal rendering path for that specific game. Last Epoch is using the Deferred rendering path, and the main benefit is that using multiple lights doesnt negatively impact the game performance since lighting is calculated and rendered all at once (while in forward rendering, each object has to be rendered once for each light). We decided early that we want to use dynamic lights for our skills or environment design, so Deferred rendering was a logical choice - but there is a catch. Modern engines can combine both rendering paths into a single frame; they can render certain elements in the Deferred path and others in the Forward and combine them into the final image. Shaders decide what rendering path they will use based on features and operations they need for calculations - specific calculations are possible just in Forward and vice-versa. We are using a wide variety of shaders, and many of them need to be rendered in a Forward rendering path - which can cause performance issues, especially if there are multiple light sources in the scene, either from the environment or emitting from skills. To demonstrate the importance of having shaders using the correct rendering path for the situation, here is an example from testing the difference between Forward and Deferred shaders in the test scene using 16 lights and a couple of particle systems emitting blood-like particles. As you will notice, the FPS difference in this specific scenario is enormous.

So to solve this issue, we are going through all of our shaders in the game, not just for visual effects, and we are making sure that all shaders, where we need to have lighting calculations, are rendered through the Deferred rendering path. We still want to have some Forward shaders in the game because they can provide different visual effects, but we are making sure that all Forward shaders we are using are not using performance expensive lighting calculations. We still have a few cases we need to clean up, but when we are done, this will allow our visual effect and level design teams to use lights more artistically, increasing the fidelity and visual depth of the game in addition to overall better performance across the board. We took all the knowledge mentioned above and created a demonstration with the proposed changes. The necrotic explosion caused by Volatile Zombie skill was one of the demonstrations.
This is our current live version of Necrotic Explosion. As you can notice, spawning multiple explosion causes noticeable stuttering, you can see the profiler graph jumping really high, which means both CPU and GPU takes a lot of time to render out these frames.

This was demonstrated with proposed changes applied - reducing both overdraw and shader rendering path fixes. Even multiple explosions have minimal impact on the performance, as you can see on the profiler graph, keeping a smooth framerate without stuttering. Please note that this video does not show the final version of the Necrotic Explosion. This is just a demonstration that helps to guide the visual effect team to improve both performance and visuals of VFX.

Projectors and Trails

During visual effects benchmarking, we noticed several patterns causing performance degradation, which lead us to investigate other systems we are using for VFX, namely Projectors and Trails. Projectors are tools that can project textures/decals on any surface. Projectors are very useful for both level design and visual effects, used either as temporary marks on the ground and other elements by skills or as permanent visual elements in many scenes. We moved from using default Unitys solution for decals to a custom solution, which is considerably faster, and you should see a considerable improvement in skills such as meteor or avalanche. Trail is a system that creates a nice smooth trail behind some projectiles, such as Hungering Souls. But as we found out during our benchmarking, our current solution can cause significant performance degradation, especially if skills have multiple projectiles. We are actively working on replacing the current solution with a better-performing one.

Working around engine limitations

There are a lot of things that can negatively impact performance in the game. We already talked about some of these issues, and sometimes the visual effect does not perform well, but the underlying issue is not immediately apparent. In these cases, we need to dig deeper into the engine and code itself, find out how the engine works, what limitations it has, and what is causing performance issues. Quite a few operations are happening under the hood on the engine level, like how game objects are structured, their hierarchy, and their timings, with each of these having a potentially significant impact on performance. One example is the hit effect. Hit effects are relatively simple visual elements, yet we noticed a performance drop in specific scenarios. We were able to optimize the game object itself, bringing performance to acceptable levels without impacting visuals in any way. Below is a video showing several dummies periodically spawning physical hit VFX. Our current version of that VFX was causing a significant performance drop which impacted fast-hitting and summon builds the most. This was an important issue to solve before the open multiplayer patch was released since up to 4 players in one game significantly increases the number of hit effects.

Improving scene baseline frame times

To ensure that Last Epoch is running well even on lower-end machines, we knew we have to improve our scenes baseline frame times. We developed new internal tools to benchmark our scenes and identify performance issues. Now we are benchmarking our scenes once per week to make sure our performance improvements are hitting their targets and we are not introducing more performance issues. There are a lot of factors affecting scene frame times, including inefficient geometry, shaders, number of draw calls or shadow casters, reflection probes, etc. We did several global changes, including adjusting our graphics settings, and changing shaders or even vegetation, which helped bring frame times down in general. Now we are targeting each scene separately to fix scene-specific issues. We still have a lot of scenes to chew through, but we are working hard to make sure our scenes have baseline frame times as low as possible. In addition, we are learning valuable lessons and are preparing guidelines for our future scenes to be performant out of the box. These images show just a few of our improvements between patch 0.8.5 and 0.9.0. Frametimes were captured using the reference PC (GF 1650 4Gb, i5 - 11600K, 16GB RAM) on medium settings. These graphs represent a grid layout of their respective scenes. Each of the circles of the graph represents a spot on the map where a player can travel, and so our tool captured an image of load times from that location. Basically, think of this as the tool walking through the map like a player and placing a circle each time it has a look around. Each square on the graph represents a physical square of the scene and how long it took that section of the scene to load. In the first graph below, were having a look at one of the more open Monolith Echo zones, which happens to be one of the Imperial Era maps. Here, because its a more open area, the tool captures many points next to each other in order to graph the complete scene evenly as different players may traverse different parts of the map. Wherever the circles can be found is the walkable area of the map.
In the second graph, were having a look at one of the early Ruined Era cavern scenes from the campaign. Since this map is a winding cavern, theres a more linear route for players to travel, and so the capture points reflect this.

We are not done

Performance optimization is an ongoing process, and even as we continue prepping for patch 0.9 and beyond, we have already identified areas where we can continue to improve. We are getting much better at collecting data, identifying sources causing performance issues, tracking them, and ultimately fixing them. We are also working on several guidelines to ensure that all our future content will be as optimized as possible on the first release. While we cant promise that every single performance issue will be fixed in the upcoming patch 0.9.0, we believe we are on the right path to give you the Last Epoch you deserve and can enjoy a smooth and fun experience. We hope you all enjoyed this deeper look into performance optimization in Last Epoch! Well be back again tomorrow with a particularly large post, which we think one or two people might be interested in potentially. Goodbye for now Travelers, and well see you all again very soon!

[ 2023-03-05 19:00:09 CET ] [ Original post ]