New Fluid system 2

Hi Factorians, Here is Dominik, with an update on the fluids. This time it is pretty much finished so I can tell you facts instead of just speculations. You will find how the new algorithm will work and some new handy usability features. In FFF-260 I wrote about how it all started, why we are doing it and what the plan is. There was a huge response from you all and I want to thank everyone for their contributions. Let me apologise to redditors, as at the beginning I started responding on the forums and when I realized there is reddit too, there were too many comments for me to handle. The forum users produced many ideas on how the system could work. About third of them was a fluid teleportation, many where known but many were entirely new and interesting. What intrigued me was the large variety of backgrounds they came from - differents kinds of engineers (mechanical, CS, electrical, ...), mathematicians, physicists, and even people with real pipes hands on experience. I wont go through them here, you can find them on the forums or reddit. There were two proposals on the forum though that were so good that they made it into the game - from quinor and TheYeast. Both of these proposals were very similar and kinda similar to the previous game logic. What it shares is that the mechanic still uses fluid physics simulation and volume in a pipe as a base for the movement calculation. As a result, not much changes on the first glance. What they add though is an emphasis on the fluid network update being independent on the current state (i.e. updating one pipe only depends on state from the last tick) and is therefore independent on evaluation order, which was one of the big pains of the old model that led to sometimes ridiculous junction behavior. Difference between these two was rather small - quinors version allowed perfect throughput with 3 passes over the fluidboxes (fluidbox is the thing managing fluids for entities, so I will talk about them), while Yeasts one was 2 pass with throughput. What was outstanding though is that TheYeast, a physicist, supported the model with a nice theoretical background and whats more, he made an amazing JS simulator to test and compare various modification of the model. Because that extra pass in quinors version was too high a price for the perfect throughput, I went with TheYeasts two pass one. Since the old algorithm only used a single pass run by entities for the update, I first needed to overhaul the whole system to allow accommodating the new one. Going from one pass to two passes necessarily means higher complexity, so we made a big effort to optimize everything we could to make sure we will still end up faster than 0.16. Kovarex wrote about it in FFF-271.

The new algorithm

The new algorithm follows realistic wave equations. It works with two variables.

• The volume of a fluid in a fluidbox (FB) and the corresponding column height.
• The flow speed in a connection between fluidboxes.

The exact behavior then depends on two constants:

• C^2 - corresponds to the mass of the liquid. Affects how quickly changes (waves) propagate.
• Friction - affects how quickly throughput drops with pipe length.

The first good news is that these variables can now be set for fluids separately so different behaviour can be achieved. E.g. crude oil pipeline will require some pumps while steam will be totally fine.
The two pass algorithm for one update (leaving out many details) is then as follows:

• Update flow speeds on all connections A. d = difference of fluid column (input has always 0 and output has max) B. d *= c^2 C. (mechanism for damping waves) D. Flow speed += d E. Clamp the flow to take max of contents (otherwise we could get below 0 - remember that we only use last tick information); fluid can go over max temporarily.
• Move the fluids along all connections A. Just move 'speed' amount of fluid from one FB to another

This algorithm is so simple and works so well, and also requires only very small changes, that it was a clear winner. The main downside is that it can only move of FB content in one tick, so FBs are enlarged to compensate. Another is that the fluids may seem to travel quite slowly into an empty pipe, but that is actually quite realistic and looks nice. The third issue might be some waves and oscillations, which are a result of the realistic model, and are very small with the current dampening model. This could be limited even further by introducing continuous fluid production/consumption, but it does not seem necessary at the moment. What you get over 0.16 is that the fluids now behave correctly and intuitively, performance is consistent (pipes to ground wont help you with throughput anymore), different fluids actually move differently.
As a small but handy improvement, you can now see flow rate information in the entity info of each pipe.
My big thanks go to quinor and mainly TheYeast for coming up with the model and doing a lot of work with me on tuning it and finding improvements to make the behaviour as nice as it is now. In case you are interested in more detail, see TheYeasts forum posts and simulator source code.

Efficiency

The overhauls and optimisations in FFF-271 cut the update time by some 50% and up to 10x on some high-end CPUs. Introducing the new algorithm made it immediately 30% slower. Long story short, through various fixes, including a small change that made the algorithm 1 pass again, this increase was cut down to 15%. So the overall result is that the fluid update is still a lot faster than it was before. I am still debating the segment merging, as it is not that simple and it would cost the simulation some detail. It is a low priority at this point compared to other parts of the update time.

Fluid Mixing

Thats right. No more fluid mixing. Once an empty fluid system (connected network of fluidboxes - pipes, crafting machines etc.) touches either a fluid or a fluid filter, the system is locked onto that fluid. It is then not possible to connect it to another system with a different fluid. There are quite a few actions which can result in merging systems, so each needed to be checked:

• Building an entity with fluidboxes (Eg. pipes, pumps, storage tanks)
• Setting a recipe with a fluid input/output
• Rotating an entity

In order to use the system for a different fluid, the system must first be reset - by draining its fluid and removing all the filters. Please note that old saves have to work with this so if your save currently contains some fluid mixing setup, it will be automatically fixed upon loading it. This will be done during migration either by removing some fluids, unsetting recipes, or destroying entities if need be.

macOS developer needed

At the start of this week our long time macOS maintainer and web admin HanziQ let us know he was leaving the team and moving on to other projects. He has been part of the Factorio team for nearly 4 years, and in that time has contributed a lot to the game and community. We all wish him the best in his future endeavours. HanziQ leaving, along with the departure of our other macOS developer Ji, means we currently have nobody on the team to work with and maintain the macOS version of the game. This is a pretty significant issue, as we have the largely untested GFX engine rewrite due to be released with 0.17. If you know anybody who may be able to help us fill this position, please direct them to our macOS developer job listing.

Steam keys direct from us

We have long sold the game through our own website, which involves receiving a code and then registering an account etc. I have received quite some community feedback, and from this feedback we have decided to start selling Steam keys directly through our site. On the buy page you will be given the choice of a Website key or Steam key.
We hope this will be more convenient for a lot of people, especially for those wanting to gift a copy of the game this holiday season.

Steam awards 2018

The Steam awards 2018 voting has begun, and Factorio is nominated for 'Most fun with a machine'. There are also 2 other Czech games nominated for the same category, so the country is quite well represented.

Animal named after the game

A new species of scorpion Neobuthus Factorio was just identified and classified. My father has a hobby of going (not only) to dangerous places and identifying undiscovered species of scorpions and spiders. He offered to name one of his classifications after the game for fun. You can find the full publication here. As always, let us know what you think on our forum.

[ 2018-12-21 13:38:00 CET ] [ Original post ]