Question: relative distance - which is fastest? about factorio-stdlib HOT 9 CLOSED

Who can resist a performance question? I couldn't and searched for my old test script: https://gist.github.com/Choumiko/7169004f0ae31eb530f66150294b05c9

Both functions used on a positions table with 160 positions. Biggest iteration count was 10000, which results in 1,600,000 function calls:

10000 loops c=-133.903980 ms squared= 396.549 ms manhattan= 530.453 ms

making Position a local variable and using local abs = math.abs :

10000 loops c=-40.832281 ms squared= 343.053 ms manhattan= 383.885 ms

above optimisation and removing the Position.to_table calls:

10000 loops c=-33.581018 ms squared= 181.226 ms manhattan= 214.807 ms

all above optimisations and inlining the functions:

10000 loops c=-36.621094 ms squared= 138.274 ms manhattan= 174.895 ms

The most interesting find i think is that the calls to Position.to_table really hurt performance (at least in this case where there's not much going on in the function itself)

from factorio-stdlib.

Lol, love it!

But yes, in general a squared distance calculation will always be faster than manhattan distance assuming perfect data access.

LUA does not have perfect data access however, but if the 'shape' is always the same then a table lookup call should be optimized after a few calls to be more efficient, however the fence guard will still incur a speed hit. An array access should be a 'tiny' bit faster than accessing a table as a dictionary (the to_table call really should be a to_array call) once JIT'd (again because of the fence guard, the actual access code ends up the same).

I wonder if it might be good to add a preprocessor to this library to be able to make certain things like to_table/to_array inlined. You could even add such macro's able to be used in user code if you did not mind adding in an optional build system. Would be useful for the people that do such calculations in tight loops, for those that do not then using the library via function calls is fine enough.

/me is now thinking of methods to have a better inlining capable language compile to lua for this use. If only Factorio accepted native-code libraries, or if it embedded in Terra instead of luajit then you could easily do all of that...

from factorio-stdlib.

Thanks for the performance infos!

I'd also come to the same conclusion about the table to/from array conversions and posted this on forums earlier, suggesting array-only format for both LuaPosition and LuaAerea objects: https://forums.factorio.com/viewtopic.php?f=34&t=34828

Love the look of Terra, but doubt they'd switch to that at this late stage of development.

from factorio-stdlib.

Love the look of Terra, but doubt they'd switch to that at this late stage of development.

Oh definitely not, it is not a drop-in replacement to lua like luajit is, it is an entire compiler (LLVM) and more that just happens to be done with lua. The lua generates terra (which is just lua as well, but typed), which then gets compiled. An interesting and highly efficient setup, but way overkill for game scripting (though would be useful to be side-used with luajit, but certainly not a replacement).

from factorio-stdlib.

LUA does not have perfect data access however, but if the 'shape' is always the same then a table lookup call should be optimized after a few calls to be more efficient, however the fence guard will still incur a speed hit. An array access should be a 'tiny' bit faster than accessing a table as a dictionary (the to_table call really should be a to_array call) once JIT'd (again because of the fence guard, the actual access code ends up the same).

Factorio is using the standard Lua C implementation, not LuaJIT (there is no LuaJIT implementation for 5.2+), so there is no JIT of any code.

I wonder if it might be good to add a preprocessor to this library to be able to make certain things like to_table/to_array inlined. You could even add such macro's able to be used in user code if you did not mind adding in an optional build system. Would be useful for the people that do such calculations in tight loops, for those that do not then using the library via function calls is fine enough.

I've considered this too, and even more drastic tools (building a write-through cache layer in front of large portions of the Game API, so that repeated calls are optimized away. game.tick is the most obvious and trivial example.), but I don't think the effort is worth the tiny savings most of the time. Most mods have terrible performance because their authors aren't aware of basic optimization principles and complexity analysis, not library overhead.

from factorio-stdlib.

Factorio is using the standard Lua C implementation, not LuaJIT (there is no LuaJIT implementation for 5.2+), so there is no JIT of any code.

OhGodReally? Well in that case death to tables, using array shaped structure only! Why are they not using luajit?! I've implemented luajit into a half-dozen different things and it is not hard by any stretch... o.O

(EDIT: Plus with luajit you can have direct structure access via lua C calls instead of going through that horrid mess of dreadfully slow C lua API horror, it is worth using luajit from the start just for the opportunity of avoiding the C lua api horror, though you can still use it if you like masochism...)

I've considered this too, and even more drastic tools (building a write-through cache layer in front of large portions of the Game API, so that repeated calls are optimized away), but I don't think the effort is worth the tiny savings most of the time. Most mods have terrible performance because their authors aren't aware of basic optimization principles and complexity analysis, not library overhead.

Yeah, I'd love to go for a better mod code design instead of the spaghetti code that near every mod I see uses. A proper separation of concerns would be a first start (ah if only immutability...).

from factorio-stdlib.

OhGodReally? Well in that case death to tables, using array shaped structure only! Why are they not using luajit?! I've implemented luajit into a half-dozen different things and it is not hard by any stretch... o.O

Yeah. I've discussed it with @Rseding91 on IRC a few times, and while there was/is interest in LuaJIT, concerns over Lua 5.1 compatibility and differences in LuaJIT across platforms (which may impact determinism in mods) led them never to seriously tackle it. It may be reviewed again in the future, but I've not seen it mentioned for 0.15 even.

Yeah, I'd love to go for a better mod code design instead of the spaghetti code that near every mod I see uses. A proper separation of concerns would be a first start (ah if only immutability...).

Lua lends itself somewhat to functional programming idioms, which I've tried to build up some libraries for in stdlib. However it is difficult, as it's just as easy to enter callback hell.

from factorio-stdlib.

Closed as this was answered (squared eked out a tiny win. But seriously, use whichever fits your use-case). The flogging of the lua standard runtime may continue.

from factorio-stdlib.

The flogging of the lua standard runtime may continue.

/me coughs

Lua lends itself somewhat to functional programming idioms, which I've tried to build up some libraries for in stdlib. However it is difficult, as it's just as easy to enter callback hell.

Yeah I helped a couple mod authors refactor a huge amount of spaghetti into a couple of simple functional calls and some immutability, substantially helped their code actually be maintainable. I should get back in to modding, maybe make an immutable mod library or something (another ohGodTheHorror was making a table differ in lua not long ago, yeesh...). It might be easier just to make a back-end for the OCaml compiler to lua or something... >.>

EDIT: Oh, and yes, if they enforced an immutable state passing style in mods that would fix all mods determinism issues entirely. Mod authors may drag their heels on that but it would help substantially (this is how I get full determinism in my C++ work, never failed if you use the right style).

I am still a fan of a style with an init callback, a set of subscriptions to listen to generated events based on the state, and an update callback that returns a new state and a possible set of commands for the runtime to perform ("kill that unit" for example), plus it is wonderfully easy to buffer up those commands and changes for sending across a network for synchronization purposes without having to do costly gamestate recordings.

from factorio-stdlib.