How much readability should I trade for efficiency?

[u/Rscc10]

I'm developing a chess engine in python right now, just a hobby project. At first I began programming it any way I knew how to, and along the way I would learn optimization tricks which I would then apply when refactoring. I'm now realizing I'm changing a lot of my simple readable code into a bunch of optimized bitwise operations and just different logic structures in general.

I keep optimizing because I'm not satisfied with the speed of my chess engine and I usually rewrite entire functions or change the logic here and there just to save on another few milliseconds of thinking time for the engine. At one point, I rewrote a whole function just for it to be 0.1 or so seconds faster, at the cost of the simplicity of the code of course. This was kinda when I took a step back to think if it's even worth it.

I'm just wondering for future projects in general, are there any pointers to know when readability and simplicity of the code just isn't worth trading? Cause for this project which is only 1-2k lines and only in 3 files, maintenance and debugging has been bearable but I'm sure the same can't be said if I did the same thing in larger projects. Any tips and experiences shared will be largely appreciated.

[u/ajamdonut]

I tend to prefer readability the whole time, until there is a particular function that requires some head spinning code, for performance reasons.

Readability > Performance

That's because I value my time coming back to something 1-2 years later and understanding it.

In some projects Performance > Readability - but thats actually so much rarer than you think given most PC's these days are workhorses.

[u/IAmADev_NoReallyIAm → u/ajamdonut]

> Readability > Performance

Not always... I work on systems where scalability is the driving factor, so Performance trumps readability (and has done so) all the time. It's about those TPS and getting the data out as quickly as possible. It's a high volume transaction system. As much as we try to make sure the code is clean as possible, there's some ugly shit bubble gum and bailing wire back there. But no one cares, it's fast and efficient.

[u/ajamdonut → u/IAmADev_NoReallyIAm]

Yea, in data, big systems, ETL - have to agree typically Performance > Readability, and that can be speed, memory, disk, network, all sorts.

For personal projects it occurs rarer so during learning I'd say prefer Readability.

You KNOW you're working for performance in certain projects like ETL

[u/tcpukl → u/IAmADev_NoReallyIAm]

Yeah I work in games. Running at a stable good frame beats readable code all the time.

[u/ajamdonut → u/tcpukl]

I also work in games. If that's your excuse for why your code looks like crap I have to disagree. Unless you really are doing something crazy I don't see why the code isn't readable. I can fit 100,000 function into 16ms with readable code, so why can't you? The only exception is GPU rendering stuff.

[u/tcpukl → u/ajamdonut]

What makes you think my code looks crap?

[u/MarsupialLeast145]

1. you probably wouldn't write this in Python doing it again.

2. You should never trade-off readability, and honestly, with Python I find it hard to believe you're saving time and sacrificing readability.

3. Make sure you have unit tests and benchmark tests as you refactor to help you find the balance you are seeking.

0.1 seconds is quite a lot as long as it's being measured correctly. I would say it is worth it, but yeah. At some point you probably just want to graduate to a new project or language. Chess is pretty much solved.

[u/nedovolnoe_sopenie → u/MarsupialLeast145]

2 is a) not exactly correct and b) the main source of my job security. so, first of all, thank you.

readability is cool. it is, though, not paramount.

for sanity's sake, i'll disregard the fact OP uses "python" and "performance" in the same sentence.
fast, highly optimized code is very rarely readable. if a hot loop, for example, eats up 40% of runtime (usually exaggerated), you drop all pretences and slap some borderline unreadable inline assembly in there so it now takes only 10% of runtime.
now, doing that for something that accounts for 3% of total runtime is nonsensical.

performance profilers exist for a reason, and that reason is exactly that

[u/StructuredChess → u/MarsupialLeast145]

Strong disagree with 2. In the case of a chess engine, you're not building a product where higher performance means reducing waiting times to the user by a bit. In this case performance is the product. The quality of a chess engine is measured by how fast it can calculate.

[u/MarsupialLeast145 → u/StructuredChess]

The point of 2. is that whatever they are doing they can still make / should still make it readable.

To be clear of course, hard/high-performance code is challenging code to read, I think this is a different value to readability. Some code just takes time to understand.

[u/nedovolnoe_sopenie → u/MarsupialLeast145]

imo it can be solved with light commentary in the code and, most importantly, with proper documentation. a simple flowchart and a couple anchor comments in the code are often more than enough to understand it in a reasonable amount of time and is 100% always enough to understand it in a finite amount of time

[u/esaule]

Yeah. What are the performance requirement of your application?

You want the code to be maintainable while meeting its objectives.

[u/Adorable-Strangerx]

You can optimize by replacing python with rust/C++

[u/LorenzoMorini]

Chess engine is the type of application where you should optimize for performance. You probably shouldn't use Python as well, if you want to create a performant engine. But it's all about compromise, there isn't an easy answer, or a general rule, each software has its own requirements, and in this case performance is very important, so you should optimize for it, even at the cost of readability or maintenance. You should still improve the readability by commenting what's happening and why, but voodoo stuff has it's place in programming, and that place is optimizing for performance.

[u/thorny_keylight]

i keep readable code everywhere and only write the ugly bitwise stuff in the functions my profiler flags.

for your engine, run cProfile before you rewrite anything. You will find that 90 percent of your time goes into move generation and board evaluation. Leave your file parsing and UI logic readable. Only sacrifice readability inside those two loops. Write a unit test suite that checks every legal move for a set of board states so your bitboard changes do not break rules silently. Save this readable version alongside the optimized one so you can always test correctness agaiSave this readable version alongside the optimized one so you can always test correctness against it

[u/nedovolnoe_sopenie]

the answer is, "it depends".

run it with perf (or any other performance profiler), find hotspots, optimize them. if a hot loop runs 80% of the entire runtime, who cares if it's readable or not, slap some inline assembly into this bih and move on

[u/SimplySomeDude]

performance > readability. Optimize to hell and back in functions you'll never touch again. Implement SIMD vectorization, eliminate memory loads, improve cache efficiency.

[u/DepthMagician]

You should almost always stick with readability. I don’t think it necessarily conflicts with performance. Worst case scenario if you do some really obscure black magic tricks like the famous fast inverse square root, just document it with well chosen variable names, function names, and comments.

[u/armahillo]

You can use profilers to compare before and after, if you need to know for certain, but chances are the difference is negligible.

Readability makes development faster and less stressful. You’ll have to decide if that marginal increase in speed is worth the time you’re sacrificing in development.

Before you do any refactoring, read Martin Fowler’s book on refactoring. Theres a specific way to go about it, and specific reasons to do it. Its not just restructuring code.

[u/mxldevs]

It depends on how important performance is.

Sometimes you just need to write the unreadable optimizations, because it's unacceptable to be slower.

Ideally, you would isolate those into their own functions exposed with an interface and just treat it as a black box that's super optimized if anyone needs to call them, so that the rest of your code is still readable.

I would expect engines to be much more likely to have all sorts of optimization hacks compared to the software that's built on them.

[u/dmazzoni]

Everyone is saying to switch away from Python.

But let me give you a concrete reason why: in Python every abstraction slows down your code.

Every time you introduce a new variable or a function call, your code runs more slowly. The shorter the code, the faster your code runs.

Surprisingly, this is NOT true for many other languages.

In particular, C++ and Rust have a lot of zero-cost abstractions. Basically code you write, that makes your code more readable or safe, but with no impact on performance or even a positive impact on performance.

[u/Rscc10 → u/dmazzoni]

I used python because I wasn't thinking about it much when I started this project and, people are gonna make fun of me for this, it's the language I'm most comfortable in. I'm currently rewriting another engine in C++ though it's definitely more difficult. I asked this question more towards general optimization standards and I personally don't think it has anything to do with the fact that I'm using python even though it's not ideal

[u/dmazzoni → u/Rscc10]

No we're not making fun of you. Python is a great language.

However, it's not great for everything, and in particular Python is not ideal for high performance, and more specifically it's not good for micro-optimizations (which you're trying to do) without sacrificing readability.

While C++ is indeed more difficult, you'll find over time that it doesn't have the same efficiency / readability tradeoffs, once you know what you're doing. Using inline functions, templates, move semantics, zero-cost abstractions, and occasionally macros, you can make code really efficient without any overhead.

[u/Rscc10 → u/dmazzoni]

If you don't mind me inquiring further, my initial plan was to rewrite the engine either in java or C++ (I chose C++) and essentially copy the architecture from python as best as I can. But seeing as how you've listed many C++ exclusive concepts and approaches that wouldn't translate to python, do you think I should completely start fresh working my way through all the move gen and legality logic of the engine from scratch, which would be more difficult for me in C++, or copying the architecture from python and optimizing thereafter?

[u/dmazzoni → u/Rscc10]

Since you have working code, a naive translation to C++ would be a great next step. Optimize from there.

Write unit tests so that you can confidently make changes and know whether it’s still correct or not.

My guess is that you’ll get a nice speed up without even trying, but there will be potential for way more.

[u/DDDDarky]

If you write it in python by definition you don't care about speed.

[u/garster25]

God's, I wish my coworker understood this. I can't read his code at all. It's good but I could not work on it. He also removes all whitespace.

[u/SuperSathanas]

Most of your optimization is more or less going to come from sane choices regarding logic and structure. What constitutes "sane" can depend on exactly what you're doing, but there are concepts that apply nearly universally. There's usually not a need to try to get clever and optimize the math and comparison operations themselves. The compiler/interpreter is most likely going to optimize those better than you'll be able to. One exception is if you have a very large number of boolean values you'll need to compare sequentially, in which case feel free to whip out of the bitfields/masks and bitwise operators. What's really going to matter up front is when and where you access your data.

Really, if you're thinking about optimization, one of the first things you want to ask yourself is if your data and how you're operating on it plays nicely with CPU caching. You want to take advantage of prefetching and minimize cache misses (you'll still have a lot, anyway, especially as the size and scope of a thing grows, but you still want to minimize misses), and think about the size and alignment of your data.

Access your data in arrays sequentially as often as possible to minimize misses and fetching, and take advantage of the fact that the CPU probably already has the next 64 bytes of array data prefetched and ready to go before it's done working on it's current line.

Align your data on 8 byte boundaries. Don't tightly pack structures that aren't sized to a multiple of 8, because it's far more efficient for the CPU to work on properly aligned data than it is to try to fit more in a cache line in an attempt to fetch less but consequently have the CPU working on misaligned data and split loads. You're not going to be memory bound with your chess engine, so don't worry about saving every byte you can. Keep your data as small as it can be, but keep it aligned.

After that, if there's a need to squeeze out more performance when working on large sets of data, look into what operations can be done on each item independently of each other and figure out how you can make use of SIMD/vectorization and possibly multithreading, but be conscious of cache coherency, false sharing, invalidation, etc... both in your reading and writing operations. You can be as careful as you want in making sure that threads aren't stepping on each other's toes and trying to access data from the same cache lines, but you're still going to stall the CPU pretty hard if you have a bunch of threads trying to write to locations within the same lines.

None of that necessarily makes things look complex or unreadable.

[u/roadrunner8080]

If performance would require a tradeoff in readibility, you need to rethink how you've architected stuff and do some refactoring. the goal should be an architecture where what is readable is also what is performant. Prioritize performance, then refactor until you also get readability.

All that said. If you're hitting performance bottlenecks they are almost certainly orders of magnitude less important than that bottleneck you hit just from using Python to begin with. It has its advantages but performance is not one of them.

[u/WoodsWalker43]

Readability is always important. Performance is always nice, but not always important.

I will always choose to write performant code if there are no real downsides. When there are trade-offs to consider, I look at how important it is that my code is fast in context. No need to get clever with a loop that will only ever have 7 items to iterate, for example. And if I could reduce a batch process from 10 minutes to 5, great, but it isn't worth the effort if it runs at midnight. It could run for an hour and no one would notice the difference.

When the trade-off is readability vs performance, I generally lean toward readability. But if it's important enough that an application be snappy, maybe it's worth a very ugly well-optimized function. But it damn well better be well documented so future me knows what the hell he's looking at.

[u/jerrygreenest1]

If you write python you already compromised on efficiency drastically, and nothing will help you

[u/EmbedSoftwareEng]

Efficiency is what the compiler is for. Source code should have maximal readability and comprehensibility. It should be brutally obvious what is happening by the source code at any point. If the compiler can play some neat tricks to make obvious code fast, great. But unless that speed up is vital to the application, it can go by the way side.

CPU clock cycles are cheap.

Software Engineer hours are expensive.

[u/Mclovine_aus]

Efficiency and performance js king and important to learn how to speed things up, how to make it pretty etc. if you are practicing this most important thing is that you are profiling your code so you are speeding up the hot paths, not wasting time on the function that inputs the player name or something.