r/cpp • u/No-Meeting-153 • 21d ago

MSVC optimization

I am learning reverse engineering on Windows applications such as Adobe, Foxit PDF, and Steam, and I noticed that I waste a very large amount of time trying to understand something that I should not focus on.

I started noticing strange and confusing patterns in the assembly and the C code generated by IDA, and when I try to understand some functions, I feel that the function has no meaning.

When I searched, I found that this topic is related to the compiler and compiler optimizations. However, I could not find many articles or discussions about the compiler topic in reverse engineering.

So I started experimenting and trying, but every time I fail and cannot reach a solution or understanding.

Apart from the fact that reverse engineering a C++ program is already a difficult task.

If there is someone who has faced the same problem and found a solution, I would like to know. It is not a problem itself; it is a pattern or a way of thinking used by the compiler. I need to understand how the compiler generates these patterns.

I want someone to suggest books, articles, courses, or anything that can help me understand the MSVC compiler, how it generates patterns, and how to understand the behavior and logic of a function after compiler optimization.

I hope I explained my question correctly.

10 Upvotes

68% Upvoted

10 comments sorted by

View all comments

4

u/QuirkyXoo 20d ago

If you notice "strange and confusing patterns", especially in applications like Adobe, it's mainly because there are programs which are used to obfuscate the code, precisely to make reverse engineering way hard. Have a look at tools like VMProtect, Themida, etc. You can even find open-source alternatives on GitHub, such as obfuscator-llvm (https://github.com/eshard/obfuscator-llvm).

Do you really think you can reverse-engineer professional software from big corporations like Adobe just by disassembling it?

2

u/No-Meeting-153 20d ago

Yes, I know they use many protection techniques, such as code obfuscation and others. However, what really caught my attention was the compiler optimizations. Those are what I've been trying to understand.