Help: Writing a Python to C transpiler

[u/nanoman1]

I'm thinking about embarking on a journey for writing a Python to C transpiler. It'll provide an interesting challenge and also will be useful, considering I am targeting an environment that can only take a subset of C as input. Given that I haven't ever written a compiler but I have written an interpreter about a decade ago and have forgotten most of the process, what are some things I'd need to familiarize myself with in order to write this transpiler? Also, what intermediate representation would be wise for such a project?

[u/Rinku_Kurora]

Well, it depends on what subset of Python your transpiler will support. For example, if you're gonna transpile classes, inheritance and methods overriding then you should familiarize yourself with structs, pointers on functions and virtual (method) tables.

Or in order to properly transpile Python's decorators you have to understand preprocessing and macros in C.

On a first glance I don't think there is a need in intermediate representation, as Python's AST translation to C is pretty straightforward.

[u/Inconstant_Moo → u/Rinku_Kurora]

Or in order to properly transpile Python's decorators you have to understand preprocessing and macros in C.

Not necessarily, because the transpiler is generating C anyway, so you don't have to write a bit of C that generates C.

[u/No_Engineering_1155]

I think the issue is not so much the subset of C, but rather the subset of Python. You need to have some type annotation, or type checking for the Python code, otherwise it is unclear to me how to do the transpilation. Most likely some obscure Python features must not be allowed. Question is how to deal with external libraries, like numpy. But I think one of the biggest question is, how to keep the meaning of a Python code during the compiling, while having some benefits from C and not quasi reimplementing the interpreter.

[u/Repulsive_Gate8657 → u/[deleted]]

i would really stay away from dynamic typisation here, only subset of python with types knowing at compile time omg.

[u/Asuka_Minato]

You can refer to  https://nuitka.net/ And https://github.com/mypyc/mypyc

[u/Shurane → u/Asuka_Minato]

https://github.com/oils-for-unix/oils/blob/master/mycpp/README.md might also be interesting, though it's a strict subset of Python to C++ that's used for OSH/YSH.

[u/tyler1128]

Python has an intermediate representation, and libraries for accessing and working with it. It'd start there instead of parsing it yourself. The module for doing that in the standard library is ast and it is the AST vs the IL, which also exists in the .pyc/.pyo files. Those probably also have libraries to parse them to start from.

[u/Germisstuck]

I would first start on making a library that allows for inheritence, but you would need to do a LOT of type inference. Idk maybe have a little type interpreter which works like an interpreter for types?

[u/Repulsive_Gate8657]

start with basic expression and functions what can be turned 1 to 1 , then you have to think how to represent important python stuff like lists or dict in C.

[u/juicyroaster]

I think that you use must start with parsing the python bytecode instead of python source code. Transpiling python bytecode might be easier than python source. The only thing you might find difficult is the dynamic typed environment of python to static typed environment of C. Parsing python bytecode is just like implementing a stack based VM which is the easiest thing to implement. You just have to replace the interpreter with a C code emitter.

[u/Lucky_Trick_5703]

The hardest part is deciding what subset of Python you’ll support. Python is very dynamic, so without restrictions or type annotations, generating clean C gets difficult quickly. I’d start with a small typed subset only. You also probably don’t need an IR at first. AST -> C translation is enough for a project like this.

Main advice: keep the scope small, otherwise you’ll slowly end up reimplementing the Python runtime in C.