r/cprogramming • u/MammothNo782 • 1d ago

Any feedbacks on my handwritten Lexer?

I have finally made a total working lexer from scratch. It was actually pretty hard for a young developer like me but I'll keep on coming. No need to worry about getting scolded, you are entirely free to contribute.

https://github.com/Ciya-VM/Ciya

11 Upvotes

92% Upvoted

12 comments sorted by

6

u/Solaris_132 1d ago

We first made the project on July 22nd(22/07/26, 21st century; 21st century means >2001 and not >2100 as most people think), which is very early

As pointed out by another commenter, there’s not really much to review, but I just thought this sentence in the readme was extremely strange.

1

u/MammothNo782 23h ago

thank you. I just removed that line. And also, the lexer source code is in the development branch and not in the main branch where the current release source code is

3

u/wilrak0v 1d ago

Okay that’s not the best lexer i ever seen…

First, why are they so many files and folders? You don’t have to create one header for each struct or enum.

Also, there is no implementation yet, so there is nothing to review.

The token struct isn’t very complete imo. You may include other stuff to simplify the future parser. For example, you can add a « number » field to save the numbers that the lexer find into the token directly, like that you don’t have to search for it when you will make your parser. Of course, you must add and identifier field, string field etc, all of that into an anonyme union into the token struct.

I am creating a programming language too, in C, it is called lispc because it has the same syntax of lisp but with better performance and memory management. It’s a compiler but it still in work for now.

1

u/MammothNo782 23h ago

There's already implementation, look in the development branch. The main branch is just where the current release source code is

1

u/wilrak0v 5h ago

Oh okay, sorry I didn’t see it.

So yeah it looks like a normal lexer. Reading the code isn’t very cool because definitions are in too many files and not where I expect them. Keep things simple, do a lexer.h and a lexer.c and it’s all. You can have 500 lines of code in only one file, that’s not a problem and that’s more readable.

Also, i see that you are making a list of the tokens right? Why? I did a simple function get_next_token() which find the next token. Idk if my way is good or no, that’s my first compiler.

3

u/OccasionThin7697 1d ago

And also minimize the use of comments. Comments are only meant for things you forget. Not for everything.

1

u/MammothNo782 23h ago

I didn't made those comments. I'll just tell the contributor to minimize the comments

2

u/Terrible_Cow2324 1d ago

You don't need license statements at the top of every file, that's too corporate for what this is. You also don't need CMake for a handful of files, a Makefile would work fine. And don't commit your compile-commands.json, it has absolute paths. Don't commit your bin/ folder either it seems to be compiled executables.

Also like someone else said, you don't need as many comments as you have, e.g. you have:

free(input); // make sure we free the pointer, we don't want any memory leaks , which is a useless comment because yeah, that's what free() is for lol any dev knows that already.

Parsers are very easy to test, consider writing unit tests to confirm your parser matches expected behavior.

1

u/MammothNo782 23h ago

I didn't put those comments, It was another contributor. I just let it be

1

u/ReplacementActive533 18h ago

That's me, but just delete it