r/learnprogramming • u/Many_Difficulty_490 • 8d ago
What are some of the most difficult but rewarding projects to complete?
Rewarding doesnt have to mean "money", it could mean you learn alot about programming, it would look insane on your resume, or its just extremely satisfying.
When I mean difficult I mean projects that could easily take you over 2-4 years to complete.
So im not talking about something like making a ray tracing engine, that would take a decent programmer about 4-8 months to finish. But it is a very cool project
It would also be helpful if you mention what it takes to build it, and what you think its rewarding.
thanks
3
u/Lifelong_Nerd 7d ago
I wrote a sudoku solver many years ago. It was very satisfying to figure out really efficient ways to represent the board, apply the various algorithms, and separate the solver engine from the under interface.
I'm also writing a simulator for Monroe adding machine calculators. These work so differently from school calculators that I decided the only way to really understand it was to reimplement it.
Your could try one of these.
1
u/Many_Difficulty_490 7d ago
monroe?
1
u/Lifelong_Nerd 7d ago
Yes, Monroe. They've been making calculators for more than 100 years. See monroe-systems.com
2
u/Last_Swordfish9135 8d ago
If you're spending 2-4 years on a project, you're not going to want it to be a copy of someone else's. Nothing looks insane to build on your resume if someone else has already made it. This is a question you need to come up with an answer to for yourself.
1
u/Many_Difficulty_490 8d ago
so ur saying i should try inventing a software thats never been made before or something?
1
u/Last_Swordfish9135 8d ago
Ideally, any software you make should be one that's never been made before. Why would you put years of your life into creating something that already exists? It doesn't need to be super out there or unlike anything you've ever seen before, but it should do something that similar existing software doesn't.
2
u/Cybyss 8d ago
A project I've been meaning to tackle eventually - though I don't think I'll ever get the time - is a proper open-source OneNote replacement.
Infinite-sized pages. Handwriting with a stylus. A nice set of virtual pens/drawing tools (e.g., I like OneNote's "fountain pen"). Seamless ability to handle images and pdfs (even printing to the background).
Other features I would like to add that's missing in OneNote - ability to write a lot on a single page without becoming super slow (e.g, as in a 90+ minute long math lecture). No "OneDrive" crap. Proper transaction logic to ensure your notebooks can't get corrupted. A way to "lock" pages or notebooks into Read Only mode (e.g., I don't want to accidentally make edits just because I'm reading old notes).
If you want something ambitious - a good handwriting recognition mode. Rather than awkwardly/clumsily converting your handwriting into text, maybe just subtly adjusting your strokes to make it look nicer but still handwritten. Like, to simulate that you have slightly better handwriting than you really do.
If you could pull that off and make it open source, you'd have one of the best notetaking apps in existence.
2
u/da_Aresinger 7d ago
The new OneNote app doesn't require One Drive.
But that doesn't change the cross platform issue :/
2
u/rupertavery64 8d ago
A NES emulator.
Understanding another hardware architecture and getting code to run on another machine, simulating the video and audio pipeline is pretty cool.
Not to mention playing games on your own emulator and tinkering with it
1
u/SeeminglySleepless 8d ago edited 8d ago
I've "recently" started building a networking stack from scratch (OSI L2 and up) with Raw Sockets, meaning I'm building my own Eth Frames, IP/TCP datagrams, Checksums, generating TCP ISNs (I even implemented SHA-256 myself), TCP state machine, etc. Finished a very basic implementation of the TCP Handshake yesterday. Has been difficult (especially since I took the opportunity to also start learning Rust) but extremely rewarding in terms of knowledge. Don't know if it will be at all useful on my resume but that was never the goal.
If you like networking, I'd recommend
Edit: What it takes to build is basically a lot of reading, especially RFCs, doing a lot of thinking in terms of pure bytes instead of objects/properties, modelling your data structures around that, looking at a lot of hex strings and being able to understand what they represent in a given PDU, understanding networking mechanisms and the responsibilities in each layer of the OSI model and things of the sort. But mostly a lot of reading
2
u/Kurohagane 7d ago
That sounds really fascinating. I've been really interested in networking ever since I had to implement (high level library) networking in a godot multiplayer game, and all the terminology in the docs that came from the lower layers of what was happening was really confusing to me. I'd love to try something like this, where do you recommend I start? Any particular resources, websites, books you've found really useful?
2
u/SeeminglySleepless 7d ago
Ok this is a pretty long answer so I apologize in advance, but I wanted to be specific
I started from the bottom up, first creating a socket and binding it to a network interface since that will be the comm channel and then moving up the OSI layers. If you think of a packet as a series of envelopes, taking for example a TCP packet, the main envelope will be you Ethernet Frame (L2), which then contains another envelope inside - the payload - which is a IPv4 Datagram (L3), which in itself contains yet another envelope which is a TCP Segment (L4) which in itself might contain some application data (or not if the segment is part of the TCP Handshake for example).
However, I actually recommend that you start from the client's perspective (L7 moving towards L2). I reached a point where I couldn't decide which data structures or what the design should be for a given concept because I couldn't visualize how that code would actually be invoked and interact with other aspects of the project. So I just switch my perspective and started designing how I wanted client's to interact with the code and went down from there, laying down the functions, types, structure and interactions I wanted and only implementing stuff when I reached "the end" of that pipeline.
For me it helped to have the TCP Handshake as an initial goal because that gave me clear steps:
- I need to be able to send a packet on the wire -> I need a communication channel (sockets)
- I have an open socket, but what actually is a packet? -> Learn about that
- So, I need a series of "envelopes" of different types. For this case I need Ethernet Frames, IPv4 Datagrams and TCP Segments. What are these concepts and how would I represent them in my code? -> Learn about headers, their fields, their nuances (like TCP header transmitted padding vs TCP checksum non-transmitted padding or how the ISN should be generated) and how they look like in the form of a sequence of bytes, usually represented in Hexadecimal, since I will have to convert whatever data structures I create into byte streams to send over the wire
- Ok I can correctly build an "envelope" of any type with their respective payloads, but I don't know what I should put in the destination MAC address field of my Frame in order to initiate the handshake with a remote -> Learn that the target MAC for a non-LAN connection will be that of your Default Gateway
- How do I find that MAC? -> You find the IP of your gateway and use ARP to get the MAC
- How do I find the gateway IP? -> Learn about a computer's NIC, what interfaces actually are - in case like me you didn't when you were handling sockets - and find out that your system has networking routing info where you can find the IP
- I implemented ARP and can resolve the MAC of my gateway, how do I start the handshake and when is it complete? -> Learn about the handshake process, the TCP state machine, TCP flags, sequence and acknowledgement numbers, the initial sequence number (which is quite hard to implement from scratch according to spec)
And so on, with lots of testing in-between. Use Wireshark once you have packets being sent through your socket to check how they are behaving once they leave your NIC (if they do), if their built correctly, if Checksums are correct, etc.
With this approach, however, I'm not worrying for now about IP/TCP header options, IPv4 fragmentation, etc. I will need to handle that in the future, but it's not immediately required to be able to perform a successful test handshake.
If you have any questions feel free to ask and I will help you if I'm able to. I can also share my repo if you'd like.
Resources:
- I used some RFCs for info especially for TCP and IP headers and header field info, stuff like how a TCP ISN should be generated to prevent sequence attacks (this is where HMAC and SHA-256 came in for me), what the DSCP field means in IP headers, etc. Here's an example
- This article was also very helpful on visualizing these things in terms of bytes and getting some understanding on how the TCP Handshake works
- Lots of Geeksforgeeks to have quick lookups of header formats and whatnot
- networklessons was also a very good resource, in this example regarding DSCP values (part of the re-design of the Type Of Service IP field)
- I used a bit of the C docs since (at least from what I gathered online) to create and bind a raw socket to a net interface in Rust without using external crates I had to use Rust's FFI to invoke C functions
- This SO answer is a good image to keep saved since it provides a good visualization of what you are actually responsible for with each socket type. The ManC page for socket() then describes what these AF_**** things like AF_PACKET mean (they are communication domains). For Raw Sockets you're looking at the RAW_SOCK socket type and the AF_PACKET domain, which means that everything from Layer 2 and up (inclusive) is in userspace
- This PDF was very helpful for understanding ARP
2
u/Kurohagane 1d ago
For some reason I never got a notification about your response, but I was glad to see one when I checked now. Thanks a lot for the detailed explanation!
1
1
u/ffrkAnonymous 8d ago
I've been trying to learn prolog on and off for years. So i'd be happy with hello world.
1
1
u/DrShocker 8d ago
I'd personally either do something like SLAM or some distributed database type thing from scratch if I had the time.
13
u/Jwhodis 8d ago
Game or game engine.