7 Great Projects to Deepen your Understanding of Computers

Push your understanding beyond just the theory and the abstractions with these awesome projects

Nowadays' programming is based on a huge stack of abstractions and friendly interfaces that hide the inner workings of the technology we use daily. Just think of how thoughtlessly we use an IDE to write our code, then compile/run it effortlessly and get friendly error messages in case something went wrong. This was completely different just a few decades ago, though. Back then, programming was certainly not an easy task someone could pick up in a couple of months.

But why can’t we stick with abstractions and call it a day?

You definitely can, but should you? Getting to know the inner workings of technology we use is the first step to master it. Here are a few reasons to get your hands dirty and advance your knowledge:

• Improve debugging skills. How many times did a random error show up and you had no idea on what might have caused it? And no, Stack Overflow doesn’t always have a clear and straight-forward answer. Learning how stuff works under the hood helps you with the debugging process.
• Optimization instinct. Understanding how something works exposes the weak and strong points of it. You will naturally know what to do and what to avoid. Your first approach will often be the best one.
• Contribute to projects that matter to you. Say there’s a feature you would like in software you use. Why not implement it yourself and help the community? If you need it, chances are someone else also does.
• Help to the community. Stack Overflow is what it is because of all those people who dug deep down the stack and mastered their business. These experts must come form somewhere and you could be one.

The actual projects

Beware that these projects might be hard and might take a long time to complete, but they will surely be worth your effort. So prepare yourself for a long journey through the lower level world.

Build a compiler

I know this might sound intimidating and hard, because it really is. However it’s among the best projects a programmer could do to deepen its skills and knowledge, in my opinion.

Building a compiler will involve:

• Re-learning some grammar and syntactical analysis.
• Data structures such as linked lists, syntax trees, symbol tables.
• Assembly languages and how a CPU works.
• Generating executables is another core step of the process and will require some engineering of the file and instruction structure, taking endianness into consideration and working with raw binary data.
• You could implement the execution through a virtual machine, thus working with byte code instructions.

Expect this project to take a long time to complete. It took me half a year to create a compiler (and it’s not yet full-featured), not including a few months of previous experience with their less cool cousin, interpreters.

Build an interpreter

Interpreters are partly similar to compilers. They involve many of the same skills:

• Syntactical analysis and grammar.
• The data structures mentioned above for the compiler (linked lists, syntax trees and symbol tables).
• It may require you to create a virtual machine, although it’s not required.
• You also have to manage variables and symbols during runtime, typically through a symbol table.

Expect this project to be somewhat simpler than a compiler, but still not an easy task. It took me a few months to learn about and build a full-featured interpreter for a new language.

Build a network proxy

Networking and socket programming are never a wasted skill to have in your arsenal.

Through this project you will learn about:

• Socket programming along with concepts like addresses and ports.
• Network protocols such as HTTP or HTTPS. You’ll be studying the structure of a request and a response with their headers and parameters.
• Threading for handling multiple concurrent connections.

This project should not be difficult to complete. It could take anywhere from a few days to a week, depending on how complex you want your proxy to be.

Database management system (DBMS)

Databases are all around us. We use them regularly in many applications. But how many developers actually know what’s behind APIs or tools that allows us to interface with databases? Or what’s inside a database file?

To implement a DBMS from scratch you will need to learn about:

• SQL, of course.
• Interpreters if you decide to also implement a SQL parser and query executor.
• Database file structure is also mandatory for actually storing the data.

This project may not be an extremely difficult one, but if you choose to implement an SQL parser yourself, expect it to take a lot longer since you have to code for the functionalities of at least an usable superset of SQL. In this case it may take up to a month.

Machine learning library

As with compilers, it may sound very intimidating to build a machine learning library from scratch. It’s definitely not easy and requires a lot of math knowledge.

During this process you will learn about:

• Advanced math concepts like derivatives, gradients, vectors, curves and probability.
• Algorithms used to train the various kind of models.
• Represent and store the trained models in files. This will involve working with raw binary data.

In this case I don’t feel like giving a time reference, since I have once started to build such a library, but never actually completed it, nor got to a remarkable milestone.

Procedural world generation

Create a game, be it 2d or 3d, whose world is based on procedural generation. Take Minecraft or Terraria as an example and implement various biomes, structures and everything that comes to your mind. Have you ever wondered how it’s done?

In this journey you will learn about:

• The concept of noise and how is it generated.
• How to combine noise maps to produce world features.
• You’ll have to create some algorithm for handling the various parameter matrices such as temperature, humidity, altitude and more, then combine them into a final world map matrix.
• Chunk loading and storing is another skill you will have to pick up, since CPU and RAM are limited resources.

The time it takes to code this really depends on how complex you choose it to be. I suggest you should read my article on procedural world generation if you decided to take on this project.

Basic social media

It doesn’t have to be the next Instagram, although it should have all the key features of such a piece of software.

This journey will cover:

• Working with a web server and eventually some web frameworks.
• Creating and managing user accounts and authentication.
• Setting up and using some kind of database.
• You have to handle and display text posts, images and videos.
• It requires a minimum usable user interface.
• You may decide to implement encryption in private chats.

The time it takes to build a social media totally depends on the complexity and features you decide to implement. However, expect it to take at least a few weeks for the most basic one.

Conclusion

Don’t be superficial with your business. Dig deep into the inner workings of the technology you build with to really be a master of your art. Programming is a journey, not a destination. There always is room for improvement.

Nobody gets better by doing what he has always been doing. Push yourself beyond your comfort zone to achieve what you never thought you could.

Projects like these are an investment worth your time and effort. All it takes is to pick one and just start: you will learn everything you need on the job.

If you know any other interesting project that involves getting your hands dirty and you think is worth doing, please share it in the comment section.

I hope you enjoyed my article and feel inspired to start building. If you would like to read more about advanced programming projects, please check out my series below:

Thanks for reading!