There are several items on the list of projects that I'm absolutely interested in doing. I think RC will be much more satisfying to me if I plan for and achieve several small milestones across different projects than working on only one project for the 3 months.

I'm trying to filter the list down to the biggest glaring holes in my understanding of computing, including bucket list type topics I've always wanted to learn about. I'm attempting to flesh out goals and milestones ahead of time, so I have clear and attainable targets to work towards.

For accountability, goal-setting, and for the sake of a momento, I'd like to start making a <=5 minute video very casually documenting what I accomplished in the day. I'll check in after the first week to see if this is useful. One thing I struggle with is sufficiently reflecting on and presenting my work -- I would like to have some sort of projects portfolio on my personal website.

I've also listed some shorter projects that I feel really confident about, to knock out in periods where I just need a break and want to work on a one-off.

Longer projects

Operating Systems
Training Neural Networks (To Play Chess)
Graphics and GPU: Rendering r/place History
Building and training a LLM
Music Synthesizer and Code DSL For Writing Music (with Rust Macros)

Shorter projects

a. Reverse Engineering -- Speed hack for Opus Magnum Zachtronics game (1 day) b. X11 Tiling Window Manager (1 day)

1. Operating Systems

I spent the past 5 years delving into low-level, assembly, compilers, CPUs, but I feel like I've only brushed up against the edges of an operating system, and have only gone in very shallow.

I've gotten the hang of using Linux syscalls from the Assembly level, but know little about how the syscalls are really implemented. I've gotten an x86_64 QEMU machine from starting out in 16-bit mode, and brought it into 64-bit mode: https://github.com/sumeet/x86-boot-probe. But haven't gotten past that part.

Here's my roadmap of goals that I would like to attain, that I'll use as my North Star for how to make progress on my project:

Achieve some sort of console that lets me navigate a filesystem. Maybe I won't have the concept of files and folders, but maybe a list programs that you can run. Not sure yet if I'll need to implement font rendering, or if I will just rely on text rendering provided by the system.
Be able to run programs. I want to start out super simple and be able to execute a "Hello, World" binary written for Linux.
After getting a Linux "Hello, World" working, maybe I can move onto exploring the lower depths of other operating system APIs. I can also implement for FreeBSD and Windows.
Another direction I could go in, is that I'm curious about the concept of virtual memory. So if the previous explorations don't get me to virtual memory, then I'll need to come up with a different project to exercise that feature.
Other topics I'd like to understand: scheduling, multiprocessing (multiple cores / CPUs), threading, "interrupts", other OS stuff going on behind the scenes that I'm not aware of.

I'm supposing there will be others at RC who are knowledgable about operating systems who I can lean on. But generally, I feel quite confident about this project, and I would expect this to go relatively smoothly without outside help.

2. Training Neural Networks (To Play Chess)

I have only a couple months introductory knowledge of machine learning. I followed a YouTube series called Machine Learning in C From Scratch by Tsoding, and I ported his neural network code from C to Rust. And then expanded the neural net to be able to train on the MNIST dataset and recognize handwritten digits: https://github.com/sumeet/nn-h

After that, I delved into chess, and started making a PyTorch model for predicting chess moves based on board position: https://github.com/sumeet/chess-stuff/blob/master/train/train.py (Sorry, it's just code w/o any explanation really. The gist is that I trained the model on moves made by winners of games from the Lichess Master Database, which includes only long-time-control games played by chess players with extremely high ratings. So basically I'm trying to teach it how to make moves like really good players make moves)

I put a lot of work into improving this model, but I'm having trouble making progress the way I'm working now, so I could use help in improving it. Here's my rough plan for making progress on this project:

For my existing PyTorch model, see if someone at RC is able to help me improve it. I don't want to sink too much time into modifying this particular model with only my current knowledge, because I've already sunk so much time into it with minimal results.
I would also like to try implementing Reinforcement Learning a'la AlphaZero. Since AlphaZero is super famous, I figure I should be able to find a lot of information about this online. I haven't put as much prep into this aspect, but given how famous this approach is, I expect I should be able to "figure it out" with a combination of Google, ChatGPT, and my existing ML and CompSci knowledge.

3. Graphics and GPU: Rendering reddit r/place history

Graphics and GPU programming feel like a big gap in my knowledge, and I have barely scratched the surface getting into GPU programming.

A friend at my local Rust user's group implemented a really interesting project that I'd like to try as well: https://maxisom.me/posts/applying-5-million-pixel-updates-per-second

He's using the pixel change data from the reddit.com/r/place (https://www.reddit.com/r/place/comments/txvk2d/rplace_datasets_april_fools_2022/) to render the canvas to the GPU. It feels like a somewhat sophisticated "Hello, World" project for GPU programming.

My plan of attack is to spend a few hours delving into the project, to scope it out. Google + ChatGPT, and referencing my friend's blog post and codebase when I get stuck.

Here are my goals:

Get the pixels rendering on screen, PERIOD
Get good framerate for screen updates
Add support for panning, zooming and seeking time via mouse

4. Building and training a LLM

Using PyTorch. There are several tutorials for this, including https://www.youtube.com/watch?v=kCc8FmEb1nY. I would like to try training my own, using my favorite books as training data. Similarly for this project, I have not done as much prep work and expect I'll be able to further flesh out this plan based on the popularity of LLMs and my existing Machine Learning knowledge. Famous last words, I know, but I feel confident.

5. Music Synthesizer and Code DSL For Writing Music (with Rust Macros)

This is similarly inspired by a Tsoding video, Making Music With Haskell: https://www.youtube.com/watch?v=FYTZkE5BZ-0

I'd like to learn more about synthesizing music, and I would also like to try my hand at coming up with a "musical language" (ie programming language), and also explore Rust's macro system more deeply. The Rust programming language (apparently) has a super expressive macro system, which lets you basically make arbitrarily complex macros, as if you were writing a programming language. I'd also like to take a look at music from the perspective of old-school video game music, and trackers, which I've been in-the-culture for since my early teens. Here's a demo of what tracker music looks like, and you may understand better the motivation behind wanting to make music out of simple waves: https://youtu.be/mW_IH54v-2k?t=9916

I've successfully synthesized a sine wave, but have not gotten deeper into doing music synthesis. Here's a rough plan of what I'd like to achieve in this project:

Achieve synthesis of common waves: sine, square, sawtooth and triangle
Achieve synthesis of effects (volume, vibrato, tremolo, reverb, attack, decay)
Create a macro system in Rust to be used for writing a song
And write a really simple song using that language!

a. Reverse Engineering -- Speed hack for Opus Magnum Zachtronics game (1-2 days)

Opus Magnum is a puzzle game for PC in which you build machines using a visual programming language to lay out code for operating the machines. I use this with two of my child students who I'm teaching programming, and for both of them, the simulation of the machine just runs too slowly. But clearly there's some way of speeding it up, because if you leave the simulation running for long enough, it starts speeding up, but not quite enough. The hack here is to find out how they're manipulating the speed through some C# reverse engineering, and see if we can permanently increase the speed of the simulation. I've worked with a friend before on reverse engineering Zachtronics games, and discovered that these are usually C# bytecode with obfuscated symbol names -- so maybe I'll be able to figure out some variables that if manipulated permanently increase the speed of the simulation.

b. X11 Tiling Window Manager (1-2 days)

Simple enough. This isn't an attempt to make a full-blown X11 Window Manager to replace the one I'm currently using. I'd like to just get a small intro into how the X11 Window Manager APIs work, and what the code for a tiling window manager could look like. I feel quite confident about being able to solve this, and have a starting point as a C library: https://github.com/mackstann/tinywm

I like to experience more of these "ancient" Linux APIs that have been around the entire time I've been using Linux, but just don't know much about myself.

Goals:

Get tinywm running
Devise a scheme to automatically arrange windows like in a tiling WM
Implement mouse controls for moving the windows around
No need for polish, this is just a toy / experiment

sumeet/rc proposal.md