So I’ve reached a point in time where it appears I’ve completed UNSW’s Advanced Operating Systems (COMP9242 2019/T2) course! After many nights of hopelessness towards the end of the course, here’s my thoughts.
Don’t do continuations in C… seriously
In the first few weeks of the course, we had to make a decision on the execution model of our operating system. Because we were building our OS on top of a microkernel (The seL4 Microkernel) we were free to choose any model of execution we wanted. In other words, we weren’t limited to the “kernel-on-top-of-userspace” that you encounter in normal OS courses.
Since the initially provided code came with a while loop, we thought the fastest way to get started was an event loop! Remembering that all system calls are IPC messages on a microkernel, if we had a while loop processing these messages already, then an event loop based server is just a simple switch-statement on top! Surely!
My AOS partner initially suggested that we try a multi-threaded model. He suggested that it’ll be easier to run long-running operations with a threaded model. But I convinced him that our velocity will be higher with the simple event loop model. Oh, how wrong I was…
Turns out, there is so much opportunity for an operating system to be blocked and require continuation at a later time. I thought it would just be limited to I/O syscalls but later I realised that because of demand paging, pretty much any kernel operation required continuations.
Later in the course, we had spent so much time converting our existing code-base to use continuations everywhere (i.e. lot of malloc, manually defining structs, turning loops into recursions and 2-state machines…) that we were burnt out from ever wanting to look at asynchronous C again…
Also, I realised too late that it’s not fun to write an operating system in this way. Having a single OS thread pretty much restricts you to a monolithic kernel, and we missed opportunities to experiment with a more creative design. (Admittedly, it was better to be kinda on-track than be creative, though.)
What I liked about the course
By far the best part about the course is the power you feel as you progress through it. If you have been puzzled by how systems software is written, and just what the hell those deep-down aspects of Linux and SRE war stories mean and how they actually work and how the hell people come to know about them (“are they just too smart?”), this course solves all your thirst issues!
By solving practical OS design issues where there is no-one to back you up (everyone’s project has different design choices – from execution models, to data structures, and project layout; the tutors can’t help you beyond basic guidelines), you have genuine exploratory discussions with your project partner, and you learn how to navigate trade-offs between simplicity of implementation, performance, and your own sanity.
The lectures aren’t too relevant for your project except for the first few weeks, but their contents are topics you wouldn’t really learn from “web learning” a.k.a. blog posts. The breadth of papers mentioned in the course and the final exam format (critique a published paper) makes it clear to you where you can now start to learn more.
Overall, I’m glad the course is over. “I can have my weekends back”, in the words of my project partner. I also won’t miss debugging where the instruction pointer has a bunch of f’s, when your muslibc mysteriously calls munmap, and looking through the 2000 page ARM manual to find the correct Fault Status Register documentation between all the EL levels… On the other hand, it’s been a course that has finally satiated my curiosity on what “OS-level programming” really means. I can’t wait for the distributed systems course now!
Bonus: Here’s what we had to document for our final submission.
Michael Yoo 