Write Your Own OS(1) — Overview

4 min readJun 22, 2020

Next: Write Your Own OS(2) — Computer Architecture Overview

Abstract

This series of tutorials aims to provide a thorough introduction to the Operating Systems and Compilers. By following through the tutorial, you will have a good understanding of these two topics. In addition, you will develop a fully functional Operating System with input and output, threading, memory management, user program loading and execution, and a file system, as well as a custom designed programming language with a compiler. The compiler could be customized to generate machine code that can run on your own Operating System!

Introduction

Operating System and Compilers are arguably the most widely used tools by developers. We rely on the Operating System to write code, Compiler to transform high level program language into machine code and Operating System to execute our code. By understanding our most heavily used tools better, we should be able to

Be a better developer since Operating System and Compilers are themselves complicated and well engineered systems.
Use our daily tools better. For example we can better debug segment fault or tune garbage collection behaviors.
Satisfy our curiosity to know the magic happening behind the screen.
And many more…

Overview

This section provides an overview of the content of the tutorial.

Operating System

Part 1 Get a bare bone OS up and running

In this part, we will create a bare bone kernel and integrate with a loader to load our Operating System into the computer memory when it powers on. After that, we will print onto the screen and serial port as well as read keyboard inputs with interrupts.

Part 2 Thread

We will implement kernel threads. After that we will implement locks, semaphores and monitors with the help of interrupts.

Part 3 Memory management

This part illustrates how to manage physical memory and map virtual memory to physical memory with page tables.

Part 4 File system

We will have a fully functional file system after this section, including directories and files.

Part 5 User program

This section is divided into two parts. We will first work on user libraries to make System Call into the kernel and compile our user program. After that, we will extend our Operating System to load the user program into memory, set up the execution environment and transfer control to it.

Part 6 Move to 64-bit world

This is a lightweight section. We will introduce the differences between 32-bit and 64-bit and how to adapt the Operating System to make use of 64-bit hardwares.

By now, we should have a fully functional Operating System. In the next few parts, we will implement a compiler to transform our own programming language into machine code.

Compiler

Part 1 Lexical analysis with Yacc

In this section, we will tokenize our program based on some regular expression rules. We will understand how we remove bytes not useful for machines, such as comments and white spaces. We will introduce how to make use of Yacc to ease the development of a compiler.

Part 2 Syntax analysis with Yacc

Tokens generated in part 1 will be fed into a syntax analyzer, which generates an abstract syntax tree (AST). At this point, we will check for syntactic errors such as statements without ending semi-colons.

Part 3 Semantic analysis

Based on the AST, we will make sure the program has the correct semantics. Some errors include assigning string to an integer or assigning incompatible objects.

Part 4 Code generation

In this section, we will walk through the AST programmatically and generate code. Code can be object code, machine code, another programming language or anything you want.

Part 5 Garbage Collection

In this last part, we will discuss garbage collection. It is not required to implement the Operating System and Compiler. However it would be beneficial to know about garbage collection since it is very prevalent!

Final Results

Some screenshots of our final product are shown below. The Operation System, which we call Meg OS, initializes interrupts, memory and file system upon booting. It sets up threads to run concurrently as well.