Home / Linux / What Does It Take to Make a Kernel?

What Does It Take to Make a Kernel?

The kernel this. The kernel that. People
often refer to one operating system’s kernel or another without
truly knowing what it does or how it works or what it takes to make
one. What does it take to write a custom (and non-Linux) kernel?

So, what am I going to do here? In June 2018, I wrote a guide
to build a complete Linux distribution from source packages, and in
January 2019, I expanded
on that guide by adding more packages to
the original guide. Now it’s time to dive deeper into the custom
operating system topic. This article describes how to write your very own kernel from scratch
and then boot up into it. Sounds pretty straightforward, right? Now,
don’t get too excited here. This kernel won’t do much of anything.
It’ll
print a few messages onto the screen and then halt the CPU. Sure,
you can build on top of it and create something more, but that is not
the purpose of this article. My main goal is to provide you, the reader,
with a deep understanding of how a kernel is written.

Once upon a time, in an era long ago, embedded Linux was not really a
thing. I kno that sounds a bit crazy, but it’s true! If you worked with
a microcontroller, you were given (from the vendor) a specification, a
design sheet, a manual of all its registers and nothing more. Translation:
you had to write your own operating system (kernel included)
from scratch.
Although this guide assumes the standard generic
32-bit x86 architecture, a lot of it reflects what had to be done
back in the day.

The exercises below require that you install a few packages in
your preferred Linux distribution. For instance, on an Ubuntu machine,
you will need the following:

  • binutils
  • gcc
  • grub-common
  • make
  • nasm
  • xorriso

An Extreme Crash Course into the Assembly Language

Note: I’m going to simplify things by pretending to work with
a not-so-complex 8-bit microprocessor. This doesn’t
reflect the modern (and possibly past) designs of any commercial
processor.

When the designers of a microprocessor create a new chip, they will
write some very specialized microcode for it. That microcode will
contain defined operations that are accessed via operation codes or
opcodes. These defined opcodes contain instructions
(for the microprocessor) to add, subtract, move values and addresses
and more. The processor will read those opcodes as part of a larger
command format. This format will consist of fields that hold a series of
binary numbers—that is, 0s and 1s. Remember, this processor
understands only high (the 1s) and low (the 0s) signals, and when those
signals (as part of an instruction) are fed to it in the proper sequence,
the processor will parse/interpret the instruction and then execute it.

Here’s the rundown of the command structure for the made-up processor:

  • 0, 1 — opcode
  • 2, 3 — source 1
  • 4, 5 — source 2
  • 6, 7 — destination

Now, what exactly is assembly language? It’s as close to machine code as
you can
get when programming a microprocessor. It is human-readable code based
on the machine’s supported instruction set and not just a series of
binary numbers. I guess you could memorize all the
binary numbers (in their proper sequence) for every instruction, but it
wouldn’t make much sense, especially if you can simplify code writing
with more human-readable commands.

This make-believe and completely unrealistic processor supports only four
instructions of which the ADD instruction maps to an opcode of 00
in binary code, and SUB (or subtract) maps to an opcode of 01 in
binary. You’ll be accessing four total CPU memory registers: A or 00,
B or 01, C or 10 and D or 11.

Using the above command structure, your compiled code will send the
following instruction:


ADD A, B, C

Or, “add the contents of A and B and store them into register C” in
the following binary machine language format:


00000110

Let’s say you want to subtract A from C and store it in the B
register. The human-readable code would look like the following:


SUB C, A, D

And, it will translate to the following machine code for the processor’s
microcode to process:


01100011

As you would expect, the more advanced the chip (16-bit, 32-bit,
64-bit), the more instructions and larger address spaces are supported.

The Boot Code

The assembler I’m using in this tutorial is called NASM. The open-source
NASM, or the Net-Wide Assembler, will assemble
the assembly code into a file format called object code. The object file
generated is an intermediate step to produce the executable binary or
program. The reason for this intermediate step is that a single large
source code file may end up being cut up into smaller source code files
to make them more manageable in both size and complexity. For instance,
when you compile the C code, you’ll instruct the C compiler to produce
only an object file. All object code (created from your ASM and C files)
will form bits and pieces of your kernel. To finalize the compilation,
you’ll use a linker to take all necessary object
files, combine them, and then produce the program.

The following code should be written to and saved in a file named
boot.asm. You should store the file in the dedicated working
directory for the project.

boot.asm


bits 32

section .multiboot               ;according to multiboot spec
        dd 0x1BADB002            ;set magic number for
                                 ;bootloader
        dd 0x0                   ;set flags
        dd - (0x1BADB002 + 0x0)  ;set checksum

section .text
global start
extern main                      ;defined in the C file

start:
        cli                      ;block interrupts
        mov esp, stack_space     ;set stack pointer
        call main
        hlt                      ;halt the CPU

section .bss
resb 8192                        ;8KB for stack
stack_space:

So, this looks like a bunch of nonsensical gibberish, right? It
isn’t. Again, this is supposed to be human-readable code. For instance,
under the multiboot section, and in the proper order of the
multiboot specification (refer to the section labeled “References”
below), you’re defining three double words variables. Wait,
what? What is a double word? Well, let’s take a step back. The
assembly DD pseudo-instruction translates to Define Double (word),
which on an x86 32-bit system is 4 bytes (32-bits). A DW or Define
Word is 2 bytes (or 16 bits), and moving even further backward, a DB or
Define Byte is 8-bits. Think of it as your integers,
short and long in your high-level
coding languages.

Note: pseudo-instructions are not real x86 machine instruction. They
are special instructions supported by the assembler and for the assembler
to help facilitate memory initialization and space reservation.

Below the multiboot section, you have a section labeled
text, which is shortly followed by a function labeled
start. This start function will set up the
environment for your main kernel code and then execute that kernel
code. It starts with a cli. The CLI command, or Clear
Interrupts Flag, clears the IF flag in the EFLAGS register. The following
line moves the empty stack_space function into the Stack
Pointer. The Stack Pointer is small register on the microprocessor
that contains the address of your program’s last request from a
Last-In-First-Out (LIFO) data buffer referred to as a Stack. The example assembly
program will call the main function defined in your C file
(see below) and then halt the CPU. If you look above, this is telling
the assembler via the extern main line that the code for
this function exists outside this file.

The Kernel’s Main Function

So, you wrote your boot code, and your boot code knows that there is an external
main function it needs to load into, but you don’t have an
external main function—at least, not yet. Create a file
in the same working directory, and name it kernel.c. The
file’s contents should be the following:

kernel.c


#define VGA_ADDRESS 0xB8000   /* video memory begins here. */

/* VGA provides support for 16 colors */
#define BLACK 0
#define GREEN 2
#define RED 4
#define YELLOW 14
#define WHITE_COLOR 15

unsigned short *terminal_buffer;
unsigned int vga_index;

void clear_screen(void)
{
    int index = 0;
    /* there are 25 lines each of 80 columns;
       each element takes 2 bytes */
    while (index < 80 * 25 * 2) {
            terminal_buffer[index] = ' ';
            index += 2;
    }
}

void print_string(char *str, unsigned char color)
{
    int index = 0;
    while (str[index]) {
            terminal_buffer[vga_index] = (unsigned
             ↪short)str[index]|(unsigned short)color << 8;
            index++;
            vga_index++;
    }
}

void main(void)
{
    /* TODO: Add random f-word here */
    terminal_buffer = (unsigned short *)VGA_ADDRESS;
    vga_index = 0;

    clear_screen();
    print_string("Hello from Linux Journal!", YELLOW);
    vga_index = 80;    /* next line */
    print_string("Goodbye from Linux Journal!", RED);
    return;
}

If you scroll all the way to the bottom of the C file and look inside the
main function, you’ll notice it does the following:

  • Assigns the start address of your video memory to the string buffer.
  • Resets your internal location marker for where you are in that string
    buffer.
  • Clears the terminal screen.
  • Prints a message (in one color).
  • Sets your internal location marker for the next line.
  • Prints another message (in another color).
  • And, returns back to the boot code (where, if you recall, it halts the
    CPU).

In the current x86 architecture, your video memory is running in protected
mode and starts at memory address 0xB8000. So, everything
video-related will start from this address space and will support up
to 25 lines with 80 ASCII characters per line. Also, the video mode in
which this is running supports up to 16 colors (of which I added a
few to play with at the top of the C file).

Following these video definitions, a global array is defined to
map to the video memory and an index to know where you are in that video
memory. For instance, the index starts at 0, and if you want to move to
the first character space of the next line on the screen, you’ll need
to increase that index to 80, and so on.

As the names of the following two functions imply, the first clears
the entire screen with an ASCII empty character, and the second
writes whatever string you pass into it. Note that the expected
input for the video memory buffer is 2 bytes per character. The first
of the two is the character you want to output, while the second is
the color. This is made more obvious in the print_string() function,
where the color code is actually passed into the function.

Anyway, following those two functions is the main routine
with its actions already mentioned above. Remember, this is a learning
exercise, and this kernel will not do anything special other than print
a few things to the screen. And aside from adding real functions, this
kernel code is definitely missing some profanity. (You can add
that later.)

In the real world…

Every kernel will have a main() routine (spawned by
a bootloader), and within that main routine, all the proper system
initialization will take place. In a real and functional kernel, the
main routine eventually will drop into an infinite while()
loop where all future kernel functions take place or spawn a thread
accomplishing pretty much the same thing. Linux does this as well. The
bootloader will call the start_kernel() routine found in
init/main.c, and in turn, that routine will spawn an init
thread.

Linking It All Together

As mentioned previously, the linker serves a very important purpose. It
is what will take all of the random object files, put them together and
provide a bootable single binary file (your kernel).

linker.ld


OUTPUT_FORMAT(elf32-i386)
ENTRY(start)
SECTIONS
 {
   . = 1M;
   .text BLOCK(4K) : ALIGN(4K)
   {
       *(.multiboot)
       *(.text)
   }
   .data : { *(.data) }
   .bss  : { *(.bss)  }
 }

Let’s set the output format to be a 32-bit x86 executable. The entry
point into this binary is the start function from your
assembly file, which eventually loads the main program
from the C file. Further down, this essentially is telling the linker
how to merge your object code and at what offset. In the linker file,
you explicitly specify the address in which to load your kernel binary. In
this
case, it is at 1M or a 1 megabyte offset. This is where the main kernel
code is expected to be, and the bootloader will find it here when it is
time to load it.

Booting the Kernel

The most exciting part of the effort is that you can piggyback off the
very popular GRand Unified Bootloader (GRUB) to load your kernel. In order
to do this, you need to create a grub.cfg file. For the
moment, write the following contents into a file of that name, and save
it into your current working directory. When the time comes to build your ISO
image, you’ll install this file into its appropriate directory path.


grub.cfg


set timeout=3

menuentry "The Linux Journal Kernel" {
        multiboot /boot/kernel
}

Compilation Time

Build the boot.asm into an object file:


$ nasm -f elf32 boot.asm -o boot.o

Build the kernel.c into an object file:


$ gcc -m32 -c kernel.c -o kernel.o

Link both object files and create the final executable program (that is, your
kernel):


$ ld -m elf_i386 -T linker.ld -o kernel boot.o kernel.o

Now, you should have a compiled file in the same working directory labeled
kernel:


$ ls
boot.asm  boot.o  grub.cfg  kernel  kernel.c  kernel.o
 ↪linker.ld

This file is your kernel. You’ll be booting into that kernel shortly.

Building a Bootable ISO Image

Create a staging environment with the following directory path (from
your current working directory path):


$ mkdir -p iso/boot/grub

Let’s double-check that the kernel is a multiboot file type (no output
is expected with a return code of 0):


$ grub-file --is-x86-multiboot kernel

Now, copy the kernel into your iso/boot directory:


$ cp kernel iso/boot/

And, copy your grub.cfg into the iso/boot/grub
directory:


$ cp grub.cfg iso/boot/grub/

Make the final ISO image pointing to your iso subdirectory
in your current working directory path:

$ grub-mkrescue -o my-kernel.iso iso/
xorriso 1.4.8 : RockRidge filesystem manipulator,
 ↪libburnia project.

Drive current: -outdev 'stdio:my-kernel.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 10.3g free
Added to ISO image: directory '/'='/tmp/grub.fqt0G4'
xorriso : UPDATE : 284 files added in 1 seconds
Added to ISO image: directory
 ↪'/'='/home/petros/devel/misc/kernel/iso'
xorriso : UPDATE : 288 files added in 1 seconds
xorriso : NOTE : Copying to System Area: 512 bytes from file
 ↪'/usr/lib/grub/i386-pc/boot_hybrid.img'
ISO image produced: 2453 sectors
Written to medium : 2453 sectors at LBA 0
Writing to 'stdio:my-kernel.iso' completed successfully.

Additional Notes

Say you want to expand on this tutorial by
automating the entire process of building the final image. The best way
to accomplish this is by throwing a Makefile into the project’s root
directory. Here’s an example of what that Makefile would look like:


Makefile


CP := cp
RM := rm -rf
MKDIR := mkdir -pv

BIN = kernel
CFG = grub.cfg
ISO_PATH := iso
BOOT_PATH := $(ISO_PATH)/boot
GRUB_PATH := $(BOOT_PATH)/grub

.PHONY: all
all: bootloader kernel linker iso
    @echo Make has completed.

bootloader: boot.asm
    nasm -f elf32 boot.asm -o boot.o

kernel: kernel.c
    gcc -m32 -c kernel.c -o kernel.o

linker: linker.ld boot.o kernel.o
    ld -m elf_i386 -T linker.ld -o kernel boot.o kernel.o

iso: kernel
    $(MKDIR) $(GRUB_PATH)
    $(CP) $(BIN) $(BOOT_PATH)
    $(CP) $(CFG) $(GRUB_PATH)
    grub-file --is-x86-multiboot $(BOOT_PATH)/$(BIN)
    grub-mkrescue -o my-kernel.iso $(ISO_PATH)

.PHONY: clean
clean:
        $(RM) *.o $(BIN) *iso

To build (including the final ISO image), type:


$ make

To clean all of the build objects, type:


$ make clean

The Moment of Truth

You now have an ISO image, and if you did everything correctly, you should
be able to boot into it from a CD on a physical machine or in a virtual
machine (such as VirtualBox or QEMU). Start the virtual machine after
configuring its profile to boot from the ISO. You’ll immediately be
greeted by GRUB (Figure 1).

""

Figure 1. The GRUB Bootloader
Counting Down to Load the Kernel

After the timeout elapses, the kernel will boot.

""

Figure 2.
The Linux Journal kernel
booted. Yes, it does only this.

Summary

You did it! You wrote your very own kernel from scratch. Again, it doesn’t
do much of anything, but you definitely can expand upon this. Now,
if you will excuse me, I need to post a message to the USENET newsgroup,
comp.os.minix, about how I developed a new kernel, and that it
won’t be big and professional like GNU.

Resources

Loading...

>> Source Link

Check Also

Smart plugs, thermostats, and more

Whether it’s controlling your household lights, operating a smart lock, or rolling a digital die …

%d bloggers like this: