CS 232 Project 5: Kernel Programming


This is to learn how system calls are implemented by implementing a one yourself. To do this, you will have to modify Linux kernel source files. The Linux kernel is written in C, not C++, and so you must do this project in C.

About the VM

We have seen that most Unix-variant operating systems are layered systems, with user-level programs at the top layer, and the OS kernel at the bottom layer, just above the host computer's actual hardware.

Our previous assigments have involved writing code that runs at the user's level -- in user mode. In this assignment, we will for the first time write code that runs at the kernel level -- in supervisor/kernel mode. As such, the code you write will have full control over and access to the kernel, its data structures, and the machine's hardware, including priveleged instructions.

Because the kernel is the foundation of the operating system, an error in the kernel's code may render the host computer unusable. Since

it would be potentially catastrophic for you to modify the actual kernel on one of our lab machines. (Also, we don't want to give out the root password for Linux on the lab machines.)

Instead, we will use the virtual machine approach we saw earlier in the course. More precisely, you will run your own personal copy of Lubuntu Linux on top of a virtual machine called VirtualBox. VirtualBox has been installed in the Systems Lab in the Windows 10 OS, or you can download and install VirtualBox on your personal machine and work there. Doing the latter is nice because then you don't have to spend so much time in the Systems Lab and you don't dominate a machine there that has to be shared between multiple courses (like CS338). However, doing the latter often does not work well because many laptops do not have enough memory.

Chris Wieringa has created an Open Virtualization Archive (OVA) file for use with VirtualBox. This OVA is a VirtualBox packaging of Lubuntu 16.04 x64 LTS. The virtual machine contains the base Lubuntu system, with the additions needed to recompile the Linux kernel.

Packages preinstalled:
- build-essential
- dkms
- kernel-package
- linux-headers
- libncurses5
- libncurses5-dev

In addition, all packages needed to rebuid the linux-image-generic-lts-vivid package were install by performing:
- apt-get build-dep linux-image-generic-lts-vivid

The mradmin user (and password: bluestone) is preconfigured with sudo access. You will need to sudo su to root to rebuild the kernel.


How to Start up VirtualBox and Run your VM (in the Syslab)

  1. Start up VirtualBox
  2. File --> Import Appliance
  3. Click the Expert Mode button.
  4. Appliance to import: C:\CS232\cs232-17sp-20170317-01.ova
  5. Double-click the CPU line, and change it to 8.
  6. Change RAM to 8192 MB
  7. Set the Virtual Disk Image to S:\cs232-17sp-20170317-01\cs232-17sp-20170317-01-disk001.vmdk
  8. Reinitialized the MAC address: checked
  9. Click Import
  10. Start it up. If it complains about the network card; just click the Intel 82579V Gigabit Network Connection.

To resume where you were next time, do the following process (once):

  1. Make sure your VM is shutdown.
  2. In the Virtualbox interface, right-click on your VM name and go to "Show in Explorer"
  3. Copy the cs232-17sp-20170317-01.vbox file to your S:\cs232-17sp-20170317-01\ folder.

To restart your VM:

  1. You'll need to re-add your VM to Virtualbox every boot.
  2. In Virtualbox, go to Machine -> Add.
    1. Navigate to S:\cs232-17sp-20170317-01.vbox
  3. Start up your VM, and continue your work.


How to Install and Compile Your Kernel

The kernel source for Lubuntu's kernel has been preloaded on the VM into /usr/src/linux-lts-vivid-3.19.0/ . In addition to Lubuntu's patches to the kernel tree, known bug #1460768 has been fixed in this kernel source.

1. sudo su
2. (root) cd /usr/src/ubuntu-xenial/

Fill in the following command as indicated with the < > symbols; for example, <name> could be: johncalvin Please note, only lowercase letters, numbers, and the + and - symbols may be used in the username.

NOTE: This command will take a LONG time -- between 20 and 70 minutes.

3. (root) make-kpkg -j 8 --initrd --append-to-version=<username> kernel-image kernel-headers


Once the package has been built, the kernel .deb files will be located in /usr/src/. Install the packages next.

5. (root) cd /usr/src/
6. (root) ls *.deb

> linux-headers-4.4.44-<blahblah>_amd64.deb
> linux-image-4.4.44-<blahblah>_amd64.deb

7. (root) dpkg -i *.deb

After the kernel has been installed and the initrd remade, it will be installed into /boot/. Additionally, it will be made the default kernel to boot within GRUB. At this point, you can reboot your machine to start running your new kernel.

8. (root) reboot

If you would like to remove your custom kernel, do the following:

1. sudo su
2. (root) dpkg --list | grep linux-image

- Note the package that you installed earlier; probably named something like:

3. (root) dpkg --list | grep linux-headers
- Again, note the package that you installed earlier; something like:

4. (root) apt-get remove <imagepackagename> <headerspackagename>
- Confirm the removal

If you are still running your custom kernel, you may want to reboot back to the original Ubuntu kernel.

The Assignment

The assignment is to create a fancy new system call, as described below. But, first I recommend you create a simple system call, called printteamname(). This system call will just output a value to the kernel log file /var/log/kern.log.

Creating a system call for Lubuntu 16.04 x86 involves these steps (file names below are relative to /usr/src/ubuntu-xenial/):

  1. Add a line to the arch/x86/entry/syscalls/syscall_64.tbl file.
  2. Define your system call function prototype in include/linux/syscalls.h
  3. Write the code for your system call by adding it to an existing file containing other system calls (e.g., kernel/sys.c). Examining the other system calls in this file may prove informative as to how you should define yours. Be sure to define it using the SYSCALL_DEFINE0 (or SYSCALL_DEFINE1, ...) directive. All your code has to do is call printk("Your team names here") and return 0.
  4. Create a header file in /usr/include/, that contains code to call syscall(<your new syscall # here>) and return the result.
  5. Write a test program in your home directory that calls your wrapper function, passing in the correct arguments, and handling the results.

After you have done these steps, do the make-kpkg and dpkg calls from above, to compile the OS and re-install it. Then compile your .c file in your home directory and run the executable. Then, inspect /var/log/kern.log to see if your string was written to the file.

Once you get this done, you can go on to the real assignment:

Your assignment is to create a system call that queries the process control blocks in Lubuntu and returns information about multiple processes.

Specifically, your code will:

struct taskinfo {
pid_t pid; /* process id */ pid_t parent_pid; /* process id of parent */
pid_t first_child_pid; /* pid of youngest child */
long state; /* current state of process */
long uid; /* user id of process owner *
char comm[64]; /* name of program executed */ int level; /* the output indentation level */


My basic algorithm for the system call is:

Many Useful Hints


Turn in: You should submit in /home/cs/232/current/<yourid>/proj5/ the following:

To save disk space, please delete parts of large source files, leaving the context surrounding your modifications. When you do this, please INSERT TEXT INTO YOUR SCRIPT FILE TO highlight your modifications (e.g., where extraneous stuff was cut out), so that the grader can find the good stuff easily. E.g., you might put this into the script file:


Due date: Friday, Apr. 28, at 11:59:59.9 pm.

Warning: This is the most time-consuming project of the semester. You will likely need the entire time to get it working correctly. Begin work on it immediately!

Calvin > CS > 232 > Projects > 4

This page maintained by Victor Norman.