################################################################################ # # # fastcall-spma # # - # # A collection of helper scripts for reproducing fastcall benchmark results. # # # ################################################################################ This repository contains several scripts that reproduce the results of the fastcall benchmarks and plot the numbers as shown in publications. Locations ========= Beside the directories containing git submodules and the scripts for performing the benchmarks, the repository contains a directory called 'results' for storing the plain (CSV) output of the benchmarks and a directory called 'plots' for saving the final graphical output. Files included ============== install.sh - Compiles and installs the components necessary for executing the benchmarks. load_kernel.sh - Configures a new kernel for use after next power cycle. execute.sh - Performs the benchmarks. Setup ===== Make sure that you have installed all necessary packages for building Google's benchmarks library and the Linux kernel. You will also need Python 3 to run the plotting script. After cloning the repository, set up its submodules by executing git submodule update --init --recursive then, on some systems (but x86 only) you can run ./install.sh from the root of this repository. Note that errors during the compilation process of any component (e.g. due to missing packages or build tools named differently on your distribution (e.g. cmake3 instead of cmake) may require you to execute the commands listed in 'install.sh' manually. Take also care when configuring the fastcall and fccmp kernels that are needed to run the benchmarks. 'install.sh' will configure them with 'make defconfig', which might not be sufficient to create a working kernel image for your machine. If in doubt, execute the installation commands manually, configuring the kernel with 'make localmodconfig'. Also take care that all required file system and networking drivers are built. For instance, when using a machine from AWS, make sure that the drivers for Amazon's Elastic Network Adapter are enabled in the respective kernel configurations. Running the Benchmarks ====================== In order to run the benchmarks for your machine, execute ./execute.sh [run-micro | run-cycle | run-misc] from the command line. This will guide you through the benchmarks. Note that in order to complete all experiments, you will have to load different kernels and thus reboot the machine multiple times. Follow the instructions printed by the 'execute.sh' script. It will provide you with the correct command line that you should enter in order to load the appropriate kernel for continuing with the experiments. After rebooting the machine, continue with the command chosen from the options above. Repeat these steps, until 'execute.sh' exits with a success message. 'execute.sh' provides the following benchmark suites: run-micro - Run microbenchmarks for obtaining the latency for bare operations as well as memcpy examples for the vDSO, system call, ioctl, and fastcall mechanism. Results are computed in nanoseconds. run-cycle - Same as above but counts CPU cycles. run-misc - Conducts auxiliary benchmarks regarding the performance of fastcall (de-) registration as well as forking. Note that 'execute.sh' will only conduct the experiments for the CPU type present on the local machine. In order to obtain the all numbers, you may have to run the same procedure on different machines. Also, the 'load_kernel.sh' script will not work for some distributions. It currently only supports a GRUB config which has an "Advanced Options" menu that contains the non-default boot entries. 'load_kernel.sh' will ask you before updating the bootloader config, so you can intervene if things go wrong. In case that your bootloader config is not supported by 'load_kernel.sh', you will have to update your bootloader manually, setting the right kernel to boot next, as well as the appropriate kernel options for continuing with the benchmarks. By default, 'execute.sh' will skip benchmark runs for which results are already present in the 'results' directory. If you wish to reset the numbers collected for your local CPU type, run ./execute.sh reset Plotting ======== In order to generate plots from the data available in 'results', run python3 -m plotting Hints for Running the Benchmarks on ARM ======================================= Kernel Setup: Regardless of the CPU architecture, the fastcall kernel (subdir linux-fastcall) and the fccmp kernel (subdir linux-fccmp) must be built and installed. Furthermore, the fastcall mechanism is currently not supported for ARM kernels with mitigations. In order to build a working fastcall kernel on ARM, you thus have to configure it accordingly: Enable the expert settings ("General Setup"), then disable KAISER which currently contradicts fastcalls on ARM ("General Setup"). Afterwards, check whether fastcalls are actually enabled ("Device Drivers"). lscpu support: On ARM, Linux may not print out human-readable CPU information by default. However, the scripts in this repository rely on this information to be present (e.g. like human-readable vendor and model information in lscpu or /proc/cpuinfo). On ARM, you may need to install an alternative version of lscpu that supports decoding of CPU types and model names. For example, on AWS machines with Graviton CPUs, the following may work: git clone https://github.com/karelzak/util-linux cd util-linux ./autogen.sh ./configure --disable-all-programs --enable-lscpu \ --enable-libsmartcols make lscpu Install the resulting lscpu binary on your system.