Cpp on rndthts.dev

Compilation Process - Compilation

Sun, 09 Feb 2025 10:12:45 +0100

In the last blog post, we covered the Preprocessing stage. Now, it’s time to dive into the Compilation stage.

What is Compilation?

Compilation is the process of converting preprocessed source code (in our case, C or C++ code) into assembly code. This phase is handled by the compiler, which performs various checks and optimizations before generating low-level code.

Unlike the preprocessing phase, which merely modifies the code textually, compilation involves deeper analysis and transformation.

Note: To stop the compilation process after the compilation phase, use the -S flag:

gcc/clang -S main.c -o main.s

Let’s explore this with an example:

foo.h

int foo(void);

main.c

#include "foo.h"

int square(int i) { return i * i; }

int main() {
  int a = 5;
  int f = foo();
  return f + square(a);
}

Key Stages of Compilation

1. Lexical Analysis (Tokenization)

The preprocessed code is broken down into tokens. For example, the line:

int a = 5;

will be tokenized into: int, a, =, 5 and ;

This stage eliminates whitespace and detects unknown symbols. For instance:

int    a    = 5  ;

will still end up with the same tokens int, a, =, 5 and ;

However, this will cause an error: int a = 5 @ 3;

error: expected ';' at end of declaration
    6 |   int a = 5 @3;
      |            ^
      |

2. Syntax Analysis (Parsing)

Tokens are grouped based on grammar rules to form an Abstract Syntax Tree (AST).
The AST represents the hierarchical syntactic structure of the code, abstracting unnecessary details like semicolons or parentheses.
This phase ensures that the code structure follows language syntax. Example: int a = 5 // Missing the ;

output:

error: expected ';' at end of declaration
    6 |   int a = 5 // Missing the ;
      |            ^
      |

3. Semantic Analysis

This phase checks for logical correctness:

Ensures variables are declared before use.

int main() {
  // int a = 5;
  int f = foo();
  return f + square(a);
}

 error: use of undeclared identifier 'a'
    8 |   return f + square(a);
      |                     ^

Ensures type compatibility (e.g., no assigning a string to an integer).

int a = "hello";

Error:

error: incompatible pointer to integer conversion initializing 'int' with an expression of type 'char[6]' [-Wint-conversion]
    6 |   int a = "hello";
      |       ^   ~~~~~~~

4. IR - Intermediate Representation

The compiler converts the AST into an Intermediate Representation (IR) that is platform-independent.
IR serves as a bridge between high-level source code and low-level machine code, enabling optimizations.

In future posts, we’ll cover this topic in depth, along with various optimizations. For now, let’s see how to instruct the compiler to emit this form.

To generate LLVM IR using clang:

clang main.c -S -emit-llvm -o main.ll`

Unoptimized Version:

; ModuleID = 'main.c'
source_filename = "main.c"
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128-Fn32"
target triple = "arm64-apple-macosx14.0.0"

; Function Attrs: noinline nounwind optnone ssp uwtable(sync)
define i32 @square(i32 noundef %0) #0 {
  %2 = alloca i32, align 4
  store i32 %0, ptr %2, align 4
  %3 = load i32, ptr %2, align 4
  %4 = load i32, ptr %2, align 4
  %5 = mul nsw i32 %3, %4
  ret i32 %5
}

; Function Attrs: noinline nounwind optnone ssp uwtable(sync)
define i32 @main() #0 {
  %1 = alloca i32, align 4
  %2 = alloca i32, align 4
  %3 = alloca i32, align 4
  store i32 0, ptr %1, align 4
  store i32 5, ptr %2, align 4
  %4 = call i32 @foo()
  store i32 %4, ptr %3, align 4
  %5 = load i32, ptr %3, align 4
  %6 = load i32, ptr %2, align 4
  %7 = call i32 @square(i32 noundef %6)
  %8 = add nsw i32 %5, %7
  ret i32 %8
}

declare i32 @foo() #1

attributes #0 = {...}
attributes #1 = {...}
...
...

Compiler Optimizations

The compiler optimizes the IR before producing assembly code. Common optimizations include:

Function Inlining
Dead Code Elimination
Loop Unrolling
Loop Invariant Code Motion
Merge Functions
etc …

We’ll explore these in future posts.

Final result, arm64 assembly

linux

	.text
	.file	"main.c"
	.globl	square                          // -- Begin function square
	.p2align	2
	.type	square,@function
square:                                 // @square
	.cfi_startproc
// %bb.0:
	sub	sp, sp, #16
	.cfi_def_cfa_offset 16
	str	w0, [sp, #12]
	ldr	w8, [sp, #12]
	ldr	w9, [sp, #12]
	mul	w0, w8, w9
	add	sp, sp, #16
	.cfi_def_cfa_offset 0
	ret
.Lfunc_end0:
	.size	square, .Lfunc_end0-square
	.cfi_endproc
                                        // -- End function
	.globl	main                            // -- Begin function main
	.p2align	2
	.type	main,@function
main:                                   // @main
	.cfi_startproc
// %bb.0:
	sub	sp, sp, #32
	.cfi_def_cfa_offset 32
	stp	x29, x30, [sp, #16]             // 16-byte Folded Spill
	add	x29, sp, #16
	.cfi_def_cfa w29, 16
	.cfi_offset w30, -8
	.cfi_offset w29, -16
	stur	wzr, [x29, #-4]
	mov	w8, #5                          // =0x5
	str	w8, [sp, #8]
	bl	foo
	str	w0, [sp, #4]
	ldr	w8, [sp, #4]
	str	w8, [sp]                        // 4-byte Folded Spill
	ldr	w0, [sp, #8]
	bl	square
	ldr	w8, [sp]                        // 4-byte Folded Reload
	add	w0, w8, w0
	.cfi_def_cfa wsp, 32
	ldp	x29, x30, [sp, #16]             // 16-byte Folded Reload
	add	sp, sp, #32
	.cfi_def_cfa_offset 0
	.cfi_restore w30
	.cfi_restore w29
	ret
.Lfunc_end1:
	.size	main, .Lfunc_end1-main
	.cfi_endproc
                                        // -- End function
	.ident	"Ubuntu clang version 18.1.3 (1ubuntu1)"
	.section	".note.GNU-stack","",@progbits
	.addrsig
	.addrsig_sym square
	.addrsig_sym foo

macos:

	.section	__TEXT,__text,regular,pure_instructions
	.build_version macos, 14, 0	sdk_version 14, 4
	.globl	_square                         ; -- Begin function square
	.p2align	2
_square:                                ; @square
	.cfi_startproc
; %bb.0:
	sub	sp, sp, #16
	.cfi_def_cfa_offset 16
	str	w0, [sp, #12]
	ldr	w8, [sp, #12]
	ldr	w9, [sp, #12]
	mul	w0, w8, w9
	add	sp, sp, #16
	ret
	.cfi_endproc
                                        ; -- End function
	.globl	_main                           ; -- Begin function main
	.p2align	2
_main:                                  ; @main
	.cfi_startproc
; %bb.0:
	sub	sp, sp, #32
	stp	x29, x30, [sp, #16]             ; 16-byte Folded Spill
	add	x29, sp, #16
	.cfi_def_cfa w29, 16
	.cfi_offset w30, -8
	.cfi_offset w29, -16
	stur	wzr, [x29, #-4]
	mov	w8, #5                          ; =0x5
	str	w8, [sp, #8]
	bl	_foo
	str	w0, [sp, #4]
	ldr	w8, [sp, #4]
	str	w8, [sp]                        ; 4-byte Folded Spill
	ldr	w0, [sp, #8]
	bl	_square
	ldr	w8, [sp]                        ; 4-byte Folded Reload
	add	w0, w8, w0
	ldp	x29, x30, [sp, #16]             ; 16-byte Folded Reload
	add	sp, sp, #32
	ret
	.cfi_endproc
                                        ; -- End function
.subsections_via_symbols

That’s it for this post! In the next one, we’ll cover the Assembler phase.

Compilation Process - Preprocessing

Wed, 05 Feb 2025 00:52:25 +0100

The compilation process consists of four key steps, as shown in the image below. In this series of blog posts, we’ll explore each of those steps, highlighting the most important aspects and sharing interesting insights along the way. 😊

Preprocessing (c/cpp -> .i)

The first phase is preprocessing. The input is a .c or .cpp file, and the output is a .i file. Let’s dive into what happens during this stage. To illustrate, we’ll use the following example:

Example 1

foo.h

// This is a comment in header file
int foo(void);

main.c

#include "foo.h"

// We have a macro
#define SQUARE(x) (x * x)

#define PI 3.14

/* Our main function
 * with multi-line comment
 */
int main() {
  int result = foo() + SQUARE(3);
  float area = PI * 5 * 5;
  return result + area;
}

Note: The compilation process can be stopped at any step. In this case, we want to stop it after preprocessing, and to do that, we use the -E flag.

gcc -E main.c -o main.i

The result:

main.i

# 1 "main.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 418 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "main.c" 2
# 1 "./foo.h" 1


int foo(void);
# 2 "main.c" 2
# 11 "main.c"
int main() {
  int result = foo() + (3 * 3);
  float area = 3.14 * 5 * 5;
  return result + area;
}

Let’s observe the result:

Header File Inclusion - The preprocessor replaces #include directives with the actual contents of the specified header files.
Line Control Statements (# Directives)
Comments are remove ( single and multi line)
Macro Expansion (#define) - The preprocessor replaces macros with their defined values.

int result = foo() + SQUARE(3); -> int result = foo() + (3 * 3);
float area = PI * 5 * 5; -> float area = 3.14 * 5 * 5;

Example 2

#ifdef DEBUG
#include <stdio.h>
#endif

int main() {
    #ifdef DEBUG
    printf("Debug mode is enabled.\n");
    #endif
    return 0;
}

Conditional Compilation (#ifdef, #ifndef, #endif) Since DEBUG was not defined, the output after preprocessing looks like this:

...

int main() {
  return 0;
}

A well-known application of conditional compilation is include guards. Consider this simplified example:

#include "foo.h"
#include "foo.h"
#include "foo.h"
#include "foo.h"

int main() { 
  return 0; 
}

Without include guards, the output after preprocessing might look something like this:

...
int foo(void);
# 2 "main_multi.c" 2
# 1 "./foo.h" 1


int foo(void);
# 3 "main_multi.c" 2
# 1 "./foo.h" 1


int foo(void);
# 4 "main_multi.c" 2
# 1 "./foo.h" 1


int foo(void);
# 5 "main_multi.c" 2

int main() {
  return 0;
}

Of course, this is a silly example, but in larger projects, this issue can happen unintentionally. For example:

#include "foo.h"
#include "bar.h"
#include "tar.h"

If both bar.h and tar.h also include foo.h, the compiler processes foo.h multiple times, which can lead to redefinition errors.

To prevent multiple inclusions, we use include guards like this:

#ifndef FOO_H
#define FOO_H

int foo(void);

#endif // FOO_H

Example 3

#include <stdio.h>

#define CONCAT(a, b) a##b

int main() {
  int CONCAT(my, Var) = 10; // Expands to int myVar = 10;
  printf("File: %s, Line: %d\n", __FILE__, __LINE__);

  return 0;
}

The result:

// Ignored the stdio.h

int main() {
  int myVar = 10;
  printf("File: %s, Line: %d\n", "main.c", 7);

  return 0;
}

Token Pasting (##) and Stringizing (#)
Including Built-in Macros in our example __FILE__ and __LINE__

The next step in the process is compilation. Stay tuned for the next post!

Writing your first pass with LLVM

Fri, 04 Aug 2023 23:59:01 +0200

What is a pass? Simply put, it is a thing that analyzes or transforms LLVM IR inputs.

What are we going to use? We will use opt. What is opt?

The opt command is the modular LLVM optimizer and analyzer. It takes LLVM source files as input, runs the specified optimizations or analyses on it, and then outputs the optimized file.

This is so-called middle end. The input to the opt is LLVM IR and the output is alsoLLVM IR

Before we start writing our first pass, we need llvm. We can install it or build it ourselves. I used LLVM 16.

We can install llvm using: brew install llvm on MacOS or apt install llvm on Ubuntu. If you decide to build llvm, you can follow the official instructions here: https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm

From this point I will refere to llvm installation directory as LLVM_PATH. To easier follow the rest of the stuff, you can just export LLVM_PATH In my case it was export LLVM_PATH=/opt/homebrew/opt/llvm@16/ To test this, we can invoke: $LLVM_PATH/bin/opt --version and this should return something similar to this:

Homebrew LLVM version 16.0.6
  Optimized build.
  Default target: arm64-apple-darwin22.5.0
  Host CPU: apple-m1

Let’s start from the point of how are we going to invoke opt

$LLVM_PATH/bin/opt -load-pass-plugin hello.dylib -passes=first -disable-output path/to/input.ll

We will have to provide our plugin, where our pass (or multiple passes) is defined, and a list of the passes. The plugin is a shared library. opt will load our plugin using dlopen and it will try to find a predefined function name in that plugin to set up the plugin. In this case, it will look for llvmGetPassPluginInfo function to initialize the plugin.

From the official documentation

/// The public entry point for a pass plugin.
///
/// When a plugin is loaded by the driver, it will call this entry point to
/// obtain information about this plugin and about how to register its passes.
/// This function needs to be implemented by the plugin, see the example below:
///
/// ```
/// extern "C" ::llvm::PassPluginLibraryInfo LLVM_ATTRIBUTE_WEAK
/// llvmGetPassPluginInfo() {
///   return {
///     LLVM_PLUGIN_API_VERSION, "MyPlugin", "v0.1", [](PassBuilder &PB) { ... }
///   };
/// }
/// ```

This means that our plugin should implement this function. Let’s quickly check the return type:

struct PassPluginLibraryInfo {
  uint32_t APIVersion;
  const char *PluginName;
  const char *PluginVersion;
  /// The callback for registering plugin passes with a \c PassBuilder
  /// instance
  void (*RegisterPassBuilderCallbacks)(PassBuilder &);
};

So, the minimal version of our plugin looks like this: first.cpp:

#include "llvm/Passes/PassBuilder.h"
#include "llvm/Passes/PassPlugin.h"
#include "llvm/Support/raw_ostream.h"

extern "C" LLVM_ATTRIBUTE_WEAK ::llvm::PassPluginLibraryInfo
llvmGetPassPluginInfo() {
  return {LLVM_PLUGIN_API_VERSION, "First", "0.1", [](llvm::PassBuilder &PB) {
            llvm::errs() << "Register pass builder callback :)\n";
          }};
}

We can build it from command line using:

$LLVM_PATH/bin/clang++ -std=c++17 first.cpp -shared -fno-rtti -fno-exceptions `$LLVM_PATH/bin/llvm-config --cppflags --ldflags --system-libs --libs core` -o first

Before invoking opt, we need the input LLVM IR file. In our case, We will create a simple c source file and compile it to LLVM IR. foo.c

int foo(int a, int b) { 
  return a + b; 
}

Compile it using:

$LLVM_PATH/bin/clang -c -S -emit-llvm foo.c

Run opt with this command

$LLVM_PATH/bin/opt -load-pass-plugin first -passes=hello -disable-output foo.ll

And, we should get something like this:

Register pass builder callback :)
/opt/homebrew/opt/llvm@16/bin/opt: unknown pass name 'hello'

Now we can update the implementation of llvmGetPassPluginInfo to register our pass.

extern "C" LLVM_ATTRIBUTE_WEAK ::llvm::PassPluginLibraryInfo
llvmGetPassPluginInfo() {
  return {LLVM_PLUGIN_API_VERSION, "First", "0.1", [](llvm::PassBuilder &PB) {
            PB.registerPipelineParsingCallback(
                [](llvm::StringRef name, llvm::FunctionPassManager &FPM,
                   llvm::ArrayRef<llvm::PassBuilder::PipelineElement>) -> bool {
                  if (name == "hello") {
                    FPM.addPass(HelloPass{});
                    return true;
                  }
                  return false;
                });
          }};
}

registerPipelineParsingCallback has several overloads, but for now, we are interested in function pass, as you can see from the signature of provided lambda.

llvm::StringRef name, llvm::FunctionPassManager &FPM,
                   llvm::ArrayRef<llvm::PassBuilder::PipelineElement>

If the provided pass name is hello, we instantiate HelloPass. So, let’s see the definition of the pass.

struct HelloPass : llvm::PassInfoMixin<HelloPass> {
  llvm::PreservedAnalyses run(llvm::Function &F,
                              llvm::FunctionAnalysisManager &) {
    llvm::errs() << "Function name: " << F.getName() << '\n';
    return llvm::PreservedAnalyses::all();
  }

  static bool isRequired() { return true; }
};

This pass is simple, it will run on functions and print function names. We added our pass to the llvm::FunctionPassManager so that the manager will invoke run on our pass, providing us llvm::Function and llvm::FunctionAnalysisManager In our case we do not even use llvm::FunctionAnalysisManager

We do not change anything when our pass is invoked, so we return llvm::PreservedAnalyses::all()

If we now compile again our plugin

$LLVM_PATH/bin/clang++ -std=c++17 first.cpp -shared -fno-rtti -fno-exceptions `$LLVM_PATH/bin/llvm-config --cppflags --ldflags --system-libs --libs core` -o first

run opt

$LLVM_PATH/bin/opt -load-pass-plugin first -passes=hello -disable-output foo.ll

we get:

Function name: foo

Now let’s update our pass to print the number of arguments as well:

    llvm::errs() << "Arg size: " << F.arg_size() << '\n';

also, let’s add more functions to our foo.c file

int foo(int a, int b) { return a + b; }
double bar(int a, char b, short c) { return a + b + c; }
int tar() { return 42; }

Compile our plugin! Compile foo.c to LLVM IR. Run opt, we will get:

Function name: foo
Function Arg size: 2
Function name: bar
Function Arg size: 3
Function name: tar
Function Arg size: 0

How to inherit from lambdas?

Wed, 13 Oct 2021 14:54:53 +0100

What happens behind the scene when lambda is created:

auto fun = [](int num) { return num; };

C++ Insights can be used to see what is going on behind the scene:

class __lambda_1_12
{
  public: 
  inline /*constexpr */ int operator()(int num) const
  {
    return num;
  }

  using retType_1_12 = int (*)(int);
  inline /*constexpr */ operator retType_1_12 () const noexcept
  {
    return __invoke;
  };

  private: 
  static inline int __invoke(int num)
  {
    return num;
  }
};

__lambda_1_12 fun = __lambda_1_12{};

This tells us that lambdas are standard types, and that actually we can inherit from them.

template <typename L1, typename L2>
struct Combo : L1, L2 {
  Combo(L1 l1, L2 l2) : L1(std::move(l1)), L2(std::move(l2)) {}
  using L1::operator();
  using L2::operator();
};

Let’s create two lambdas:

auto l1 = []{return 42;};
auto l2 = [](const int n){ return n*n;}

When instantiating Combo we will take advantage of Class Template Argument Deduction from C++17

Combo combo(l1, l2);
std::cout << combo() << std::endl;
std::cout << combo(3) << std::endl;

A more generic version can be created and used as

template <typename... Ls>
struct Combo : Ls... {
  Combo(Ls... ls) : Ls(std::move(ls))... {}
  using Ls::operator()...;
};

auto l1 = [] { return 42; };
auto l2 = [](const int n) { return n * n; };
auto l3 = [](const int n, int m) { return n * m; };

Combo combo(l1, l2, l3, [](const std::string& s){return s + " " + s;});
std::cout << combo() << std::endl;
std::cout << combo(3) << std::endl;
std::cout << combo(3, 43) << std::endl;
std::cout << combo("cpp") << std::endl;

First version: https://godbolt.org/z/TWs3e8raY

Second version: https://godbolt.org/z/7qvchhvzx

Google Test - Missing Feature

Sun, 03 Oct 2021 20:37:24 +0200

Test-Driven Development. Red-Green-Refactor. Let’s start. You are working on an awesome project in C++ and for TDD you are using the Google testing framework.

A typical scenario when using TDD:

The new behavior is introduced and described by a falling test. We have RED.
Logic is implemented, the test is PASSING. We have GREEN.
The last step, REFACTOR test and implementation if possible.

After that, we run all the tests and we notice that we have 17 falling tests.
What about re-running only failing tests?
What about re-running the first and last failing tests?

This is something that kept popping up to me. Google Test does not support anything like this, so I decided to write my solution. Github link

What is it?

Wrapper around Google tests

What it solves?

It allows you to re-run failed tests as well as to filter those failed tests by using numeric tags

How to use it?

Set which executable file will be executed:

./gtester.py --exe exe_name

Run all tests:

./gtester.py

Run just failing tests:

./gtester.py --run_failing

Run several failing tests using tags:

./gtester.py --run_failing 1 3

Example

Hello World

Thu, 01 Jul 2021 23:59:01 +0200

Hello World

#include <iostream>

int main(){
  std::cout <<"Hello World!" << std::endl;
  return 0;
}