DEV Community: Artem

Trivy Vulnerability Scans Adnvanced Filtering

Artem — Thu, 12 Dec 2024 18:50:10 +0000

Hi there!

It's been a while since I posted anything, but it is all cause of the good reasons. The last 2 years were busy for me both at work and day-to-day.

Anyway, I just wanted to share the cool feature I discovered in Trivy that really sets it apart from all other OSS security scanners. I am talking about the advanced filtering, that is using Open Policy Agent and Rego scripts to make decisions on what should be ignored from the scan results. It is described in details in the Trivy's official documentation, and although it is an experimental feature, it has been around since older version of Trivy.

I want to wrap this short blog post, by sharing a Rego script allowing to filter the CVEs based on the grace period:

package trivy

import data.lib.trivy

default ignore = false

now_ns := time.now_ns()
days_7_ns = 7 * 24 * 60 * 60 * 1000000000
days_30_ns = 30 * 24 * 60 * 60 * 1000000000
days_90_ns = 90 * 24 * 60 * 60 * 1000000000
days_180_ns = 180 * 24 * 60 * 60 * 1000000000


published_date = d {
    d := input.PublishedDate
}

ignore {
    input.Severity == "CRITICAL"
    published_date_ns := time.parse_rfc3339_ns(published_date)
    time_diff_ns = now_ns - published_date_ns
    time_diff_ns < days_7_ns
}

ignore {
    input.Severity == "HIGH"
    published_date_ns := time.parse_rfc3339_ns(published_date)
    time_diff_ns = now_ns - published_date_ns
    time_diff_ns < days_30_ns
}

ignore {
    input.Severity == "MEDIUM"
    published_date_ns := time.parse_rfc3339_ns(published_date)
    time_diff_ns = now_ns - published_date_ns
    time_diff_ns < days_90_ns
}

ignore {
    input.Severity == "LOW"
    published_date_ns := time.parse_rfc3339_ns(published_date)
    time_diff_ns = now_ns - published_date_ns
    time_diff_ns < days_180_ns
}

The following script queries the results of the scans and checks for severity and evaluates against the set grace policy.

It is really exciting to have this capability in the OSS, since normally you would have to pay for premium subscription to get a scanner use advanced filtering in the policies.

Terraform Interpolation vs. Directives

Artem — Fri, 01 Jul 2022 00:14:54 +0000

Every Terraform user should be familiar with HashiCorp Language (HCL) aka Terraform interpolation syntax.

Let say I have a variable defined as filename, then interpolation allows me to do stuff like that:

resource "local_file" "index" {
  filename = "${var.filename}.txt"
  content  = "foo!"
}

The Terraform official website says:

[Interpolation] evaluates the expression given between the markers, converts the result to a string if necessary, and then inserts it into the final string.

In other words, it can do some elementary operation and dump result as a string.

This is neat, and can be helpful if we want to add suffix or prefix to an EC2 instance, but what if we are tasked with generating a config file dynamically with unknown number of nested blocks?

That's where directives come into play!

The Terraform docs have this short definition of the directive:

[It] allows for conditional results and iteration over collections.

It is kinda like for_each, but inside of string rather than resource.

An example should make things clear!

variable "filename" {
  default = "index.html"
}

variable "facts" {
  default = ["fun", "hard"]
}

variable "complex_facts" {
  default = [
    {
      element: "h2"
      content: "FUN!"
    },
    {
      element: "h1"
      content: "EASY!"
    }
  ]
}

resource "local_file" "index_html" {
  filename = "${path.module}/${var.filename}"
  content  = <<EOT
    <html>
    %{for fact in var.facts}
      <p>Terraform is ${fact}</p>
    %{endfor}

    %{for fact in var.complex_facts}
      <${fact.element}>Terraform is ${fact.content}</${fact.element}>
    %{endfor}
    </html>
  EOT
}

You can see how list facts is used to generate HTML

elements:

%{for fact in var.facts}
  <p>Terraform is ${fact}</p>
%{endfor}

The same syntax can be used on objects, like with in complex_facts case:

%{for fact in var.complex_facts}
  <${fact.element}>Terraform is ${fact.content}</${fact.element}>
%{endfor}

The resulting HTML file is pretty ugly:

    <html>

      <p>Terraform is fun</p>

      <p>Terraform is hard</p>



      <h2>Terraform is FUN!</h2>

      <h1>Terraform is EASY!</h1>

    </html>

This can be used to generate tiny and ugly HTMLs (although there are better tools for this purpose), but it can also be used for more complex configuration files!

It can also be used to replace ternary expression, which some people find hard to read.

This interpolation sequence:

resource "local_file" "txt" {
  filename = "${path.module}/file.txt"
  content  = <<EOT
    ${ var.content != "" ? var.content : "NO CONTENT PROVIDED"}
  EOT
}

can be replace by the following directives sequence:

resource "local_file" "txt" {
  filename = "${path.module}/file.txt"
  content  = <<EOT
    %{ if var.content != "" }${var.content}%{ else }NO CONTENT PROVIDED!%{ endif }
  EOT
}

That's all for today! Have fun hacking Terraform!

Why NOT TO Ship NodeJS Containers With NPM?

Artem — Thu, 06 Jan 2022 15:37:42 +0000

There is a number of great guides on "containerizing" NodeJS applications, including this one from Snyk. However, I am yet to see a resource recommending to omit NPM from the final container image.

Let's say I have the following "dummy" application:

index.js

const express = require('express')
const app = express()

app.get('*', function (req, res) {
  res.send('bla bla bla')
})

app.listen(3000)

package.json

{
  "name": "test",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "express": "^4.17.2"
  }
}

One common way to structure a Dockerfile for this app would be using two stage build. First stage, installing dependencies; and second creating the final image. Both stages are using Alpine image with pre-installed NodeJS and NPM. With our simple app, we can even omit the first step, but let's pretend we need it.

bad.Dockerfile

# Build stage
FROM node:16-alpine3.15 as build

# Install dependencies
WORKDIR /
COPY package-lock.json .
COPY package.json .
RUN npm ci --production

# Final stage
FROM node:16-alpine3.15 as final

# Setup application
RUN mkdir -p /app/simple-server
WORKDIR /app/simple-server
COPY . .
COPY --from=build node_modules node_modules

# Run application
ENTRYPOINT ["node", "index.js"]

As you can see NPM will be shipped with the final container image. So what's the problem here?

The issue is the final image will have the dependency that is not used, but you would have to maintain it.

Not a big deal? It actually is, and can potentially become a blocker preventing to ship your application to production (or other environment depending on security controls in place). A good example is CVE-2021-3807. There is a GitHub Issue open, where engineers complaining how vulnerability presented in NPM blocks them in one or another way.

The solution here is simple - omit NPM from your final image. In Docker multi-stage build, it would look very similar to the bad example. The main difference is the final image is bare Alpine, and only NodeJS is installed as build step.

good.Dockerfile

# Build stage
FROM node:16-alpine3.15 as build

# Install dependencies
WORKDIR /
COPY package-lock.json .
COPY package.json .
RUN npm ci --production

# Final stage
FROM alpine:3.15 as final

# Upgrade APK
RUN apk --no-cache add --upgrade nodejs~16

# Setup application
RUN mkdir -p /app/simple-server
WORKDIR /app/simple-server
COPY . .
COPY --from=build node_modules node_modules

# Run application
ENTRYPOINT ["node", "index.js"]

Another benefit of excluding NPM from the final image is reduced size. The "dummy" server without NPM is 53.9MB, while with the package manager 112MB!

Not much else to say here. If you still have NPM in your final container image, ask yourself why!

Thank you for reading this article, and I would like to see the feedback on this one! Please let me know in comments what are YOUR legitimate reasons for having NPM in the final container image.

Installation Using Operator | Prisma Cloud Compute

Artem — Tue, 14 Dec 2021 05:12:08 +0000

Prisma Cloud Compute is a self-hosted solution to secure containerized workflows. "Self-hosted" means organizations have to deploy and manage updates themselves.

For Kubernetes and OpenShift deployments, these tasks can be outsourced to Prisma Cloud Compute operator.

In today's video I am going to demonstrate how to use operator to deploy Prisma Cloud Compute console:

As always, throw any questions in comments. Cheers!

Import Existing Resources in Terraform | Prisma Cloud Compute

Artem — Sat, 11 Dec 2021 00:35:54 +0000

In this video I am showing how to use import and output workflow in order to take obtain source code for Terraform resources that are tricky to create "by hand".

As in the previous vide, the focus is on Prisma Cloud Compute provider, however this workflow can be used with other Terraform providers too.

This method works great, and have been proven to work especially well in situations were resources are easy to initially create in UI, but hard using HCL.

As always, let me know if you have any questions, or feedback in comments!

Reference

Terraform import - https://www.terraform.io/docs/cli/import/index.html
Terraform show - https://www.terraform.io/docs/cli/commands/show.html

Starting with Terraform Provider | Prisma Cloud Compute

Artem — Wed, 08 Dec 2021 22:02:49 +0000

Automating your security is crucial, especially for companies operating in the cloud! I have recently made a video tutorial on how to use Terraform provider for Prisma Cloud Compute.

It would help anyone using Palo Alto Network's security platform to start building automation around their security practices.

I will include code snippets from this tutorial below:

creds.json

{
  "username": "test",
  "password": "test",
  "console_url": "https://192.168.64.2:32677"
}

main.tf

terraform {
  required_providers {
    prismacloudcompute = {
      source = "PaloAltoNetworks/prismacloudcompute"
      version = "0.1.0"
    }
  }
}

provider "prismacloudcompute" {
  config_file = "creds.json"
}

resource "prismacloudcompute_collection" "node_alpine" {
  name              = "node-alpine-collection"
  description       = "Collection for Node images based on Alpine"
  color             = "#68A063"
  application_ids   = ["*"]
  code_repositories = ["*"]
  images            = ["node:17-alpine3.12", "*/node:17-alpine3.12"]
  labels            = ["*"]
  namespaces        = ["*"]
}
resource "prismacloudcompute_ci_image_vulnerability_policy" "ruleset" {
  depends_on = [
    prismacloudcompute_collection.node_alpine,
  ]

  rule {
    collections = [
      prismacloudcompute_collection.node_alpine.name,
    ]
    disabled   = false
    effect     = "alert, block"
    grace_days = 30
    name       = "${prismacloudcompute_collection.node_alpine.name}-ci-policy"
    notes      = "CI policy for ${prismacloudcompute_collection.node_alpine.name}"
    only_fixed = true
    verbose    = false

    alert_threshold {
      disabled = false
      value    = 1
    }

    block_threshold {
      enabled = true
      value   = 2
    }

    cve_rule {
      description = "Ignore ansi-regex"
      effect      = "ignore"
      id          = "CVE-2021-3807"

      expiration {
        date    = "2022-01-06T06:00:00Z"
        enabled = true
      }
    }
    cve_rule {
      description = "Ignore busybox"
      effect      = "ignore"
      id          = "CVE-2021-28831"

      expiration {
        date    = "2022-01-06T06:00:00Z"
        enabled = true
      }
    }
  }

  rule {
    collections = [
      "All",
    ]
    disabled   = false
    effect     = "alert, block"
    grace_days = 30
    name       = "default"
    notes      = "Default policy for CI scans"
    only_fixed = true
    verbose    = false

    alert_threshold {
      disabled = false
      value    = 1
    }

    block_threshold {
      enabled = true
      value   = 2
    }
  }
}

Cross-platform C/C++ Development with Conan

Artem — Sat, 13 Mar 2021 17:52:44 +0000

Conan is an open source and multi-platform package manager to create and share native binaries (mostly C and C++). It is written in Python and supports Windows, Linux, and Mac.

Besides managing dependencies, conan simplifies development process by integrating with various build tools. I touched on its most basic usage in tandem with cmake in my previous blog-post titled "Building HTTP Service in C++". However, that post aimed primary towards Mac and Linux users, and developers using Windows platform were left aside.

Today, let's see how we can use conan to manage cross-platform development.

Let's start by creating a conanfile. Instead of using plaintext file, like in the HTTP service post, we use a Python script. Conan has ability to work with both types.

conanfile.py

from conans import ConanFile, CMake, tools

class MyApp(ConanFile):
    name = "my_app"
    version = "1.0"
    license = "unlicensed"
    author = "Artem Akatev"
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": False, "fPIC": True}
    generators = "cmake"
    requires = "boost/1.75.0"

    def config_options(self):
        if self.settings.os == "Windows":
            del self.options.fPIC

    def build(self):
        cmake = CMake(self)
        cmake.configure(source_folder="src")
        cmake.build()

Two important directives in the file are generators and requires. The first one contains a list of build tools, and the second a list of all dependencies. Both lists are comma delimited.

The build section (Python function) contains cmake configuration. The cmake is set to expect CMakeLists.txt file in src folder, so let's create the file.

src/CMakeLists.txt

cmake_minimum_required(VERSION 2.8.12)
project(my_app)

add_definitions("-std=c++11")

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

add_executable(my_app main.cc)
target_link_libraries(my_app ${CONAN_LIBS})

Now we need some application to compile. I will "steal" an example of HTTP client from Boost Beast.


#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/version.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <cstdlib>
#include <iostream>
#include <string>

namespace beast = boost::beast;     // from <boost/beast.hpp>
namespace http = beast::http;       // from <boost/beast/http.hpp>
namespace net = boost::asio;        // from <boost/asio.hpp>
using tcp = net::ip::tcp;           // from <boost/asio/ip/tcp.hpp>

// Performs an HTTP GET and prints the response
int main(int argc, char** argv)
{
    try
    {
        // Check command line arguments.
        if(argc != 4 && argc != 5)
        {
            std::cerr <<
                "Usage: http-client-sync <host> <port> <target> [<HTTP version: 1.0 or 1.1(default)>]\n" <<
                "Example:\n" <<
                "    http-client-sync www.example.com 80 /\n" <<
                "    http-client-sync www.example.com 80 / 1.0\n";
            return EXIT_FAILURE;
        }
        auto const host = argv[1];
        auto const port = argv[2];
        auto const target = argv[3];
        int version = argc == 5 && !std::strcmp("1.0", argv[4]) ? 10 : 11;

        // The io_context is required for all I/O
        net::io_context ioc;

        // These objects perform our I/O
        tcp::resolver resolver(ioc);
        beast::tcp_stream stream(ioc);

        // Look up the domain name
        auto const results = resolver.resolve(host, port);

        // Make the connection on the IP address we get from a lookup
        stream.connect(results);

        // Set up an HTTP GET request message
        http::request<http::string_body> req{http::verb::get, target, version};
        req.set(http::field::host, host);
        req.set(http::field::user_agent, BOOST_BEAST_VERSION_STRING);

        // Send the HTTP request to the remote host
        http::write(stream, req);

        // This buffer is used for reading and must be persisted
        beast::flat_buffer buffer;

        // Declare a container to hold the response
        http::response<http::dynamic_body> res;

        // Receive the HTTP response
        http::read(stream, buffer, res);

        // Write the message to standard out
        std::cout << res << std::endl;

        // Gracefully close the socket
        beast::error_code ec;
        stream.socket().shutdown(tcp::socket::shutdown_both, ec);

        // not_connected happens sometimes
        // so don't bother reporting it.
        //
        if(ec && ec != beast::errc::not_connected)
            throw beast::system_error{ec};

        // If we get here then the connection is closed gracefully
    }
    catch(std::exception const& e)
    {
        std::cerr << "Error: " << e.what() << std::endl;
        return EXIT_FAILURE;
    }
    return EXIT_SUCCESS;
}

Next, open a terminal window in the project's root directory (same as the directory containing conanfile.py), and follow these steps to build and run the application:

1. Install build dependencies

conan install . --install-folder build

2. Build binary

conan build . --build-folder build

Run your application

Mac and Linux

./build/bin/my_app google.com 443

Windows

build\bin\my_app.exe google.com 443

Implement HTTPS using Sockets and OpenSSL

Artem — Sun, 07 Feb 2021 20:55:00 +0000

The Internet Protocol Stack

Most developers are familiar with visualizing the Internet stack as a layered cake. Each layer aims to abstract away the complexity of the lower layer. There are two main conceptual models: OSI and TCP/IP. Despite using different concepts for layering, the models closely resemble each other.

Both models built around the encapsulation method. The main idea is logically separate functions in the network are abstracted from their underlying structures by inclusion or information hiding within higher level objects.

Each layer builds a protocol data unit (PDU) by adding a header containing control information to the data, referred as service data unit (SDU), from the layer above.

The result of encapsulation is that each lower layer provides a service to the layer or layers above it. At the same time, each layer communicates with its corresponding layer on the receiving end.

One of the most widely used protocols residing on the top level is HTTP. It is a protocol which allows the fetching of various kinds of resources. It is the foundation of any data exchange on the Internet.

Hypertext Transfer Protocol (HTTP)

HTTP is designed to be simple, and human readable (HTTP/2 is an exception). It uses client-server architecture. When a client wants to communicate with a server it first creates a TCP connection, and then send a message in HTTP format.

It can be something as simple as:

GET / HTTP/1.1
Connection: close

The usual response can look like that:

HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html

Hello world!

Sockets

For the actual implementation we need some programming primitives to abstract transport layer, TCP protocol. In modern operating systems such an abstractions is provided by the sockets.

A socket is one endpoint of a two-way communication link between two programs running on the network. A socket is bound to a port number so that the TCP layer can identify the application that data is destined to be sent to.

The following examples will use Ruby, but all modern programming languages have some library providing access to the underlying operating system sockets.

Look at the the simple HTTP client making a request to http://google.com:

require "socket"

socket = TCPSocket.new "www.google.com", 80

socket.print "GET / HTTP/1.0\r\n"
socket.print "\r\n\r\n"

while line = socket.gets
  puts line
end

socket.close

Transport Layer Security (TLS)

The obvious issue with HTTP, it's not only human readable, but the data is transferred as a plaintext. It makes it impossible to use the protocol to exchange any sensitive data. To solve this issue, the modern Internet stack introduces TLS.

The primary goal of the TLS protocol is to provide privacy and data integrity between two communicating applications. The protocol is composed of two layers: the TLS Record Protocol and the TLS Handshake Protocol.

One advantage of TLS is that it is application protocol independent. Higher-level protocols can layer on top of the TLS protocol transparently. In the Internet stack is resides between TCP and HTTP.

OpenSSL

The actual implementation of the TLS is really complex. It is not a human readable protocol, and the data is transferred as bytes. The extensive use of cryptography makes it even harder to grok.

OpenSSL is a robust, commercial-grade, and full-featured toolkit for the TLS protocols. It is also a general use cryptographic library. OpenSSL is widely used by web servers, including Nginx and HAProxy, and operating systems, including many Linux distributions. OpenSSL derivatives, like BoringSSL and LibreSSL power even greater number of systems and applications, like OsX, Android, Chromium, and many more.

OpenSSL module for Ruby provides a wrapper for C API, and can be used to implement a simple HTTPS client making a request to https://google.com:

require "socket"
require "openssl"

socket = TCPSocket.new "www.google.com", 443
context = OpenSSL::SSL::SSLContext.new
secure_socket = OpenSSL::SSL::SSLSocket.new socket, context

secure_socket.connect
secure_socket.print "GET / HTTP/1.0\r\n"
secure_socket.print "\r\n\r\n"

while line = secure_socket.gets
  puts line
end

secure_socket.close

C++ Pointer to Function

Artem — Wed, 27 Jan 2021 01:15:57 +0000

Hey guys! I haven't written anything in a while, and this post will be very brief too. That said, I think you can greatly benefit from it, especially if you regularly have to deal with C/C++ (and even more likely if you learned C++ in school together with OOP).

Anyhow, I will show you a paraphrased snipped from Google's crypto library BoringSSL. It is a great example of using a pointer to function.

Here it is!

#include <iostream>
#include <string>

typedef void (*tool_func_t)();

struct Tool {
  const char *name;
  tool_func_t func;
};

void info_func() { std::cout << "You selected info" << std::endl; }

void help_func() { std::cout << "You selected help" << std::endl; }

const Tool kTools[] = {
    {"info", info_func},
    {"help", help_func},
};

tool_func_t find_tool(const std::string &name) {
  for (size_t i = 0;; i++) {
    const Tool &tool = kTools[i];
    if (tool.func == nullptr || name == tool.name) {
      return tool.func;
    }
  }
}

int main(int argc, char **argv) {
  if (argc == 1) {
    std::cout << "You didn't povide any arguments" << std::endl;
    return 1;
  }
  tool_func_t tool = nullptr;
  tool = find_tool(argv[1]);
  if (tool == nullptr) {
    std::cout << "You can either get info, or ask for help!" << std::endl;
    return 2;
  }
  tool();
  return 0;
}

The key step is typedef declaration which defines the function's signature (in this example takes no arguments and returns void). After that, you have great flexibility in working with functions having this signature.

Of course, you can read more about it on www.learncpp.com.

Building HTTP Service in C++ (Using Modern Tools)

Artem — Sat, 05 Dec 2020 21:50:12 +0000

Today I will show you how to build an HTTP service in C++ using modern tools. The point of this article is to show how easy (or hard) it is to setup an environment for cross-platform C++ development, and package the binary in one of the modern distribution formats, container.

First, a disclaimer:

I am not a professional C++ developers, and any opinions shared in this tutorial should not be treated as absolute truth!

That said, since I am not profession C++ developer, this tutorial is suitable for "everyone-and-their-mother". The shell commands shown should work in most recent versions of Linux and OSX. Windows users can find themselves making some extra research, but overall the tools used are cross-platform.

Requirements

First things first, install the following tools:

CMake - system designed to build cross-platform software.
Conan - C/C++ package manager.
Make - tool controlling the generation of executables.
Docker - set of tools for OS-level virtualization.

What are we building?

The HTTP service and is mostly based on this particular example from Boost Beast library. I modified it, replacing plaintext responses with json, and adding Boost Log library.

Setting up development tools

Start by creating a src directory in the root of your project, and putting this C++ file inside:

mkdir src
curl -o src/main.cpp https://gist.githubusercontent.com/hi-artem/07e896d5d7b43d1cf63cf58e8665524b/raw/73d3148603c16e0505af52071b7658e6eda967b4/main.cpp

Although Boost is one of the well supported libraries, I always struggle manually installing it on my development machine. This is where Conan comes into play. For people familiar with Node, think of Conan as NPM or Yarn for C/C++.

Create conanfile.txt with the following content:

[requires]
boost/1.74.0

[generators]
cmake

Next, configure CMake to let it know about location of the C++ source file, and enable Conan integration. It can be done by creating CMakeLists.txt looking like so:

cmake_minimum_required(VERSION 3.10)

project(http-service VERSION 0.0.1)
set(CMAKE_CXX_STANDARD 14)

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

add_executable(http-service src/main.cpp)
target_link_libraries(http-service ${CONAN_LIBS})

Building executable

This is all you need to build and run the project locally! The following shell script contains all the required build commands:

#!/bin/bash

# remove all build folder if exists
rm -rf build

# create new build folder
mkdir build && cd build

# install dependencies
conan install ..

# generate build files
cmake ..

# generate executable
make

Once build succeded the service can be started by running:

./build/bin/http-service 0.0.0.0 8080

You should see something like:

[2020-12-05 13:17:27.928789] [0x000000011670fdc0] [info]    Service is listening on 0.0.0.0:8080

Use curl or web browser, to test that service is running. For example:

curl -i 0.0.0.0:8000/api/status

## Should return something similar to:
HTTP/1.1 200 OK
Connection: close
Content-Type: application/json
Content-Length: 60

{"status":"OK","request_count":"1","timestamp":"1607196274"}

Containerizing service

The previous steps configured development environment, however modern stacks require application to be packaged as container so it can be run by one of the orchestrators, like Kubernetes or Nomad.

Since we are using Docker to containerize the service, let's start by creating Dockerfile:

# Create image with all required build tools
FROM ubuntu:18.04 as build-tools

RUN apt-get update && apt-get install -y \
  build-essential \
  cmake \
  python3-pip \
  && rm -rf /var/lib/apt/lists/*

RUN pip3 install conan


# Build binary
FROM build-tools as builder

WORKDIR /usr/src/service

COPY . .

WORKDIR /usr/src/service/build

RUN conan install ..
RUN cmake ..
RUN make


# Copy binary to fresh ubuntu image
FROM ubuntu:18.04 as runner

RUN groupadd -r ubuntu && useradd --no-log-init -r -g ubuntu ubuntu
USER ubuntu:ubuntu

COPY --from=builder /usr/src/service/build/bin/http-service /usr/bin/http-service
EXPOSE 8080

CMD [ "http-service", "0.0.0.0", "8080" ]

The image is created using multi-stage build process: first, creating the image with all required tools to build the service; second, build the service; finally, copy the executable in fresh ubuntu image. The process is run as non-root user to avoid privileges escalation.

The last thing before the image build, create .dockerignore file, which serves similar purpose as .gitignore:

build
.vscode

Finally, let's build the image. For folks no familiar with Docker, this commands will build and run the HTTP service:

docker build -t http-service .
docker run --rm --init -p 8080:8080 http-service

As previously, use curl or web browser to test that service is running:

curl -i localhost:8000/api/status

## Should return something similar to:
HTTP/1.1 200 OK
Connection: close
Content-Type: application/json
Content-Length: 60

{"status":"OK","request_count":"1","timestamp":"1607196274"}

At this point, the image can be tagged and pushed in a Docker registry, from where it can be pulled by a container orchestrator.

The purposes of this article was to reduce frustration associated with setting environment for C++ development. I hope you were able to follow. If not, the source code is available at github.com/hi-artem/http-service. Let me know in comments if you have any questions.

Cyptography for Beginners

Artem — Sat, 24 Oct 2020 19:40:35 +0000

Glossary

Cryptography - the practice and study of techniques for secure communication in the presence of third parties. Also known as crypto and cryptology.

Cipher - an algorithm for performing encryption or decryption.

Plaintext - unencrypted message or information. Also known as cleartext.

Ciphertext - encrypted message or information.

Encryption - a process of encoding messages or information in a form that only authorized parties can read it.

Decryption - a process of converting messages or information from ciphertext to plaintext.

Random Number Generator (RNG) - a device designed to generate a sequence of numbers that lacks any pattern.

Key - a parameter that determines the functional output of the cryptographic cipher.

Hash Function - a one-way cryptographic function considered practically impossible to invert.

Digest - the output of the hash function.

Symmetric Algorithm - an algorithm the uses the same cryptographic key for both encryption and decryption.

Asymmetric Algorithm - an algorithm the uses different cryptographic key for both encryption and decryption. Also known as public-key algorithms.

Symmetric Ciphers

There are two types of symmetric encryption algorithms:

Block Cipher - operates on fixed-length groups of bits, called blocks.

Stream Cipher - operates on stream of bits.

Common Block Ciphers

Data Encryption Standard (DES)

In modern cryptography, DES was the first standardized cipher for securing electronic communications. Due to the processing power of modern computers, DES is not used anymore as it is considered weak. It can be used in variations as 2-key or 3-key 3DES.

Advanced Encryption Standard (AES)

This is the standard set by the US National Institute of Standards and Technology in 2001 for the encryption of electronic data. It supersedes DES which had been in use since 1977. AES has various forms, depending on the key length. They are abbreviated as AES-128, AES-192 and AES-256 where the last number is the key length in bits.

Common Stream Ciphers

Rivest Cipher 4 (RC4)

RC4 stream chipher has been used in various protocols including Wired Equivalent Privacy (WEP) and Wi-Fi Protected Access (WPA) as well as in Transport Layer Security (TLS). However, due to some vulnerabilities discovered in 2015, it usage started to decline. RFC 7465 prohibits the use of RC4 in all versions of TLS.

Asymmetric Ciphers

Stream and block terms are not usually used with asymmetric ciphers. That said, public-key encryption also encrypts blocks of data. For example, RSA where block sizes are based on the key size.

Common Asymmetric Ciphers

Rivest–Shamir–Adleman (RSA)

One of the oldest and most widely used asymmetric algorithms. It is based on the fact that if you multiply two giant prime numbers it is almost impossible to derive the original primes from the result. In fact, there is a published research claiming that it would take around 1500 years of computing time to crack 768 bit RSA key!

Elliptic Curve Cryptography (ECC)

Like RSA, ECC works on the principle of irreversibility. In ECC, a number symbolizing a point on the curve is multiplied by another number and gives another point on the curve. Now, to crack this puzzle, you must figure out the new point on the curve. The mathematics of ECC is built in such a way that it’s virtually impossible to find out the new point, even if you know the original point.

Symmetric vs. Asymmetric Ciphers

In short, this comparison is more about the cost of an elementary operation (encryption and decryption) than anything else. It would be prohibitively expensive to do asymmetric encryption for large amounts of data. Symmetric keys give you cheap computation but the problem of a shared secret. Public keys give you expensive computation but easily shared information needed to communicate securely.

The effect usage of these algorithms come from creating another layer of abstraction. In other words, a researcher will have to build a system using these ciphers as primitives.

For example, the following hybrid approach makes sense:

Key encryption - use public key cryptography to encrypt a symmetric key generated on the fly. It is expensive operation, but you only do it once.

Data encryption - use the symmetric key to encrypt and decrypt the actual data. It is a much cheaper operation.

Hash Algorithms

These algorithms are commonly used to verify information integrity or store some types of sensitive data, like passwords. Why passwords? Because the same hash function will always yield the same hash for the same input. Therefore any attempted password can be hashed and the result compared against the saved hash to verify an authentication attempt.

Another memorable concept involving cryptographic hash functions is hash-based message authentication code (HMAC). It is a type of message authentication code (MAC) involving a cryptographic hash function and a secret cryptographic key. HMACs are used to simultaneously verify both the data integrity and the authenticity of a message.

Common Hash Algorithms

MD5 Message-Digest Algorithm (MD5)

MD5 was released in 1992 and was also built as a successor to MD4, which was successor to MD2. MD5 isn't as fast as MD4 but it is considered to be more secure than the previous MDx implementations. Although it is true, MD5 usage should be avoided in any capacity, as previous research has demonstrated, it should be considered cryptographically broken and unsuitable for further use. It can still be used as non-cryptographic hashing algorithm.

Secure Hash Algorithms (SHA)

It is a a family of cryptographic hash functions published by the National Institute of Standards and Technology (NIST) as a US Federal Information Processing Standard (FIPS), and includes the following algorithms:

SHA-1 - developed in 1993, and widely used in security applications and protocols, including SSL/TLS, PGP, SSH, IPsec, and S/MIME. SHA-1 produces a 160-bit digest and has a block size of 512 bits. Although SHA-1 is still widely used, cryptanalysts in 2005 were able to find vulnerabilities on the algorithm that detrimentally compromised its security. These vulnerabilities came in the form of an algorithm that speedily finds collisions with different inputs, meaning that two distinct inputs map to the same digest.

SHA-2 - published in 2001, it consists of two hash functions known as SHA-256 and SHA-512, using 32- and 64-bit words, respectively. There are additional truncated versions of these hash functions, known as SHA-224, SHA-384, SHA-512/224, and SHA-512/256. SHA-2 produces 224 or 256-sized digests and has block sizes that contain 1024 bits, or 512 bits.

SHA-3 - the latest member of the SHA family of standards, released in 2015. SHA-3 is a subset of the broader cryptographic primitive family Keccak and internally varies greatly from SHA-1 and SHA-2. Although SHA-3 is a better algorithm, NIST does not currently plan to withdraw SHA-2 or remove it from the revised Secure Hash Standard.

[Learn Hugo] Develop, Stage and Prod Configurations

Artem — Wed, 07 Oct 2020 21:56:43 +0000

Welcome back!

In the previous tutorials we talked about the differences between dynamic and static websites, introduced Hugo, and its typical project structure.

Today we will talk about something really important - managing configurations for multiple environments. Yes, we are jumping right into this topic without talking about the layouts and templates! Why? Well, at some point you want your website to be available on the World Wide Web. Which means you will have to put it on a hosting. Depending on the hosting provider, there could be numerous things that should be configured differently compare to the computer you develop on.

Instead of thinking about the deployment last, when the website already have grown big, why don't we establish some great practices straight away?

Deployment Environments

For a short time, I want to pause talking about Hugo, and cover the concept of having separate environments.

Typical software development workflow consists of more than one environment to make development cycles of a software application smoother and hassle free. The model usually consists of development, staging and production environments. Multiple environments suppose to help developers of the application at various phases, such as development, testing and releases.

For different projects, these terms can mean different things. For a hobby website the development environment is on the developer computer, while for the bigger projects it is distributed between multiple servers. It is common for staging and production to be identical, with the main difference being a presence of the real user data.

Environment Configurations in Hugo

Getting back to Hugo, and let's jump right into coding! I will continue working on the hello-world project from the previous tutorials. You can create a new project with hugo create site command, since we won't really reuse any of the existing code.

First thing, let's add (or create if started a new project) the following to layouts/index.html:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta 
      name="viewport" 
      content="width=device-width, initial-scale=1">
    <title>{{ .Title }}</title>
    <link 
      rel="stylesheet" 
      href="https://stackpath.bootstrapcdn.com/bootstrap/5.0.0-alpha2/css/bootstrap.min.css" 
      integrity="sha384-DhY6onE6f3zzKbjUPRc2hOzGAdEf4/Dz+WJwBvEYL/lkkIsI3ihufq9hk9K4lVoK" 
      crossorigin="anonymous">
  </head>
  <body class="d-flex text-center text-white bg-dark" style="font-size: 1.5em;">
    <div class="container w-100 pt-5 mt-5 mx-auto flex-column">
      <h1>Welcome to {{ .Title }}!</h1>
      The website is served at {{ .Site.BaseURL }}
    </div>
  </body>
</html>

If you feel lost looking at all the {{ ... }} it is totally fine. It is called templating. We will cover it in great details in the coming posts. For now just think of it as a way to variablize a piece of HTML.

Start Hugo development server (hugo server), and navigate to http://localhost:1313. You should see the following:

If you open the config.toml it becomes clear where the My New Hugo Site part is coming from. There is also baseURL property which let you set the domain name for the website. This one can be confusing. When you use the development server the baseURL property gets partially overwritten. Read more about it here.

By using the configuration files, you can easily manage Hugo project in various stages of the development.

Development, Staging and Production

Let's look at some real world example. Suppose a website have three environments: development, on a local machine; staging, on Github Pages; and production, on Netlify. How would the configuration for this particular website look?

Well, the only way to tell is to create such a website! This is not a tutorial on how to deploy a website to Netlify or Github Pages, so excuse me rushing through some of the steps. The point I want to make here is how easy it is to manage configuration with Hugo out of the box.

First, let's look at the development config, config.toml:

baseURL = "/"
languageCode = "en-us"
title = "Development Environment"

Easy.

Move on to staging. I will create a GitHub repository, and called it learn-hugo. You are free to call it however you want. My GitHub account name is aakatev, so the address for my staging website is https://aakatev.github.io/learn-hugo/. The usual schema is https://<username>.github.io/<repo-name>/. GitHub Pages serves files from docs directory, so we should take it into account.

Considering all this, here is my config-stage.toml:

baseURL = "https://aakatev.github.io/learn-hugo/"
languageCode = "en-us"
title = "Staging Environment"
publishDir = "docs"

In order to deploy our staging version, the website has to be built locally, and pushed on GitHub.

Before this moment, we have always been using the default configuration file. Hugo knows about config.toml. However, now that the name is different, we have to provide --config flag in order to tell Hugo which config to use:

hugo --config config-stage.toml

After the website is pushed into the repo, it will automatically be deployed to GitHub Pages.

Finally, our production environment. Netlify comes with CI/CD for static websites by default. Connect your existing GitHub repository, and provide minimum configuration:

The default app name generated by Netlify is pretty random. You can always change it, but in my case I decided to leave it as is - dazzling-curran-c212c7. The website domain name can be derived as <netlify-app-name>.netlify.app, and here is the config-prod.toml:

baseURL = "https://dazzling-curran-c212c7.netlify.app/"
languageCode = "en-us"
title = "Production Environment"
publishDir = "public"

Once deployed, the production version should look like so:

Round Up

Again, this example was more for illustration purposes rather than actual tutorial. That said, in case you do want to try it out yourself, the complete code is on my GitHub.

One last thing before I let you go. Although we used configuration files for this project, Hugo allows developers to separate configs by directories.

For example, the config directory can have the following layout:

├── config
│   ├── _default
│   │   ├── config.toml
│   ├── production
│   │   ├── config.toml
│   │   └── extra.toml
│   └── staging
│       ├── config.toml
│       └── extra.toml

Note, the _default sub-directory. In Hugo, it is a special name that is pretty self explanatory. This directory will be used in case no --configDir flags is provided.

That's it for today! Hopefully, you learned something new. Next time we will talk about Hugo layouts and templates.

Stay tuned!