DEV Community: Mohammad Reza Karimi

[TechStory]: How to add distributed tracing using Jaeger and OpenTelemetry into a Golang application

Mohammad Reza Karimi — Wed, 13 Dec 2023 08:39:43 +0000

I recently had a tough time adding distributed tracing to our services using OpenTelemetry, Although that sounds not to be hard, lots of deprecated packages, changed solutions, no up-to-date documents, etc. led me to write this article for other engineers not to fall into the same hole.

I will start with a summary of the Tracing definition, then we will go through the right solution and mention holes along the way, it's my own experience (TechStory) and not a tutorial.

What is Tracing

Tracing is the process of capturing and recording information about the execution of software, traces come along with metrics and logs and they help us to have a better #observablity over our services.

Traces tell us what happened when, where, and took how long.

What is distributed tracing

In the microservices world, it's hard to see the request flow and integration between services, distributed tracing is a set of techniques that help us track user flow across multiple distributed services.

What tools can we use

Tracing and Distributed Tracing come with a bunch of dependencies (they may not be a good choice for small code bases with a few users)

The way of doing it and the number of dependencies also varies based on the tools you use and the architecture and design you follow, there might be:

Agent (a network daemon that listens for spans sent over UDP)
Collector (receives traces from the SDKs or Jaeger agents, runs them through a processing pipeline for validation and clean-up/enrichment, and stores them in a storage backend)
Trace DB (for example, Elasticsearch)
SDK and Libraries (OpenTracing, OpenTelemetry, agent-specific libraries, etc).
UI

But we don't go through all of them in this article, we use:

OpenTelemetry
Jaeger all-in-one docker image (or Maintained Jaeger on Cloud)

Basic Setup

To setup OpenTelemtry SDK (a bunch of codes and function calls), you need to inject (pass) a Provider (Jaeger) to it,
To initialize the provider, you need to write a code like this:

tp := trace.NewTracerProvider(
  // The sampler determines how many requests to trace

 trace.WithSampler(trace.TraceIDRatioBased(cfg.SamplerParam)),
  // Always be sure to batch in production.
  trace.WithBatcher(exporter),

  // Record information about this application in a Resource.
  trace.WithResource(resource.NewWithAttributes(
    semconv.SchemaURL,
    semconv.ServiceNameKey.String(cfg.Name),
  )),
)

Then if you run go mod tidy you may face this error:

go.opentelemetry.io/otel/semconv: module go.opentelemetry.io/otel@latest found (v1.13.0), but does not contain package go.opentelemetry.io/otel/semconv

In that case, I wrote a good explanation and solution for you here.

TL;DR: Old Deprecated Jaeger versions caused this error, and they are more than one,
There were Deprecated Jaegers for OpenTracing, but OpenTracing itself is also deprecated.

If you want to use OpenTelemetry Jaeger libraries instead:

https://pkg.go.dev/go.opentelemetry.io/otel/exporters/trace/jaeger
https://pkg.go.dev/go.opentelemetry.io/otel/exporters/jaeger

These are also deprecated!

So What should we do? 😰
In fact, All Jaeger libraries are deprecated and you should use OTLP instead nowadays (OTLP is not deprecated at the time of writing this article! but the world has gone wild! I don't guarantee tomorrow 😂)

So what is this OTLP thing?

In simple terms, OTLP (OpenTelemetry Protocol) is a protocol designed to standardize the telemetry data transfers between different clients, it's here to help the community not get the headache of being involved with a vast range of different tools and have a common standard instead.

For more information, I recommend you read their design goals.

In cases where you use a Jaeger all-in-one image (jaegertracing/all-in-one), you should be aware that this image includes an OpenTelemetry collector in versions >= 1.35 and you don't need to run a separate container (this official documentation confused me about it).

You should also expose some ports and enable some variables, I wrote a docker-compose file here that shows them all.

But if you have mismatched versions, you may get these errors:

traces export: failed to send to http://localhost:14268/api/traces: 400 Bad Request
traces export: failed to send to http://localhost:4318/api/traces: 404 Not Found
traces export: Post "http://localhost:4318/api/traces": read tcp [::1]:60409->[::1]:4318: read: connection reset by peer
traces export: Post "http://localhost:4318/v1/traces": dial tcp [::1]:4318: connect: connection refused

Simply say, you can run an all-in-one version >= 1.35, expose port 4318, and send your traces to http://localhost:4318/v1/traces which is the default OTLP config. (Jaeger agent was on e.g. :6831/api/traces, but you don't need that anymore)

Most of these issues were on Localhost, we had a managed Jaeger for production on our private cloud, and I got a new error like this:

traces export: context deadline exceeded: retry-able request failure

It happened because I was sending the traces to Jaeger Agent's address on production but I should've sent them to the collector, so if you faced the same error, double-check your address.

Propagation

Another hard topic was Propagation, what is it?
propagated traces are the ones that keep their identity the same across multiple services and help us see the request flow through a distributed system, this is possible by attaching some trace-specific headers to your request, and it's applicable in different scenarios whether it's a simple HTTP request or Kafka event, etc.

There are two important notes here:

Each service should support tracing and it should also be enabled (services that don't support tracing, will not be in the flow)
OpenTelemetry support Propagation Generally, it means you are responsible for "what headers" and it just attaches them to the request for you.

In case of implementation, you should first setup the propagator like this:

import (
  jaegerPropagator "go.opentelemetry.io/contrib/propagators/jaeger"
  "go.opentelemetry.io/otel"
)
...
otel.SetTextMapPropagator(jaegerPropagator.Jaeger{})

Then, you can Inject context to request headers when you are sending a request, and Extract them on the receiver side, and when the service is both receiver and sender, it may implement both Inject and Extract.

Inject:

// ctx is the context that included some trace data
otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(request.Header))

Extract (happening in middleware in this case):

ctx := otel.GetTextMapPropagator().Extract(
  request.Context(),
  propagation.HeaderCarrier(request.Header),
)
ctx, span := otel.Tracer("middleware").Start(ctx, "middleware")

Summary

We talked about different challenges I faced during the Tracing setup, but the challenges still apply to other scenarios (even other languages), #mismatched_version, #wrong_addresses, and #deprecated_pkgs, etc.
I hope you enjoyed this article.

[TechStory]: Migrating services and databases from an OpenShift (or K8s) cluster to another

Mohammad Reza Karimi — Sat, 03 Jun 2023 14:39:47 +0000

Welcome Friends to another TechStory!

My first experience with OpenShift was with a task which was kind of a heavy task, I didn't know anything about OpenShift or K8s! but I was in charge of migrating resources from a cluster to one another! (twice!)

But I learned a lot of things in the middle, so let's have a look at some of those.

Before going deep dive, you may wonder what is OpenShift and why do I put its name beside K8s everytime, It's actually becuase Openshift is built on top of K8s and it's kind of the same (+few more features), so don't worry about the difference.

So, What's the problem?

As I told you before, I had to move resources such as Applications (pods+services+routes), Databases (+their data) from an OKD3 cluster to an OKD4 cluster (different versions of OpenShift)

The reason of this migration? I'm not sure!
To be honest, this was already decided and it was on the board,
Also in some companies, Kubernetes has two sides, cloud side and developer side, we (developers) don't lunch Kubernetes, instead, we lunch our services on it.

Where did I start?

I asked two question on Stackoverflow because it was not clear for me what to do at the first place, and there were a lot of suggestions, which I just mention:

monokle
Velero
OpenShift Official Guide called Migration Toolkit for Containers (MTC)
Another Official Guide

Note: Unfortunately I can't provide a link to questions, although they were well received but a moderator deleted them 🤔

I also found and provided my answer, I followed it and I think it taught me a lot of things and there is nothing wrong with it, but it was not good enough solution for this case.

Not good enough solution!

For making resources in a K8s cluster you should be careful about their priority,

First things to make are secrets and config maps, because they don't depend on anything but everything can depend on them!

And last things are routes, guess why? because they depend on services but nothing depends on them!

So the priority is:

Secrets and ConfigMaps
StatefulSets
Deployments
Services
Routes

You may not need to move pods directly, because they are usually made and controlled by other resources such as
Deployments or StatefulSets.

Steps to follow:

oc login --token="your-token-for-first-server" --server="your-first-server"

Export your resources:

oc get -o yaml cm > configmaps.yaml
oc get -o yaml secrets > secrets.yaml
...

There are also some default ConfigMaps and Secrets which you don't need to copy, you can erase them from generated yaml files.

oc login --token="your-token-for-second-server" --server="your-second-server"

If you forget this step, you may get an error that says resource already exists, but be careful not to forget this step.

Load resources to the second cluster

oc create -f configmaps.yaml
oc create -f secrets.yaml
...

Better solutions

There was a better solution, we can use Templates and process them for different clusters, environments or namespaces.

Also, combining them with CI/CD pipelines, helps us to move our resources by just changing some variables!

Helm works even better! I personally prefer Helm over basic templating.

I improved our CI/CD pipelines and Helmified (used Helm 😁) our applications for easier, clean and seamless deployments.

What about DB?

DB is also another important & long discussion here, but not the database application itself, we can easily create that, but what should we do about the data? data is everything!

In my case, they were two scenarios:

Production
Stage (testing environment - production with fake data)

In Production, we had a managed Database controlled and owned by DBA team, we just had URL for Accessing it and that was enough.

So in cases that your accessing data which is in an external database (out of cluster), you don't need any extra work.

But in Stage, our data was stored in a PVC attached to a Pod controlled by a StatefulSet.

Note: We had MySQL, you may have something else, but the fellow is almost the same.

Steps to follow:

Step 1: Login and port-forward

oc login --token=<YOUR-TOKEN> --server=<YOUR-SERVER>

List all pods and select your db pod (if replicated, choose master (0)) and copy it:

oc get pods

Make a port-forwarding to your local computer:

oc port-forward <MYSQL-POD-NAME> <desired-local-port>:<mysql-on-server-port>

So your server database is now (like) your local database! and any change you want on server can be done on localhost.

This method is the same for all databases and also applications.

Step 2: tools

Install mysql command line tools (if you have them installed, skip this step).
Specifically, we need mysqldump and mysql commands.

To do so, on macos run:

brew install mysql-client

I needed to add them to path (it usually happens automatically), run command below and add /opt/homebrew/opt/mysql-client/bin to the end of the file:

sudo nano /etc/paths

Other editors such as vscode (type code in terminal) can be used instead of nano.

Read here and here for more information.

Step 3: take a dump

mysqldump --column-statistics=0 --no-create-info -P <local-port> -h 127.0.0.1 -u <YOUR-USERNAME> -p "YOUR-DATABASE-NAME" > file_name.sql

Then you'll be asked to enter the database password for .

In my case, I previously ran migration scripts so I didn't want my dump to include CREATE TABLE... queries, so I added --no-create-info argument, but you may want to erase it.

Also, I was facing with Unknown table 'COLUMN_STATISTICS' in information_schema (1109) error, so I added --column-statistics=0 argument.
For more information, read here and here.

About the host IP, do not use localhost, it probably connects by socket rather than by port.

Step 4: load the dump

Now, login to the second cluster, make the StatefulSet, and port-forward its database as explained before,
(you can run your migration scripts here on this local-like database and they will be done on the cluster as well)

Then, load the dump with below command:

mysql -P <local-port> -h 127.0.0.1 -u <YOUR-USERNAME> -p -f "YOUR-DATABASE-NAME" < file_name.sql

In my case, I had some unimportant errors, I've passed -f argument to skip them, but be careful when using this argument as it forces the database to follow your introduction defined on file_name.sql.

Conclusion

TechStories are my real job experiences, they might be confusing as they include a lot of information and links to other articles, that's my point! we (developers) don't work in the way people see in usual articles and tutorials, so I want to share my knowledge beside teaching a lot of different tips that get related in my task.

I hope you enjoyed! 🥳✌🏼

Make your cool commands in terminal

Mohammad Reza Karimi — Thu, 25 May 2023 07:23:35 +0000

Hi guys,
I wanna teach you how to make cool things in your terminal and increase your productivity like a pro.

You don't need any docker, k8s, Openshift or Golang experience for this article, we just used them as examples but you'll get the point in other cases too.

TL;DR

Open your shell config file (e.g. ~/.zshrc in Mac)
Write aliases commands like this alias mycmd="cmd long long long"

and functions like this:

mf() {
    mkdir $1 
    cd $1
    touch $2
}

And call it in terminal like this: mf mydir myfile and it will make a directory named mydir and file within it named myfile

The story behind this article

As a developer, you may work with CLIs a lot, CLIs are made to be general and composable, but in some specific cases, you may want to compose them as you wish once forever!

For example, I need some specific compositions of oc command for everyday use.

Also, I needed 2 different VPN connections using 2 different clients, they had long commands!

Sometimes they are long, sometimes they are hard to remember, etc.

Define aliases

There is an alias command available on *nix systems (Linux, Unix, MacOS, etc)

It's an alternative command which you can use to make your commands shorter.

In my case, I wanted to connect to a VPN with openfortivpn client like this:

sudo openfortivpn vpn.host.domain:port -u myusername -p mypassword --trusted-cert averyveryveryverylonghash

And another VPN using openconnect client:

sudo openconnect -u myusername -p mypassword vpn.host.domain:port

In addition to that, every company or personal project that I worked for, I used docker-compose up --build -d command, what does it do? doesn't matter!

For this article, we just know that these commands are the same most of the time and typing them out is cumbersome.

Long story short, shorten these commands like this:

alias mycmd="my very very long command"

But it's not permanent, it's limited to the current shell tab, to make it live forever in your system, open your shell config and write your aliases in it.

So according to this, in my system (MacOS), I followed these steps:

Step 1: run code ~/.zshrc (code opens up VScode editor, but you can use something else such as vim or nano)
Step 2: write this command inside the opened file and save it:

alias dub="docker-compose up --build -d`

Step 3: you are done! open up a new shell and use dub command and it works totally the same.

And about VPNs:

alias ofv="sudo openfortivpn vpn.host.domain:port -u myusername -p mypassword --trusted-cert averyveryveryverylonghash"

alias opc="sudo openconnect -u myusername -p mypassword vpn.host.domain:port"

Another alias I defined:

alias ocp="oc port-forward"

Define functions

Sometimes, you need to define more than one command or you need to have some placeholders in your commands to keep those parts still general purpose.

Let's say, sometimes you need some functions to do different operations and accept some inputs

Some examples

When I want to instantiate a new Golang project, I should run these commands:

mkdir <my_project_name> 
cd <my_project_name>
touch main.go
go mod init <my_project_name>

The rest might be different but these four commands are always the same.

I also have better solutions for this part of the article, but they are not in the context of this article and it's just an example.

So, to simply those four commands and make them one, write a function like below:

gomake() {
    mkdir $1 
    cd $1
    touch main.go
    go mod init $1
}

I called this function gomake and I can easily access it in my shell using gomake <my_project_name> and it will do that 4 commands.

$1 is a function argument, if you need more, you can access them by $2, $3, ...

Note: you can use echo "package main\n\nfunc main() {\n\t\n}" | cat > main.go instead of touch main.go in Golang projects, this command simply make main.go file and write

package main

func main() {

}

in it.

Another example, using oce <pod-name> instead of oc exec <pod-name> -it -- sh

oce() {
    oc exec $1 -it -- sh
}

Another example with 2 arguments,
Sometimes I want to see the structure and some cool information about the projects (e.g. number of lines of code)
For this case, I know two commands:

tree command, which is a tool to show directory structure of your projects, and it's pre-installed or you may install it using brew install tree in mac.

cocnt which is a tool to count the number of lines of code, link to their github repo: https://github.com/InnoFang/code-counter

And I always forget the second! 😂 (what was it's command?)

So for helping myself remembering the second command and first command flags, I wrote this functions and named it struct

struct() {
    tree $1 -dCL $2 -I vendor
    cocnt search $1
}

About tree flags, I is used for excluding a folder (vendor in my case), d to show only directories, L for level of going inside (which we get as an argument), C for colorizing the output.

So I call struct <project-path> 2 and it will print out the project with level two of directories without vendor folder.

Just if you are wondering (Used terms)

oc is command used for working with Openshift clusters.

Openshift is a platform made on top of Kubernetes for container orchestrations.

Docker is a platform designed to help developers build, share, and run modern applications.

To learn more about bash functions:
https://devqa.io/create-call-bash-functions/

Conclusion

These aliases and functions are still useful beside all modern terminal tools such as fish or warp, etc.

In addition to that, sometimes I write my own CLI tools or I use existing ones,

In case of making a project (Golang in this article), templates are so useful and good alternative.

Possibilities are endless...

Finally, I hope you enjoyed reading this article and learned something useful.

How docker will save your time, and why should you learn it immediately. (2/2)

Mohammad Reza Karimi — Mon, 14 Nov 2022 07:23:48 +0000

Hi guys!
In the last part, we discussed a little about what an image and a container is, in this part, we will make our own images and wrap our application in a container.

Make a Dockerfile in the root

First of all, make a file named Dockerfile, without any extension, in the root directory:

Tip: you can call this file whatever you want, but Dockerfile is the default recognizable name, other names, should be introduced in later steps.

Starting our image

As I said before:

Image is a collection of applications and commands running on a base image and this base image itself is a simplified Linux distro.

And a Dockerfile, is an introduction file for our image.
First of all, we should specify what base image we're going to run our application and commands on, and we do it like this:

FROM base_image

Alpine latest version as base image:

FROM alpine

Alpine specific version (3.15) with a pre-installed node.js with a specific version (18):

FROM node:18-alpine3.15

And you can find other base images and their tags (versions) in the Dockerhub.

Tip: Always try to use tagged versions, even in the case of latest.

Command the image!

You can make different users with different accessibilities in your image and run commands by their help, but for now, we're not going into that.

You're already familiar with FROM command that is used for pulling base image, there are more:

RUN:
The RUN instruction is used to run commands.
From docker:

Executes any commands on top of the current image as a new layer and commit the results.

Commands could be for example npm install or go mod tidy or pip install -r requirements.txt or more complex commands like apt-get update && apt-get install -y curl or any other command you run inside a linux!

CMD:
Provide defaults for an executing container, for example npm run start or running a complied executable.

Tip: there can only be one CMD instruction in a Dockerfile.

COPY:
Copy files or folders from root project directory (called source) to a image's filesystem path (called dest).

# COPY source dest
COPY ui .
COPY . ./
# Or many files, last argument is dest
COPY go.mod go.sum ./
# Or
COPY ["go.mod", "go.sum", "./"]

WORKDIR:
The WORKDIR instruction sets the working directory for the next instructions that follow it in the Dockerfile.

FROM alpine
WORKDIR /app
# Next commands will run in /app
COPY file1 file2 ./
COPY file3 file4 ./
# now, we have /app/file1 /app/file2 etc.
CMD ["./main"]
# command above will look for /app/main

ENV: defines environment variables.

ENV PG_PASS=12@3fl

There are many other commands like EXPOSE, ADD, ENTRYPOINT which you can read on Dockerfile reference.

Build images

Let's start with a node.js example, but don't worry if you're not a node.js developer.

After writing a complete Dockerfile like this:

FROM node:18-alpine3.15
COPY src .
COPY package*.json .
RUN npm install
CMD npm run start

Tip: package*.json means all files starting with package and ending with .json => package.json and package-lock.json

We can build the image by running this command:

docker build <path>

We usually run this command inside project root directory and use a single dot . for path: docker build .

This command alone, assign a random name for our container, but we can name it on our own using tags:

docker build -t <image_name> <path>

Caching and layered images

Last version will work, but is not performant, it's highly recommended to consider caching techniques.

Docker images are layered, simply said, every line in Dockerfile is a layer, and Dockerfile is read line by line.

What do we cache in the world of docker?
Mostly, project dependencies and packages and also we do not COPY them from source to destination.

As you might know, npm i command is a so heavy command that installs project dependencies and it read dependency needs from package.json file.

Look at example below:
project structure:

- index.js
- node_modules (dependencies)
- package.json

COPY index.js package.json ./
RUN npm install
CMD npm run start

It's good and working, but what happens if we change package.json (we add new dependency)

They will install again and that's OK.

But what happens if we change index.js without changing dependencies?
Dependencies will install again! and that's so NOT OK!

And that happens because we copy package.json and index.js at the same time, and we do installation after that.

We should copy all files except dependency introducer (pakcage.json) after installing dependencies:

COPY package.json ./
RUN npm install
COPY index.js ./
CMD npm run start

It's way much faster!

in this case, if we change only index.js, docker will read Dockerfile from line 3 and go after that, because it's defined on line 3.

Running your image (container)

Now we've learned how to make custom images, and we know how to run a container or expose ports from last tutorial

Running your own container is the same as running a Redis or Postgres container.
You can also make your own Redis or Postgres image!

Exposing port

A container is an isolated environment, that means, all ports are closed by default.
If you have a http server running inside a container, you should open (expose) your preferred port.
There is a EXPOSE instruction, but I personally prefer using -p <port>:<port> when running a container in most cases, this actually overwrites EXPOSE as well.

Resources and Final quote

I tried to help you understand that docker is not a challenge, it's a tool to make your life easier, this part covered basics of writing a Dockerfile and how to make your custom images.

There are so many tutorials out there, but I'm gonna suggest the ones that I enjoyed the most.
As an instruction that show how containerization will help us: Containerization explained
Docker and Kubernetes course by Stephan Grider

How to make a simple interactive shell in Go

Mohammad Reza Karimi — Tue, 06 Sep 2022 00:54:23 +0000

What is more exciting than making real-world applications? 🤔
for me, it's teaching people how to make them again!

Our Study

As I've spoken about my Dockeroller project earlier in my articles, it's a project that help you control your docker deamon through different gates, such as Telegram or API.

I realized this gates should be easily configurable by user,
We should be able to turn some gates on or off, change their tokens, ports, and passwords, etc.
One approach, was using Viper and dockeroller.yml file (implementing and teaching this approach is in my todo list)

Another approach, can be an interactive shell!
Let's do it!

Completed code is available in my Github gists (Link).

Welcome to the stage!

Try to be clean by separating things!
In this case, I separated my code into different stages,
such as:

Welcome stage
Help stage
Gates stage
Telegram
API

Each stage has a message that we defined using constants at the beginning of the code, in bigger projects, they could be placed in for example a msg package.

We used backtick for multiline strings and we wrote constant type only once because all of them are in same type.



const (
    msg_welcome string = `
Welcome to Dockeroller!
It's sample multiline!
`

    msg_help = `
Dockeroller is an open-source project made for fun.
It's sample multiline!
`

Do it forever!

The simplest form of interactive shells, use an infinite loop and it iterates until you break it,
But don't worry, it's not bad at all, I'll tell you why.

In any iteration, we get something new from the user, in that time, our app is not consuming any resource,
After passing desired input to the shell, it'll process the result and go to the next iteration.
Actually, our shell is constantly sleeping between iterations, it's waiting for us to pass it our input.

Have a look at this code:



func main() {
  var stage int = 0
  for {
    switch stage {
    case 0:
      stage = StageWelcome()
    case 1:
      stage = StageHelp()
    case 2:
      stage = StageGates()
    case 11:
      stage = StageTelegram()
    case 12:
      stage = StageAPI()
    }
  }
}

Functions started with the word StageXxx() are our handlers, they handle what happens in (on 😅) a stage, we'll get into them soon.

Also it may be clear that any stage is assigned to a number, but it could be assigned to anything else, but that may not be needed for a project this size.

As a recap
We have a for loop, we iterate over it and in every iteration, we check which stage we are on (using switch) and handle that stage.

Also, we update our stage after each iteration, but why?

Where would you like to go after?

It's time to update our welcome message!



const (
    msg_welcome string = `
Welcome to Dockeroller!
Where would you like to go after? (choose only number)
1 - help
2 - gates
`

So we let user decide which stage is its next place!
and it's the reason we update stage every time.
Updating stage variable is a part of navigating between Stages.
And each handler, according to its inputs, will decide where we would go next.

Shell input & output

What is the point of this article without going into terminal itself?
This is just so easy, you may know how to use fmt.Print() function, it Prints something on the terminal, Without any default formatting.
There are some other variants of it such as fmt.Println() for printing with a line break and fmt.Printf() for printing with a specified format.

OK, there are a few more functions in fmt package, here we need fmt.Scanln() or fmt.Scan() (there is also a fmt.Scanf() for scanning according to a specified format)

Let's have a very simple example:



package main

import "fmt"

func main() {
  var name string
  fmt.Scanln(&name)
  fmt.Println("hello", name)
}

If you don't pass pointer to Scanln, it won't have any effect.

We can add extra print statements to make it more beautiful.
We can also pass multiple pointers of different types to it, Scan will separate them by space and then populate them.



package main

import "fmt"

func main() {
  var name string
  var age int
  fmt.Print("> ") // Just for beauty
  fmt.Scanln(&name, &age)
  fmt.Println("Hello", name, "you're", age, "years old.")
}

Output will be like:

Basic Handlers

I think this should make sense now:



// Welcome stage handler
func StageWelcome() int {
  fmt.Print(msg_welcome)
  return getStage()
}

// Help stage handler
func StageHelp() int {
  fmt.Print(msg_help)
  return getStage()
}

// Helper function for asking stage number and return it for next decisions
func getStage() (stage int) {
  fmt.Print("> ")
  fmt.Scanln(&stage)
  return
}

If you think it's not clear, I suggest you to put this codes together now and then move on.

Gates handlers

Welcome is a first level stage
Help and Gates are Second level
Telegram & API are third level
and there can be more levels.

How can we realize it?
well, there are many algorithms for it, but here, if we are in gates stage and we should go on with option 1 (telegram) or 2 (api) I add 10 to stage number, and in main switch, telegram will be 11 and api will be 12.
if we want to come back from this level, we will return 0 that means Welcome (stage) handler.



func StageGates() int {

  fmt.Print(msg_gates)

  if stage := getStage(); stage != 0 {

    return stage + 10

  }

  return 0

}

Telegram & API handlers

I just explain one of them, follow the same approach.



func StageTelegram() int {

  fmt.Print(msg_telegram)

// Get a value (token) 

  var token string

  getInput("Token: ", &token)

// Get another value (username) 

  var username string

  getInput("Username: @", &username)

  // Telegram usernames start with @,

  // Some users may include it, some may not,

  // So we helped users by including it.

  // And username variable won't have @ in it.

fmt.Println("Username and token successfully sat!")

  return 0 // Return to the main menu after succefull config

}

// Helper function to print a message and get a value (by populating a pointer)

func getInput(msg string, value interface{}) {

  fmt.Print(msg)

  fmt.Scanln(value) // DO NOT include &, because it's a pointer when is passed to this function!

}

Final quote

I hope you enjoyed this article, feel free to share your thoughts and critics with me.
For more beautiful UIs, you can use this open source package: promptui

How docker will save your time, and why should you learn it immediately. (1/2)

Mohammad Reza Karimi — Thu, 01 Sep 2022 19:20:09 +0000

What is Docker? And why is it popular?
Everywhere on the internet you find articles about how docker is compared with VM and diagrams like this:

But I'm going to teach you Docker just as a tool, so it will be so easier to understand, and we don't dissect it's underlying functionality a lot (we do it just a few 😅).

Don't install new apps! just pull them!

The most interesting thing about docker for a beginner, is the ability to pull different tools so easily.
DockerHub will be like the App Store!

As you see, we can pick up any app that we want, actually we call them images in the world of Docker.

What is an image?

Image is a collection of applications and commands running on a base image and this base image itself is a simplified Linux distro.
This Linux doesn't have a GUI and could be any distro such as Ubuntu, Alpine, Bullseye, etc.

For example, you can install almost any version of any database such as Postgres, Redis, etc, by pulling their image and run them as a container.
You can have multiple versions of the same application at the same time.

What is a container?

Image itself can be supposed as an executable file, it is nothing unless you run it, anytime you run it, a container (process) will be made.
you can run many same containers using an image.

Getting dirty with Redis!

Let's have a hands-on experience with Redis,
Go to this link

In this picture,
Tags can be assumed as different version of images, tag name may include base image name and its version.

Official images are famous projects that are being made constantly by docker team, actually everyone including you and me, can make an image and push it on Dockerhub as a public or private docker image.

At the top right, you can see docker pull redis command, it's the simplest way to pull this image, but we can find more detailed commands at the bottom of that page.

For example this command will pull the redis image and run a container called "my-redis-container":

docker run --name my-redis-container -d redis

-d is used for detaching the process, it will run this container and leave the shell alone for your future use.

But you're not able to use this container, you should expose the Redis port.

docker run --name my-redis-container -p 6379:6379 -d redis

-p 6379:6379 is a port mapping that maps port 6379 of the container to the 6379 of our localhost.
And you can now access Redis through localhost:6379

What other things can I do?

Are you a windows user but you like to use Linux without getting into the trouble of VM or Double boot? Run this command:

docker run --name itubu -it ubuntu

This will make a container named "itubu" and connect your shell to its shell.

the "run" command, first look in your local to see if there is any image called "ubuntu", then if it doesn't exists, docker will pull it from Dockerhub.

Do you want to use Tor network without getting into the trouble of configuring tor? Run this command:

sudo docker run -p 8118:8118 -p 9050:9050 -d dperson/torproxy

dperson/torproxy is an unofficial docker image.

-p 8118:8118 (and -p 9050:9050) is a port mapping that maps port 8118 of container to the 8118 of our localhost.

Now you can proxy your requests through Tor network using this ports.

Do you want to use a headless browser? Run this command

docker run -p 3000:3000 browserless/chrome

And you'll have a chrome browser on localhost:3000
It could be so helpful for scrapping.

We spoke about databases before 😄 and there are so many other usecases.

It mostly needs no effort

Docker itself plays an important role in DevOps world, and without a doubt, it has so many tips and tricks.
But in most cases, as a beginner, you will just need a one line command or a 10 line Dockerfile, to make your project way much better.
The advantages of using Docker is explained in a video I added to the end of this article and I'm not covering them.

Other resources

Final quote

I tried to help you understand that docker is not a challenge, it's a tool to make your life easier, this part included some cool stuff, next part we will cover basics of writing a Dockerfile and how to make your custom images.

Next Part: How docker will save your time, and why should you learn it immediately. (2/2)

[TechStory]: Trying to shutdown a gin server... (goroutines and channels tips)

Mohammad Reza Karimi — Thu, 01 Sep 2022 09:59:26 +0000

Today, I was trying to gracefully shutdown my gin server and start it again through the application itself, without re-running my application.
I learned some precious things that I'd really love to share with you.

So, What's the problem?

Running a server, using gin.Run() or http.ListenAndServe(), will block goroutine, if it's in the main process, so it will block the main goroutine:

func main() {
  ginApp := gin.Default()
  ginApp.Run(":80") // This code is blocking
  // next lines will never run because of blocking code.
  FuncForStoppingGinServer() // This function can't be used.
}

Use another goroutine

We can simply block another goroutine and help main goroutine to remain free.

func main() {
  ginApp := gin.Default()
  go ginApp.Run(":80") // This code is not blocking anymore.
  FuncForStoppingGinServer() // This function will run now.
}

But, what should we do inside FuncForStoppingGinServer()??

Nothing!
The answer is that gin itself doesn't have any function for shutting down its server!
But before getting into that, let's try another wrong way!!

Just kill it!

Are you thinking about stopping your gin server using goroutines and channels? (forget about handling active connections for now)

var ginApp *gin.Engine

func main() {
  // Make an engine
  ginApp = gin.Default()

  // Make a stop channel
  stopChan := make(chan struct{})

  go func() {
    ginApp.Run(":80") // blocking process
    fmt.Println(">> 1 <<")
    for range stopChan {
      return
    }
  }()
  fmt.Println(">> 2 <<")
  // terminate gin server after 3 seconds
  time.Sleep(time.Second * 3)
  stopChan <- struct{}{}
  fmt.Println(">> 3 <<")
}

This sounds to be a good approach, but what do you think is wrong here?

Which numbers will be printed? (1, 2, 3)?

As I mentioned before, ginApp.Run(":80") is a blocking process and it blocks its goroutine, so we will send to channel stopChan,
BUT,
we will never be able to read from the stopChan using our for range loop, because that came after a blocking process and ">> 1 <<" will not be printed.

And as our channel is not buffered, sender that is main goroutine also will be blocked until somebody read from that!, exactly before printing ">> 3 <<" and it will not be printed too.

">> 2 <<" will be printed anyway 😄

Additional fun fact: If we make our channel buffered (size 1 is enough for this case), main goroutine will send to it and go ahead, then it will complete its work and close, whenever main goroutine job is done, the whole program exists! and any other goroutine will die with it, just make this change and see the result:

stopChan := make(chan struct{}, 1)

I also should mention that chan struct{} as the channel type and empty struct struct{}{} will consume no memory at all, and it's known as signal only channel.

I finally realized there is no way to solve this problem this way:
Is there a way to stop a long blocking function?

You can also learn more about stopping goroutines in this Question: How to stop a goroutine
And I'll cover more examples later. (you can follow me to get noticed)

Back into the problem!

So using Goroutines didn't solved our problem,
How about trying http.Server.Shutdown() method?

var ginApp *gin.Engine

func main() {
  // Make an engine
  ginApp = gin.Default()

  // Make a http server
  httpServer := &http.Server{
    Addr:    ":80",
    Handler: ginApp,
  }

  // Launch http server in a separate goroutine
  go httpServer.ListenAndServe()

  // Stop the server
  time.Sleep(time.Second * 5)
  fmt.Println("We're going to stop gin server")
  httpServer.Shutdown(context.Background())
}

It may be helpful to read this question: Graceful stop of gin server

It works, but what is still wrong?

Now, we are shutting down our server inside of our program, but how can we start it again?
Maybe by running ListenAndServe() again?

As I asked in this question: How to stop and start gin server multiple times in one run
I'm trying to implement something like this semi-code:

srv := NewServer()
srv.Start()
srv.Stop()
srv.Start()
srv.Stop()
srv.Start()

Let's do it by a little refactor.

type Server struct {
  httpServer *http.Server
}

func main() {
  srv := NewHTTPServer()
  srv.Start()
  time.Sleep(time.Second * 2)
  srv.Stop()
  time.Sleep(time.Second * 2)
  srv.Start()

  select {} // Simulate other processes
}

func NewHTTPServer() *Server {
  return &Server{
    httpServer: &http.Server{
      Addr:    ":80",
      Handler: gin.Default(),
    },
  }
}

func (s *Server) Start() {
  go func() {
    if err := s.httpServer.ListenAndServe(); err != nil {
      fmt.Println(">>>", err)
    }
  }()
}

func (s *Server) Stop() {
  s.httpServer.Shutdown(context.Background())
}

Run this code and see what happens!
Output:

...Gin logs...
>>> http: Server closed
>>> http: Server closed

What? we called srv.Stop() just once, but our server closed twice!
Why?
According to .Shutdown() functions docs:

Once Shutdown has been called on a server, it may not be reused; future calls to methods such as Serve will return ErrServerClosed.

So, it's officially not working!

But there is still one little change, we can make the server struct again from scratch, every time we start it.

Add a serve function to the last code and change main function, other codes are the same.

func main() {
  srv := serve()
  time.Sleep(time.Second * 2)
  srv.Stop()
  time.Sleep(time.Second * 2)
  srv = serve()
  srv.Start()

  select {} // Simulate other processes
}

func serve() *Server {
  srv := NewHTTPServer()
  // Register handlers or other stuff
  srv.Start()
  return srv
}

This is something that we want, but you can make the code cleaner.

Final quote

This article had a different feeling for myself, it was a up and down path with some tips and some not working approaches, I hope you feel the same and I'll be happy hear your thoughts.

How to Unmarshal JSON in a custom way in Golang

Mohammad Reza Karimi — Thu, 04 Aug 2022 13:39:28 +0000

When I was working on gocoinex library for the first time, I needed some custom configuration on JSON unmarshalling process, I want to share my learnings with you here, but before that, we're going to have a recap on how we can unmarshall JSON in golang.

These are real world examples. (coinex exchange API)

First, we make our request like this:

raw_response, _ := http.Get("https://api.coinex.com/v1/market/list")

And we'll get a json object like this:

{
    "code": 0,
    "data": [
        "LTCBCH",
        "ETHBCH",
        "ZECBCH",
        "DASHBCH"
    ],
    "message": "Ok"
}

But we should parse it,
There are different ways to parse it,
You can use NewDecoder or Unmarshal functions of json package,
and you can decode it to a struct or to a map[string]interface{}

It depends on your preferences, but in this case, I prefer NewDecoder and struct combination.

So we should make a struct like this:

type AllMarketList struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    []string
}

Also we can have embedded structs, for example, we break last struct to two:

type GeneralResponse struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
}

type AllMarketList struct {
    GeneralResponse
    Data    []string
}

And there is no difference.

Finally, use NewDecoder to decode the raw_response to AllMarketList struct:

var allMarketList AllMarketList
json.NewDecoder(raw_response.Body).Decode(&allMarketList)

Completed code

package main

import (
    "encoding/json"
    "fmt"
    "net/http"
)

type AllMarketList struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    []string
}

func main() {
    raw_response, _ := http.Get("https://api.coinex.com/v1/market/list")

    var allMarketList AllMarketList
    if err := json.NewDecoder(raw_response.Body).Decode(&allMarketList); err != nil {
        fmt.Println(err)
    }
    defer raw_response.Body.Close()
    fmt.Printf("%+v\n", allMarketList)
}

Example Two

Think we have a json like this:

{
  "code": 0,
  "data": {
    "date": 1513865441609, # server time when returning
    "ticker": {
        "open": "10", # highest price
        "last": "10.00", # latest price 
        "vol": "110" # 24H volume
    }
  },
  "message" : "Ok"
}

We're going to improve some things in decoding process

In this case, Unix timestamp can't be parsed, we should provide it a way.
We want to remove "ticker" key and access to "open", "last", "vol" directly from "data".
"last" should be exported to field named "Close"! (same happens with "vol" and "Volume".
"open", "last", "vol" should be float, not string, but we'll leave them for next example.

Problems 1 and 2, can be solved by implementing UnmarshalJSON method on any struct we want to decode.
problem 3 will be easily solved with json tags. (I've mentioned it in the code below)

Our final struct should be like this:

// Final struct
type SingleMarketStatistics struct {
    Code    int    `json:"code"`
    Message string `json:"message"`
    Data    TickerData
}

// Inner struct that we should implement to solve problem 2
type TickerData struct {
    ServerTime CTime   `json:"date"` // CTime is short for CustomTime
    Open       float64 `json:"open"`
    Close      float64 `json:"last"` // Different Attribute name and tag name
    Volume     float64 `json:"vol"` // Different Attribute name and tag name
}

// Custome time
// Inner struct that we should implement to solve problem 1
type CTime struct {
    time.Time
}

Custom time implementation

func (t *CTime) UnmarshalJSON(data []byte) error {
    // Ignore null, like in the main JSON package.
    if string(data) == "null" || string(data) == `""` {
        return nil
    }
    // Fractional seconds are handled implicitly by Parse.
    i, err := strconv.ParseInt(string(data), 10, 64)
    update := time.UnixMilli(i)
    *t = CTime{update}
    return err
}

And we don't get an error anymore!
This method will be automatically used (thanks to interfaces!) whenever we want to decode a time to CTime!

Custom Data implementation

func (t *TickerData) UnmarshalJSON(data []byte) error {
    if string(data) == "null" || string(data) == `""` {
        return nil
    }

    // This is how this json really looks like.
    var realTicker struct {
        ServerTime CTime `json:"date"`
        Ticker     struct {
            // tags also can be omitted when we're using UnmarshalJSON.
            Open   string `json:"open"`
            Close  string `json:"last"`
            Volume string `json:"vol"`
        } `json:"ticker"`
    }

    // Unmarshal the json into the realTicker struct.
    if err := json.Unmarshal(data, &realTicker); err != nil {
        return err
    }

    // Set the fields to the new struct,
    // with any shape it has,
    // or anyhow you want.
    *t = TickerData{
        ServerTime: realTicker.ServerTime,
        Open:       realTicker.Ticker.Open,
        Close:      realTicker.Ticker.Close,
        Volume:     realTicker.Ticker.Volume,
    }

    return nil
}

Now, just use NewDecoder as before, no change is needed.

var singleMarketStatistics SingleMarketStatistics 
json.NewDecoder(raw_response.Body).Decode(&allMarketList)

Example Three

Imagine a JSON like this:

{
  "asks": [ // This is a array of asks
    [ // This is a array of ONE ask
      "10.00", // Price of ONE ask
      "0.999", // Amount of ONE ask
    ]
  ],
  "bids": [ // Same structure as asks
    [
      "10.00",
      "1.000",
    ]
  ]
}

As it's totally clear, in the non professional way, we should decode "asks" to [][]string and access first ask price like this asks[0][0] and amount asks[0][1]
Who remembers 0 is price and 1 is amount? What was which? 😄
So we'll manage them on the UnmarshalJSON method.
Also, we'll solve problem 4 of previous example that also exists here.

type BidAsk struct {
    // Tags are not needed.
    Price  float64 `json:"price"`  // Bid or Ask price
    Amount float64 `json:"amount"` // Bid or Ask amount
}

func (t *BidAsk) UnmarshalJSON(data []byte) error {
    // Ignore null, like in the main JSON package.
    if string(data) == "null" || string(data) == `""` {
        return nil
    }

    // Unmarshal to real type.
    var bisask []string
    if err := json.Unmarshal(data, &bisask); err != nil {
        return err
    }

    // Change value type from string to float64.
    price, err := strconv.ParseFloat(bisask[0], 64)
    if err != nil {
        return err
    }
    amount, err := strconv.ParseFloat(bisask[1], 64)
    if err != nil {
        return err
    }

    // Map old structure to new structure.
    *t = BidAsk{
        Price:  price,
        Amount: amount,
    }
    return err
}

type MarketDepth struct {
    Asks   []BidAsk `json:"asks"` // Ask depth
    Bids   []BidAsk `json:"bids"` // Bid depth
}

Again, we simply use:

var marketDepth MarketDepth 
json.NewDecoder(raw_response.Body).Decode(&marketDepth)

And enjoy the beauty of your result:

for i, ask := range data.Data.Asks {
    fmt.Printf("Ask %v\n", i)
    fmt.Printf("  Price: %v\n", ask.Price) // here is the beauty
    fmt.Printf("  Amount: %v\n", ask.Amount) // here is the beauty
    fmt.Println()
}
for i, bid := range data.Data.Bids {
    fmt.Printf("Bid %v\n", i)
    fmt.Printf("  Price: %v\n", bid.Price) // here is the beauty
    fmt.Printf("  Amount: %v\n", bid.Amount) // here is the beauty
    fmt.Println()
}

How to use Docker Multistage to make tiny images (TS and Golang)

Mohammad Reza Karimi — Wed, 03 Aug 2022 23:09:14 +0000

Have you ever wondered how you can remove redundant build dependencies from your docker image?

Did you know your image can be easily optimized?

We're not much language-related in this article, and reading all examples can be beneficial.

TS example

For example, maybe you've always been using this kind of Dockerfile to build and run your Typescript project:

FROM node:18-alpine3.15
WORKDIR /usr/app
COPY package.json .
RUN npm install --include=dev
COPY tsconfig.json .
COPY src src
RUN ["npm", "run", "build"]
CMD ["npm", "run", "start"]

Simple, but it can be smaller without losing anything important,
Because, on the line

RUN ["npm", "run", "build"]

We are converting some TS codes to JS codes with some Tools (dev dependencies)
What happens if we remove those tools after conversion? Nothing!

Have a look at picture below, we've built our image in three stages, instead of one, and we saved 35% of space!

With code below:

FROM node:18-alpine3.15 as ts-compiler
WORKDIR /usr/app
COPY package*.json ./
COPY tsconfig.json ./
RUN npm install --include=dev
COPY src src
# Build the project (TS to JS conversion)
RUN ["npm", "run", "build"]

FROM node:18-alpine3.15 as ts-remover
WORKDIR /usr/app
# We need package.json again
COPY --from=ts-compiler /usr/app/package*.json ./
# Move built codes from last stage here
COPY --from=ts-compiler /usr/app/build ./
# We don't need dev dependencies anymore
RUN npm install --omit=dev

# Using google optimized containers can make it even smaller
FROM gcr.io/distroless/nodejs:18
WORKDIR /usr/app
COPY --from=ts-remover /usr/app ./
USER 1000
CMD ["index.js"]

Be careful about environment variables

Imagine you are using a special package, for example Puppeteer
Puppeteer will download a chrome browser when you're installing it, but you can disable it if you want, by setting a ENV.
Have a look at the change we made on stage one of last code:

FROM node:18-alpine3.15 as ts-compiler
WORKDIR /usr/app
COPY package.json .

# This line is added 👇🏻
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true

RUN npm install --include=dev
COPY tsconfig.json .
COPY src src
RUN ["npm", "run", "build"]

So if you run multistage build again, you'll see a huge increment 😳

What is happening?
The fact is, you should use ENV and ARG commands per Stage, we are installing dependencies twice in two stage, so we should skip chrome installation twice!
Nothing talks more than a piece of code 😄:

FROM node:18-alpine3.15 as ts-compiler
WORKDIR /usr/app
COPY package.json .

# Define ENV once here
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
RUN npm install --include=dev
COPY tsconfig.json .
COPY src src
RUN ["npm", "run", "build"]

FROM node:18-alpine3.15 as ts-remover
WORKDIR /usr/app
COPY --from=ts-compiler /usr/app/package.json .
COPY --from=ts-compiler /usr/app/build .

# Define ENV once again!
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
RUN npm install --omit=dev

# We don't need that ENV in this stage, 
# because we're not installing anything more.
FROM gcr.io/distroless/nodejs:18
WORKDIR /usr/app
COPY --from=ts-remover /usr/app ./
USER 1000
CMD ["index.js"]

Go example

Go containers (and any other compile language) can be optimized even more. (much more 😄)
Because their runtime is tied to them, you don't need something like Nodejs installed and you can drop compiler itself in the second stage and just use a binary executable.

FROM golang:1.18-alpine3.15 AS builder
WORKDIR /app
# Copy go.mod and go.sum first, because of caching reasons.
COPY go.mod go.sum ./
RUN go mod download
COPY . ./
# Compile project
RUN CGO_ENABLED=0 GOOS=linux go build -a -o main .

# Use another clean image without Golang compiler.
FROM alpine:3.15 AS production
COPY --from=builder /app/main .
CMD ["./main"]

It's right! we saved about 96% of our space!
First result is without multistage technique,
Second is using alpine-3.15 as production.
Third is using gcr.io/distroless/static-debian11 that is the smallest image out there.

Including UI

You can even add a UI to your Dockerfile in another stage!
In this case, I have a Svelte SPA project beside my project (in ui folder) and I'm serving it from Go Backend.

FROM golang:1.18-alpine3.15 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . ./
RUN CGO_ENABLED=0 GOOS=linux go build -a -o main .

# This stage convert svelte project to vanilla HTML, CSS, JS.
FROM node:18-alpine3.15 AS frontend
WORKDIR /ui
COPY ui .
RUN npm install
RUN npm run build

FROM alpine:3.15 AS production
COPY --from=builder /app/main .
COPY --from=frontend /ui/public /ui
CMD ["./main"]

This code is a part of Blogo project.
https://github.com/arshamalh/blogo

I hope you learned something from me here. 😃✌🏻
Any suggestion or correction is welcome.

DEV Community: Mohammad Reza Karimi

[TechStory]: How to add distributed tracing using Jaeger and OpenTelemetry into a Golang application

What is Tracing

What is distributed tracing

What tools can we use

Basic Setup

So what is this OTLP thing?

Propagation

Summary

[TechStory]: Migrating services and databases from an OpenShift (or K8s) cluster to another

So, What's the problem?

Where did I start?

Not good enough solution!

Steps to follow:

Better solutions

What about DB?

Steps to follow:

Step 1: Login and port-forward

Step 2: tools

Step 3: take a dump

Step 4: load the dump

Conclusion

Make your cool commands in terminal

TL;DR

The story behind this article

Define aliases

Define functions

Some examples

Just if you are wondering (Used terms)

Conclusion

How docker will save your time, and why should you learn it immediately. (2/2)

Make a Dockerfile in the root

Starting our image

Command the image!

Build images

Caching and layered images

Running your image (container)

Exposing port

Resources and Final quote

How to make a simple interactive shell in Go

Our Study

Welcome to the stage!

Do it forever!

Where would you like to go after?

Shell input & output

Basic Handlers

Gates handlers

Telegram & API handlers

Final quote

How docker will save your time, and why should you learn it immediately. (1/2)

Don't install new apps! just pull them!

What is an image?

What is a container?

Getting dirty with Redis!

What other things can I do?

It mostly needs no effort

Other resources

Final quote

[TechStory]: Trying to shutdown a gin server... (goroutines and channels tips)

So, What's the problem?

Use another goroutine

Just kill it!

Back into the problem!

It works, but what is still wrong?

Other solutions

Final quote

How to Unmarshal JSON in a custom way in Golang

Completed code

Example Two

Custom time implementation

Custom Data implementation

Example Three

How to use Docker Multistage to make tiny images (TS and Golang)

TS example

Be careful about environment variables

Go example

Including UI