.NET HttpClient extensibility: Support for UnixDomain Sockets & Named Pipes

.NET HttpClient is main interface for sending HTTP requests and receiving HTTP responses from a resource identified by a URI. Generally this API supported TCP stack. In .NET 5 , library is extended to support other transports such as Unix Domain Sockets & NamedPipes because many web server on Unix supported HTTP server implementations using Unix Domain Sockets . Windows also added Unix domain socket support in Windows 10 ( for more details see here)

ASP.NET Core web server (Kestrel) also added support for UNIX domain sockets and named pipes( from .NET 8 on-wards).

Alternative transport options is introduced in HttpClient SocketsHttpHandler. With this it is possible to connect to server with Unix Domain Socket or Named Pipes.

HttpClient with Unix Domain Socket

SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
// Custom connection callback that connects to Unixdomain Socket
socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
{
    Uri dockerEngineUri = new Uri("unix:///var/run/docker.sock");
    var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);

    var endpoint = new UnixDomainSocketEndPoint(dockerEngineUri.AbsolutePath);
    await socket.ConnectAsync(endpoint, ctxToken);
    return new NetworkStream(socket);
};

// create HttpClient with SocketsHttpHandler
var httpClient  = new HttpClient(socketsHttpHandler);
// make Http Request .

HttpClient with Named Pipe

SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
// Custom connection callback that connects to NamedPiper server
socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
{
    Uri dockerEngineUri = new Uri("npipe://./pipe/docker_engine");
    NamedPipeClientStream pipeClientStream = new NamedPipeClientStream(dockerEngineUri.Host,
                                            dockerEngineUri.Segments[2],
                                            PipeDirection.InOut, PipeOptions.Asynchronous);
    await pipeClientStream.ConnectAsync(ctxToken);
    return pipeClientStream;
};

// create HttpClient with SocketsHttpHandler
var httpClient  = new HttpClient(socketsHttpHandler);
// make Http Request .
        

Complete demonstration sample that uses HttpClient connecting to Docker daemon serving REST API via NamedPipe on Windows & Unix Domain socket on Linux.


using System.IO.Pipes;
using System.Net.Http;
using System.Net.Http.Json;
using System.Net.Sockets;
using System.Runtime.InteropServices;

HttpClient client = CreateHttpClientConnectionToDockerEngine();
String dockerUrl = "http://localhost/v1.41/containers/json";
var containers = client.GetFromJsonAsync<List<Container>>(dockerUrl).GetAwaiter().GetResult();
Console.WriteLine("Container List:...");
foreach (var item in containers)
{
    Console.WriteLine(item);
}


// Create HttpClient to Docker Engine using NamedPipe & UnixDomain
HttpClient CreateHttpClientConnectionToDockerEngine()
{
    SocketsHttpHandler socketsHttpHandler =
        RuntimeInformation.IsOSPlatform(OSPlatform.Windows) switch
        {
            true => GetSocketHandlerForNamedPipe(),
            false => GetSocketHandlerForUnixSocket(),
        };
    return new HttpClient(socketsHttpHandler);

    // Local function to create Handler using NamedPipe
    static SocketsHttpHandler GetSocketHandlerForNamedPipe()
    {
        Console.WriteLine("Connecting to Docker Engine using Named Pipe:");
        SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
        // Custom connection callback that connects to NamedPiper server
        socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
        {
            Uri dockerEngineUri = new Uri("npipe://./pipe/docker_engine");
            NamedPipeClientStream pipeClientStream = new NamedPipeClientStream(dockerEngineUri.Host,
                                                    dockerEngineUri.Segments[2],
                                                    PipeDirection.InOut, PipeOptions.Asynchronous);
            await pipeClientStream.ConnectAsync(ctxToken);
            return pipeClientStream;
        };
        return socketsHttpHandler;
    }
    // Local function to create Handler using Unix Socket
    static SocketsHttpHandler GetSocketHandlerForUnixSocket()
    {
        Console.WriteLine("Connecting to Docker Engine using Unix Domain Socket:");
        SocketsHttpHandler socketsHttpHandler = new SocketsHttpHandler();
        // Custom connection callback that connects to Unixdomain Socket
        socketsHttpHandler.ConnectCallback = async (sockHttpConnContext, ctxToken) =>
        {
            Uri dockerEngineUri = new Uri("unix:///var/run/docker.sock");
            var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP);

            var endpoint = new UnixDomainSocketEndPoint(dockerEngineUri.AbsolutePath);
            await socket.ConnectAsync(endpoint, ctxToken);
            return new NetworkStream(socket);
        };
        return socketsHttpHandler;
    }
}

/// <summary>
/// Record class to hold the container information
/// </summary>
/// <param name="Names"></param>
/// <param name="Image"></param>
/// <param name="ImageID"></param>
/// <param name="Command"></param>
/// <param name="State"></param>
/// <param name="Status"></param>
/// <param name="Created"></param>
public record Container(List<String> Names, String Image, String ImageID,
                        String Command, String State, String Status, 
                        int Created);

Conclusion:

.NET HttpClient extensibility allows connecting to HTTP server implemented using non TCP transport such as Unix Domain Socket & Named Pipes.

Reference:

SocketsHttpHandler Extension Points

Docker Engine REST API Reference

Accessing Docker API from .NET Core

Recently I had to access container stats ( CPU & Memory) for the container apps on Linux system with limited access ( only ssh access via multiple intermediate jump servers). We could not install cAdvisor or other UI tools. I tried capturing container stats by running Docker stats command every 5 seconds but the output of the Docker stats less than ideal for analysis like plotting memory/cpu graphs.

So I decided to write simple tool using Docker.DotNet C# SDK . Requirement was to write a self contained utility that can run on Linux & log container stats entries in comma separate format(CSV) for easier analysis using excel.

This tool uses Docker.DotNet , a C# SDK that uses Docker REST API to interact with Docker Daemon and .NET self contained publish profile that produces a platform-specific executable.

Complete source code is available on my GitHub repo ,

Here is the sample run & the output

Output format is has follows ContainerName,CpuPercent,MemUsageInKB,MaxAviMemInKB,MemUsageInPercent,DateTime

Conclusion

The Docker API is a powerful tool that allows you to control Docker from your own applications. By accessing the Docker API from C#, you can automate Docker operations, create your own Docker tools, and integrate Docker with other systems.

Reference

Load Balanced DICOM Classic CStore Service deployed on Docker

This post covers the demonstration setup of DICOM CStore service with load balancer that supports scaling of DICOM request across multiple instance.

DICOM Classic CStore service is a DICOM image transfer service that allows DICOM Application Entities (Client’s) to send DICOM objects to a DICOM storage server (Server). The CStore service is based on the DICOM Imaging Network Communication Standard (DICOM Standard) and is defined in Part 7 of the DICOM Standard. DICOM is an application protocol built on top of TCP/IP stack/

The CStore service is a widely used in hospital Radiology where it is typically used to store DICOM images from medical imaging devices, such as MRI scanners and CT scanners.

The CStore service is a request/response service, which means that the sending AE ( client) sends a request to the receiving AE(server) to store a DICOM object. The receiving AE then sends a response to the sending AE indicating whether the DICOM object was successfully stored.

Following example demonstrate the setup where multiple instances of CStore service deployed in Docker load balanced using Nginx Proxy .

Complete source code & deployment script on my GitHub repo

Reference

DICOM Standard

Java DICOM Library Reference

Using host.docker.internal to access the Docker host from inside a container

Docker containers are isolated from the host machine, which means that they cannot access the host machine’s network by default. However, there are a ways to allow containers to access the host machine’s network, one of which is to use the host.docker.internal hostname.

host.docker.internal is a special hostname that resolves to the IP address of the Docker host machine. This means that containers can access the host machine’s network by connecting to host.docker.internal.

One common use case for using host.docker.internal is to allow a container to access a service ex: database that is running on the host machine. For example, you might have a container that is running a web application that needs to access a MySQL database that is running on the host machine as shown in the diagram below. By configuring web application to use host.docker.internal with port number , application can reach the database service.

Below example shows connecting to service running on host from inside the container on Docker desktop running on Windows

On Windows Docker installation accessing host.docker.internal is automatic, on Linux this requires adding --add-host command line option for docker run command .

docker run --rm -it –add-host=host.docker.internal:host-gateway <<container>>

Conclusion:

host.docker.internal is a useful tool for accessing services running on the host machine from inside a container. It is easy to use, portable, and reliable.

Reference

Docker Networking Documentation

Docker Run command line options

Docker Init CLI with ASP.NET Core application

Docker Init CLI is a new command-line interface (CLI) command that simplifies the process of adding Docker to a project. It can be used to generate a Dockerfile and Docker Compose file for an ASP.NET Core application, as well as build and run the application in a container.

To use Docker Init CLI with an ASP.NET Core application, first install the Docker CLI and Docker Compose or install Docker Desktop 4.23 or greater Then, navigate to the directory of your ASP.NET Core application and run the following command:

docker init

Based on your current project settings, cli will prompt for series of questions, you can choose to use defaults and the end it will generate a Dockerfile and Docker Compose file in the current directory.

To build the Docker image for your ASP.NET Core application, run the following command:

docker build -t aspnetapp .

This will build the Docker image and tag it with the name aspnetapp.

To run the ASP.NET Core application in a container, use docker run or docker-compose up

docker-compose up -d

This will start the Docker Compose stack, which will create and start a container for your ASP.NET Core application. You can now access the ASP.NET Core application at http://localhost:8080 in your web browser.

Conclustion

Docker Init CLI makes it easy to add Docker to an ASP.NET Core project. It generates a Dockerfile and Docker Compose file for you, so you don’t have to write them yourself.

Reference

Docker Init Cli command Reference

Docker Desktop 4.24