Getting network interface information in Java

In my Android application, I’ll need to find out which network interfaces are available and which IP addresses they have. Turns out the necessary API is really simple:
java.net.NetworkInterface. And being part of java.net it’s not even Android specific. 🙂

There’s a static function to get all active network interfaces:

Enumeration<NetworkInterface> ifs = null;
try {
    ifs = NetworkInterface.getNetworkInterfaces();
} catch (SocketException ex) {
    // error handling appropriate for your application
}
if (ifs == null)
    // cancel the examination of "ifs"

ifs will be null if there are no active interfaces. Otherwise, just get each interface in turn and read its addresses. For example, you could write them to standard output:

while (ifs.hasMoreElements()) {
    NetworkInterface iface = ifs.nextElement();
    System.out.println(iface.getName());
    Enumeration<InetAddress> en = iface.getInetAddresses();
    while (en.hasMoreElements()) {
        InetAddress addr = en.nextElement();
        String s = addr.getHostAddress();
        int end = s.lastIndexOf("%");
        if (end > 0)
            System.out.println("\t" + s.substring(0, end));
        else
            System.out.println("\t" + s);
    }
}

The string modification at the end might seem strange, but without it, IPv6 addresses would be printed with an appended “%X” where X is a number indicating the address scope.

If you want to try this on Android, you could append the addresses to an android.widget.TextView. Note that your application will need the INTERNET permission to read the requested data. Just put the following into the application’s AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET" />

Testing my Android SDK

After building my Android SDK, it is now time to learn how to write applications. I have quite a bit of experience with Java, but there’s a bunch of Android-specific stuff to take care of. Unlike the Android SDK build I complained about, the documentation about application development is easy to find and comprehensive. I started with reading about “Application Fundamentals” and switched to the “Hello World” tutorial after the basics to get some hands-on practice before going into theoretical details.

I prefer to work with Emacs instead of using an IDE, so I set up the project as described here. Things mostly worked as expected, but I had to solve two small problems before the application would build successfully. I use the

$ ant install

target for building and testing. This should build the application, sign it with a debug key (only for testing, not release) and install it into a currently running emulator (exactly one emulator must be running for this to work). In the first try, I got this message:

For 'AOSP' SDK Preview, attribute minSdkVersion in AndroidManifest.xml must be 'AOSP'

Yes, my self-build SDK is an AOSP (“Android Open Source Preview”) and the solution was to add a <uses-sdk> element to the manifest:

<uses-sdk android:minSdkVersion="AOSP" />

The second try failed as well, because for some reason the debug key was not created automatically (as the documentation says it should). So I had to create it by hand:

$ keytool -genkey -alias androiddebugkey -keyalg RSA -keystore ~/.android/debug.keystore -storepass android -validity 365

The next try failed with “device not found,” but after rebooting the emulator (as suggested in the documentation), it worked. 🙂

A modified "Hello World" application running in the Android emulator

As you can see, I added an input field to add text to the TextView. I really like the possibility to build the view structure through XML files, though I won’t put too much more effort into GUI stuff for now. What I need to write is a service.

What is a mutex?

It took me a while to understand what a mutex really is, so here is my attempt to write a simple explanation.

So, what is a mutex?

“Mutex” is short for “mutual exclusion.” Technically, when talking about “a mutex” I’m referring to a special type of variable that can be in one of two states: “locked” or “unlocked”. Mutexes are used to prevent access conflicts between different threads in an application. The principle of using a mutex is simple: A thread locks the mutex before a potentially conflicting operation, does the operation, then unlocks the mutex again. It is important to understand that a mutex does not “magically” protect anything. It is like a baton: If one thread has the mutex locked, the next one trying to lock it will have to wait until it becomes unlocked. All of this depends on the programmer putting the lock and unlock calls in the right places.

When do I need a mutex?

You should use a mutex wherever concurrent access to a variable by different threads can be a problem. Read-only accesses are okay, but writes should usually be protected. Note that in this case reads must be protected as well to prevent reading during a “halfway done” write. There are exceptions, if you want to learn about them look for “atomic operations.”

Example

So, on to a little example that will launch a thread for each command line argument. Each thread will print its argument with a line number in front. Note that the example uses declarations inside for-loop initialization, so you’ll need a C99 compatible compiler if you want to compile it, e.g.:

$ gcc -std=c99 -pthread mutex_example.c

#include <pthread.h>
#include <stdio.h>

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
unsigned int count = 0;

void *thread_function(void *arg)
{
	pthread_mutex_lock(&mutex);
	int i = count;
	count = i + 1;
	printf("%i: %s\n", i, (char *) arg);
	pthread_mutex_unlock(&mutex);
	return NULL;
}

int main(int argc, char **argv)
{
	int n = argc - 1;
	pthread_t threads[n];
	for (int i = 0; i < n; i++)
		pthread_create(&threads[i], NULL,
			thread_function, argv[i + 1]);
	for (int i = 0; i < n; i++)
		pthread_join(threads[i], NULL);
	return 0;
}

The line numbers make only sense if they are consecutive, so we need to make sure that the line count does not change until the line was printed. Otherwise, two threads could read the same current count and write back the same incremented value, resulting in a too low number. So, each thread does this:

  1. Get the mutex.
  2. Get and increment the line number.
  3. Print the line.
  4. Unlock the mutex.

If there is more than one thread, one of them will get the mutex first and the others will have to wait until it is done. I know this could be done in a much shorter way which would not require a mutex to be thread-safe (bonus points if you know, too), but I think it shows the idea behind using mutexes. 😉

Caveats

I don’t claim this list to be conclusive, but you should look out for the following when using threads and mutexes:

  • Deadlocks: Thread one has mutex one locked and is waiting to get mutex two, while thread two has mutex two locked and is waiting to get mutex one.
  • Concurrency problems that are not results of concurrent variable access and can not be solved by mutexes.
  • Using mutexes where not necessary. This should not break the program, but it can hurt performance.
  • Obvious, but still: forgetting places that would need a mutex.

Have fun with threads!

E-Post? We’ve had that…

The Deutsche Post (German Mail company) is offering a new email-like service called “E-Postbrief” with which they claim to bring “privacy of correspondence to the internet.” It is also supposed to allow reliable delivery of documents, suitable for contracts and similar stuff. Others have written enough about the various privacy, security and other concerns, so I’m just going to summarize them here:

There might be some more, but I just want to add a historical side note. The domain the Deutsche Post uses for the E-Postbrief service is “epost.de”, and that reminded me of something: Starting from 2000-06-26, they offered a “lifelong valid” email address under the same domain. Well, “livelong” was a bit of exaggeration in this case, because they closed the service on 2005-02-28. I’ll let you draw your own conclusions from that… 😉

(Personally, I prefer to rely on GnuPG and Enigmail for E-mail privacy.)

Building the Android SDK

When I was asked if I could write an application of Android, one of the first things I did was taking a look at the SDK download page, where I was disappointed: There are binary SDKs for Linux i386, Mac OS and Windows, but the most important thing was missing: A source download. It took me a rather long time to find what I was looking for: “How to build an Android SDK.” I wonder why this document seems to exist in the git tree only and isn’t mentioned anywhere on the Android developer website.

Preparations

The first thing to do is get the Android source code. There are some strange requirements listed:

  • A 32-bit system or a 32-bit compatible environment on a 64-bit system
  • JDK 5.0, update 12 or higher, but not Java 6

The first requirement is annoying because I usually use pure 64-bit systems. However, my development system supports KVM, so I decided to set up a dedicated 32-bit virtual Xubuntu. The “Java 5 only” is really weird – in a worst case scenario javac’s -source option should still make it possible to compile old code. I chose to just ignore that and use Ubuntu’s openjdk-6-jdk. The process to actually download everything and start the build is described in the documentation, so I won’t repeat that here.

CFLAGS Trouble

After a while, the build failed with an error about gnu/stubs-64.h not found from gcc and an incompatible libstdc++ error from the linker. I guess that the second one was caused by the first, but that one surprised me quite a bit. gnu/stubs-64.h is included from /usr/include/gnu/stubs.h, with an #if construct that chooses stubs-64.h or stubs-32.h depending on the __WORDSIZE macro. As recommended by the official docs I’m using a 32-bit system for the build. So why is the build system looking for a header used on 64-bit systems? stubs-32.h is where it should be. I tried

CFLAGS="-m32" make -j4

to force a 32-bit build, but it didn’t help. I searched and found the forum thread linked above, and from there the cause of the problem. For reasons unknown to me, someone thought it would be a good idea to force a 64-bit build if Java 6 is in use, even on a 32-bit system. It is not!

Well, the next step was simple: Edit the files in question and remove the “-m64.” The files:

external/clearsilver/java-jni/Android.mk
external/clearsilver/cgi/Android.mk
external/clearsilver/util/Android.mk
external/clearsilver/cs/Android.mk

The code to remove (or uncomment):

# This forces a 64-bit build for Java6
ifneq ($(filter 1.6%,$(java_version)),)
    LOCAL_CFLAGS += -m64
    LOCAL_LDFLAGS += -m64
endif

After that, “make -j4 sdk” worked (modify the number of jobs as appropriate for your system). After a long time, I had my own SDK in out/host/linux-x86/sdk/. The finished SDK is a zip file, after unpacking it took 456MB – not much compared to the 9.3GB in the build directory (sources, temporary files, …). Of course, I immediately tried the emulator:

The Android emulator is running in a virtual Xubuntu

As you can see, it worked. 🙂 And the official documentation is clearly wrong: You do not need Java 5 to build Android. 😛

The toolchain problem

Sounds good? Well, there’s a problem: While my new SDK is locally built, not everything used to build it is, and I’m not talking about distro packages here. When you download the Android source, you’ll also get a directory called “prebuilt.” This contains a bunch of binaries, for example the ARM toolchain used for cross-compiling. So I’ll have to look into replacing that as well to get a really independent SDK build…

The difference between だ and ある or いる

Depending on context, all three of the verbs だ, ある and いる might be translated with some form of “to be.” So, what’s the difference between them?

だ is the literal translation for “to be” and also the simple form for present tense. Depending on the social situation it might be a good idea use the more polite form です instead. Someone named 夏子 (なつこ) could introduce him or herself like 「夏子です。」 which means “I am Natsuko.” Another example would be 「いちごがすきだ。」 (“I like strawberries.”)

ある and いる

“To exist” is probably a better translation for these two verbs. They can describe both that something exists at all (no matter where) and at a particular location (so it is there – that’s the connection with “to be”). The difference between them is that いる is used for living beings (humans, animals) and ある for everything else.

こうえんにはねこがいる。 (“There is a cat in the park.”)

ケーキがある。 (“There is a cake” or maybe “I have a cake.”)

Notice

だ, ある and いる are simple forms, used when talking to good friends or family members. When talking to others, you should consider using more polite forms as appropriate.

Using GVariant tuples

A GVariant stores data and type information for that data. The data can be simple, like numeric values or strings, or more complex like arrays of simple types or tuples of other valid types. The type(s) of data contained in a GVariant is (are) described by a format string. This format string can be a short "s" or "i" for a string or 32 bit integer, respectively, "as" for a string array or something longer like "(asii)" for a tuple consisting of a string array and two integers. There are a lot more possibilities, but "(asii)" is what I needed today and also what this post will be about.

So what did I want to do? I have a function that should return multiple values of different kinds, but of course in C functions can only return a single variable. There are a number of ways to solve this problem, but I am going to use a GVariant container, because that’s what GIO‘s D-Bus API needs and my function is supposed to be called via D-Bus.

Building the GVariant object

GLib has a function g_variant_new() that will construct a GVariant from an arbitrary format string and a variable number of values, but also typed functions like g_variant_new_int32() (which I prefer). The first GVariant I need is a string array. There is a typed function for this:

GStrv x;
/* fill x with data */
GVariant *x_variant = g_variant_new_strv((const gchar * const *) x, -1);
g_strfreev(x);

g_variant_new_strv() takes the array as first argument and the number of elements in the array as second. Here I used “-1”, which tells the function to expect a NULL terminated array. I don’t need the GStrv afterwards, so I free it.

Building my "(asii)" type GVariant is only slightly more complex: I will use g_variant_new_tuple() to create a tuple GVariant from an array of GVariant *.

GStrv x;
int a, b;
/* fill a, b and x with data */
GVariant **t = g_new(GVariant *, 3);
t[0] = g_variant_new_strv((const gchar * const *) x, -1);
g_strfreev(x);
t[1] = g_variant_new_int32(a);
t[2] = g_variant_new_int32(b);
GVariant *tuple_variant = g_variant_new_tuple(t, 3);
g_free(t);

Note that the array is freed, but not the component GVariants. Only the references are copied to the tuple GVariant, so freeing the components would cause trouble. g_variant_get_type_string() may be useful if you’re trying to build a complex GVariant and it doesn’t seem to have the structure you wanted.

Disassembly

I use the GVariant described above as a return value with the caller responsible for freeing the allocated memory. There are many ways to read data from GVariants (even an iterator for containers like tuples), but I’m just going to write about the method I used.

GVariant *v = my_function_call(/* parameters */);
g_assert(v != NULL);
g_assert(g_variant_n_children(v) == 3);

Ok, I get a GVariant * from my function. The first thing to check is that it isn’t NULL (you should replace this with error handling). The second assertion is to prevent mistakes: I have control over both caller and callee, but you can’t be too careful. As you might have guessed, g_variant_n_children() returns the number of elements (“children”) in a container GVariant, and that should really be three in this case. 😉 Using g_variant_get_child_value(), I get the child GVariants from the tuple. The string array is first:

GVariant *tmp = g_variant_get_child_value(v, 0);
GStrv y = g_variant_dup_strv(tmp, NULL);
g_variant_unref(tmp);

There are two functions to get a string array from a GVariant containing one: g_variant_get_strv() and g_variant_dup_strv(). I use the latter because it creates a deep copy of the array stored inside the variant object, so I can do whatever I want with the result. The GVariant is no longer needed, so it is unref‘d. I don’t think the following needs more explanation:

tmp = g_variant_get_child_value(v, 1);
gint32 c  = g_variant_get_int32(tmp);
g_variant_unref(tmp);

tmp = g_variant_get_child_value(v, 2);
gint32 d = g_variant_get_int32(tmp);
g_variant_unref(tmp);

The only thing left to do is cleaning up the container:

g_variant_unref(v);

That’s it!

First steps with libnl

My GLib based program needs to get a list of the IP addresses configured on an interface in my Linux system. I knew that the preferred way to get this kind of information is via netlink socket communication and that libnl provides an abstraction layer for that, so I had to figure out how to use the libnl API. (If you don’t know what a netlink socket is, read “Why and How to Use Netlink Socket“.) The libnl on-line API documentation is for the current development tree and does not match the version 1.1 I have installed, but that could be solved by generating the API doc from the sources. The only tutorial style example I found was “Adventures in Netlink“, which was really helpful, especially because the use case is quite similar.

Connecting to netlink

struct nl_handle *handle = nl_handle_alloc();
nl_connect(handle, NETLINK_ROUTE);
struct nl_cache *link_cache = rtnl_link_alloc_cache(handle);
struct nl_cache *addr_cache = rtnl_addr_alloc_cache(handle);

The first step is establishing the netlink connection to use. The code above allocates the socket, connects and uses the socket to fill caches for link and address data with data from the kernel. The NETLINK_ROUTE parameter to nl_connect lets connects a routing family socket, which can be used to perform many operations related to link, address and route management.

Get the relevant addresses

struct rtnl_addr *filter = rtnl_addr_alloc();
rtnl_addr_set_ifindex(filter, rtnl_link_name2i(link_cache, name));

It is normal for one interface to have multiple IPv6 addresses. While this is unusual for IPv4, it is possible. The filter structure is used to select which of these I want to read. Basically, it is an address structure in which only some fields are set, and addresses that match those fields are selected. The first thing I set is the interface. name is a char * that contains something like eth0, rtnl_link_name2i() gets the matching interface index.

GPtrArray *a = g_ptr_array_new();
g_ptr_array_set_free_func(a, g_object_unref);

GPtrArray is a dynamically sized array that contains pointers to arbitrary objects – really useful! This will be used to store the addresses.

rtnl_addr_set_family(filter, AF_INET6);
nl_cache_foreach_filter(addr_cache, (struct nl_object *) filter,
                        add_addr_to_array, a);
rtnl_addr_set_family(filter, AF_INET);
nl_cache_foreach_filter(addr_cache, (struct nl_object *) filter,
                        add_addr_to_array, a);

These are the actual filter calls. nl_cache_foreach_filter() calls the function specified as second argument for each matching address with the address and the third argument (the array) as parameters. Note that I change the filter’s address family attribute to get both IPv6 and IPv4 addresses.

The callback function

void add_addr_to_array(struct nl_object *obj, void *data)
{
    struct nl_addr *naddr =
        rtnl_addr_get_local((struct rtnl_addr *) obj);
    gint family = nl_addr_get_family(naddr);
    GInetAddress *addr = NULL;
    if (family == AF_INET6)
        addr = g_inet_address_new_from_bytes(
                nl_addr_get_binary_addr(naddr),
                G_SOCKET_FAMILY_IPV6);
    else if (family == AF_INET)
        addr = g_inet_address_new_from_bytes(
                nl_addr_get_binary_addr(naddr),
                G_SOCKET_FAMILY_IPV4);
    else
        g_assert_not_reached();
    g_ptr_array_add((GPtrArray *) data, addr);
}

This is actually quite simple:

  1. Get the netlink address object.
  2. Get the address family (only IPv6 or IPv4 because that’s what I’m filtering for).
  3. Depending on the family, create a GInetAddress with the right socket family.
  4. Add the address to the array and return.

After both calls to nl_cache_foreach_filter() are done, the array will contain GInetAddress objects for all IP addresses on the selected interface, just what I need. 🙂

Cleaning up

nl_cache_free(addr_cache);
nl_cache_free(link_cache);
rtnl_addr_put(filter);
nl_handle_destroy(handle);

Some of the libnl cleanup functions have unusual names, to I wanted to mention them here. Also remember that this simple example does no error handling, so be careful. 😉

SCTP socket in stream mode

SCTP sockets can operate in two modes: SOCK_SEQPACKET or SOCK_STREAM. When migrating my server from UDP to SCTP, I initially used the sequential packet mode. However, changing to stream mode offered me two significant advantages:

  1. I use threads to process data from different clients. With sequential packet mode, the callback for new data has to store the data and meta information in a custom structure and pass that to the handler thread. With stream mode, the callback uses g_socket_accept() and passes the resulting GSocket to the thread which will take care of the rest.

  2. Some of the target platforms do not support SCTP. With stream mode, only minimal changes are necessary to make the server use TCP. Of course, this means that the SCTP features are lost, but it is better than nothing.

The only change compared to the example code in the previous post is that the socket has to be created in stream mode:

GSocket *sock;
GError *err = NULL;
sock = g_socket_new(G_SOCKET_FAMILY_IPV6,
                    G_SOCKET_TYPE_STREAM,
                    G_SOCKET_PROTOCOL_SCTP,
                    &err);
g_assert(err == NULL);

The changes in the callback went deeper. My event callback has access to the listening socket through a data structure passed via the data pointer (see the GIOFunc definition), I’ll just assume a GSocket * for this example callback:

gboolean new_message_callback(GIOChannel *source,
                        GIOCondition condition,
                        gpointer data)
{
        GSocket *sock = data;
        g_assert(G_IS_SOCKET(sock));

        GError *err = NULL;
        GSocket *new_sock = g_socket_accept(sock,
                                        NULL,
                                        &err);
        g_assert(err == NULL);

        /* start handler thread */
        g_thread_create((GThreadFunc) handler_thread_func,
                        new_sock,
                        FALSE,
                        &err);
        g_assert(err == NULL);

        return TRUE;
}

The thread can now read from and write to the socket as usual. An additional advantage is that each thread gets its own socket: No need to lock it. In my server, the handler threads close and unref the sockets when done. You might want to keep the connections alive, if that’s the case you’ll have to handle connection end and memory freeing some other way.

Design a site like this with WordPress.com
Get started