Contributors: Manuel López-Ibáñez, Carlos M. Fonseca, Luís Paquete, Andreia P. Guerreiro.
This is the C library and associated command-line tools of the moocore project. This README describes just this component. There is also a moocore Python package and a moocore R package.
The moocore project collects fast implementations of core mathematical functions and algorithms for multi-objective optimization and makes them available to different programming languages via similar interfaces. These functions include:
- Identifying and filtering dominated vectors.
- Quality metrics such as (weighted) hypervolume, epsilon, IGD, etc.
- Computation of the Empirical Attainment Function. The empirical attainment function (EAF) describes the probabilistic distribution of the outcomes obtained by a stochastic algorithm in the objective space.
The goal of moocore is to provide an implementation of these functions that is efficient, thoroughly tested, well-documented, multi-platform (Windows, Linux, MacOS), multi-language (C, R, and Python, so far), with minimal dependencies and reusable from other libraries and packages, thus providing an easy-to-use and high-performance building block.
Most critical functionality is implemented in C, with the R and Python packages providing convenient interfaces to the C code.
Keywords: empirical attainment function, summary attainment surfaces, EAF differences, multi-objective optimization, bi-objective optimization, performance measures, performance assessment
We recommend that you compile specifically for your architecture using GCC option -march=. The default compilation is done with:
make all
This uses the option "-march=native". If your GCC version does not support "native", you can give an explicit architecture, e.g.,
make all MARCH=x86-64-v2
See the GCC manual for the names of the architectures supported by your version of GCC.
You can compile the code with various runtime checks enabled using the option DEBUG=1. This will slow down the code significantly. If this is too slow for your purposes, you can disable the sanitizers using make all DEBUG=1 SANITIZERS="".
The build system will try to pick good compiler flags for you, but if you need to, you can override them by passing the option OPT_CFLAGS. For example, to disable compiler optimization and enabled debug symbols, you could run:
make all OPT_CFLAGS="-O0 -g"
If you do not want to see the command line of each compiler
invocation, also pass S=1 to make.
Most functions are implemented in headers such as nondominated.h. For other functions, it is possible to build a shared library for embedding the code into your own C/C++ programs:
make shlibs
Functions exported by the library are marked with MOOCORE_API.
The moocore executables are validated using a comprehensive testsuite. Running the testsuite requires Python >= 3.10 and additional Python packages (see testsuite/requirements.txt). To run the testsuite yourself, follow these steps:
# Download moocore
git clone https://github.com/multi-objective/moocore moocore
# Download the testsuite
git clone https://github.com/multi-objective/testsuite moocore/testsuite
# Install required python packages
python3 -m pip install -r testsuite/requirements.txt
# Testing
make -C moocore/c/ test
# Timing
make -C moocore/c/ timeUnder moocore/c/, make test will compile the code with DEBUG=1 SANTIZERS="" OPT_CFLAGS="-Og -g3". This is useful for debugging failures.
make time will compile the code with DEBUG=0 SANITIZERS="" OPT_CLAGS="-O3 -ftlo -march=native". This is useful for benchmarking code and making sure compiler optimizations do not break anything.
Neither make test nor make time recompile the code that has not been modified. So you can compile the code first with your preferred compiler flags and then run the testsuite. For example, if you compile with make all DEBUG=1, the code will be compiler with several sanitizers enabled. You can then call make test to run the testsuite with the sanitizers enabled. This is much slower, but very helpful to track crashes.
Under c/ there are various targets to run only parts of the testsuite, for example:
make test-hv
make test-nondominated
...
All command-line tools can read the input as a filename in the command line:
hv datafile
or from standard input:
cat datafile | hv
In an input file, each point is given in a separate line, and each coordinate
within a line is separated by whitespace. An empty line denotes a separate
set. Most tools have an --union option that ignores those empty lines. All
tools assume that all objectives must be minimized but this can be changed via
options --maximise or --obj=. Also, maximization objectives may be
multiplied by -1 to convert them to minimization (see the option --agree of nondominated).
Check dominance, filter and transform the sets given as input.
Usage:
nondominated [OPTIONS] [FILES]
nondominated [OPTIONS] < [INPUT] > [OUTPUT]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some extra information;
-q, --quiet print as little as possible;
--no-check do not check nondominance of sets (faster but unsafe);
-o, --obj=[+|-]... specify whether each objective should be minimised (-)
or maximised (+). By default all are minimised;
--maximise all objectives must be maximised;
-u, --upper-bound POINT defines an upper bound to check, e.g. "10 5 30";
-l, --lower-bound POINT defines a lower bound to check;
-U, --union consider each file as a whole approximation set,
(by default, approximation sets are separated by an
empty line within a file);
-s, --suffix=STRING suffix to add to output files. Default is "_dat".
The empty string means overwrite the input file.
This is ignored when reading from stdin because output
is sent to stdout.
The following options OVERWRITE output files:
-a, --agree=<max|min> transform objectives so all are maximised (or
minimised). See also the option --obj.
-f, --filter check and filter out dominated points;
-b, --force-bound remove points that do not satisfy the bounds;
-n, --normalise RANGE normalise all objectives to a range, e.g., "1 2".
If bounds are given with -l and -u, they are used
for the normalisation.
-L, --log=[1|0]... specify whether each objective should be transformed
to logarithmic scale (1) or not (0).
Computes the exact hypervolume. A reference point can be given by the option -r.
hv -r "10 10 10" data
If no reference point is given, the default is maximum + 0.1 * (maximum - minimum), where the maximum and minimum values are calculated for each
coordinate from the union of all input points.
For other options, check the output of hv --help.
Approximate the hypervolume value of each input set of each input file. The
approximation uses (quasi-)Monte-Carlo sampling, thus gets more accurate with
larger values of --nsamples. With no file, or when file is -, read standard
input.
Usage: hvapprox [OPTIONS] [FILE...]
hvapprox [OPTIONS] < [INPUT] > [OUTPUT]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, maximum, etc).
-q, --quiet print just the hypervolume (as opposed to --verbose).
-u, --union treat all input sets within a FILE as a single set.
-r, --reference=POINT use POINT as reference point. POINT must be within
quotes, e.g., "10 10 10". If no reference point is
given, it is taken as max + 0.1 * (max - min) for each
coordinate from the union of all input points.
-s, --suffix=STRING Create an output file for each input file by appending
this suffix. This is ignored when reading from stdin.
If missing, output is sent to stdout.
-n, --nsamples=N Number of Monte-Carlo samples (N is a positive integer).
-m, --method=M 1: Monte-Carlo sampling using normal distribution;
2: Hua-Wang deterministic sampling (default).
-S, --seed=SEED Seed of the random number generator (positive integer).
Only method=1.
Calculates the epsilon measure for the sets given as input.
Usage:
epsilon [OPTIONS] [FILES]
epsilon [OPTIONS] < [INPUT] > [OUTPUT]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, number of points, etc.).
-q, --quiet print as little as possible;
-a, --additive epsilon additive value (default).
-m, --multiplicative epsilon multiplicative value .
-r, --reference FILE file that contains the reference set
-o, --obj=[+|-]... specify whether each objective should be minimised (-)
or maximised (+). By default all are minimised;
--maximise all objectives must be maximised;
-s, --suffix=STRING Create an output file for each input file by appending
this suffix. This is ignored when reading from stdin.
If missing, output is sent to stdout.
Calculates quality metrics related to the generational distance (GD, IGD, IGD+, avg Hausdorff distance).
Usage:
igd [OPTIONS] [FILES]
igd [OPTIONS] < [INPUT] > [OUTPUT]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, number of points, etc.)
-q, --quiet print as little as possible;
, --gd report classical GD
, --igd report classical IGD
, --gd-p report GD_p (p=1 by default)
, --igd-p (default) report IGD_p (p=1 by default)
, --igd-plus report IGD+
, --hausdorff report avg Hausdorff distance = max (GD_p, IGD_p)
-a, --all compute everything
-p, exponent that averages the distances
-r, --reference FILE file that contains the reference set
-o, --obj=[+|-]... specify whether each objective should be minimised (-)
or maximised (+). By default all are minimised;
--maximise all objectives must be maximised;
-s, --suffix=STRING Create an output file for each input file by appending
this suffix. This is ignored when reading from stdin.
If missing, output is sent to stdout.
Computes the empirical attainment function (EAF) of all input files.
With no file, or when file is -, read standard input.
Usage: eaf [OPTIONS] [FILE...]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, input points, output
points, etc) in stderr. Default is --quiet
-o, --output FILE write output to FILE instead of standard output.
-q, --quiet print just the EAF (as opposed to --verbose)
-b, --best compute best attainment surface
-m, --median compute median attainment surface
-w, --worst compute worst attainment surface
-p, --percentile REAL compute the given percentile of the EAF
-l, --level LEVEL compute the given level of the EAF
-i[FILE], --indices[=FILE] write attainment indices to FILE.
If FILE is '-', print to stdout.
If FILE is missing use the same file as for output.
-d[FILE], --diff[=FILE] write difference between half of runs to FILE.
If FILE is '-', print to stdout.
If FILE is missing use the same file as for output.
, --polygons Write EAF as R polygons.
Calculates the number of sets from one file that dominate the sets of the other files.
Usage: dominatedsets [OPTIONS] [FILE...]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, number of points, etc.)
-q, --quiet print as little as possible;
-p, --percentages print results also as percentages.
--no-check do not check nondominance of sets (faster but unsafe).
-o, --obj=[+|-]... specify whether each objective should be minimised (-)
or maximised (+). By default all are minimised;
Perform nondominated sorting in a list of points.
Usage: ndsort [OPTIONS] [FILE...]
Options:
-h, --help print this summary and exit;
--version print version number (and compilation flags) and exit;
-v, --verbose print some information (time, number of points, etc.)
-q, --quiet print as little as possible;
-k, --keep-uevs keep uniquely extreme values
-r, --rank don't break ties using hypervolume contribution
-o, --obj=[+|-]... specify whether each objective should be minimised (-)
or maximised (+). By default all are minimised;
See the LICENSE and COPYRIGHTS files.