add lttng to kernel build steps by flynneva · Pull Request #18 · ros-realtime/linux-real-time-kernel-builder

flynneva · 2021-09-01T22:21:42Z

this PR tries to add lttng tracer to the kernel build process as brought up in #16.

building the kernel now so there might be a few more commits to come to get it fully working.

im not 100% sure if all the CONFIG settings are needed but I've added them all and commented out a few that I dont think are necessary...feel free to chime in here if you think some should be uncommented.

with lttng enabled, the kernel build step (make ARCH=...) prompts the user with some questions, unfortunately I wasnt fast enough this time to copy it down but will try later on. I just stuck with the "default" options (capitol letters) like M, Y or N.

once merged this should close #16

Signed-off-by: Evan Flynn <evanflynn.msu@gmail.com>

flynneva · 2021-09-02T16:24:08Z

making this a draft as im still unsure if this actually works yet. need to test the generated kernel today

flynneva · 2021-09-02T18:20:23Z

I rebuilt it today and this time copied the prompts here:

LTTng support (LTTNG) [M/n/y/?] (NEW) y
  LTTng experimental bitwise enum support (LTTNG_EXPERIMENTAL_BITWISE_ENUM) [N/y/?] (NEW) N
  Use test plugin as trace clock (LTTNG_CLOCK_PLUGIN_TEST) [M/n/y/?] (NEW) y

these prompts have the same names as the ones I added to the .config-fragment file...which makes me worried that they are not being applied properly.

LanderU · 2021-09-03T10:14:15Z

@flynneva, you can check if they are active in your .config file

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

flynneva · 2021-09-07T17:29:14Z

almost there i think. no user input other than following the README commands as already documented.

looks like lttng is being built into the kernel during the make command:

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

flynneva · 2021-09-07T19:46:43Z

I've confirmed the output deb files work as expected and integrate the lttng-modules package right in with the kernel.

when following the ros2_tracing installation instructions just leave out the installation of lttng-dkms-modules as it is already built in.

flynneva · 2021-09-07T19:52:59Z

output of the ros2_tracing launch example:

ros2 launch tracetools_launch example.launch.py 
[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2021-09-07-19-47-06-778068-ubuntu-2086
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [tracetools_launch.action]: Writing tracing session to: /home/ubuntu/.ros/tracing/my-tracing-session
[INFO] [test_ping-1]: process started with pid [2113]
[INFO] [test_pong-2]: process started with pid [2115]
[test_pong-2] [INFO] [1631044029.972198136] [test_pong]: [output] some random ping string
[test_ping-1] [INFO] [1631044029.973497659] [test_ping]: [output] some random pong string
[test_ping-1] spinning
[test_pong-2] spinning
[INFO] [test_pong-2]: process has finished cleanly [pid 2115]
[INFO] [test_ping-1]: process has finished cleanly [pid 2113]

`cat metadata` from tracing logs

$ cat metadata 
typealias integer { size = 8; align = 8; signed = false; } := uint8_t;
typealias integer { size = 16; align = 8; signed = false; } := uint16_t;
typealias integer { size = 32; align = 8; signed = false; } := uint32_t;
typealias integer { size = 64; align = 8; signed = false; } := uint64_t;
typealias integer { size = 64; align = 8; signed = false; } := unsigned long;
typealias integer { size = 5; align = 1; signed = false; } := uint5_t;
typealias integer { size = 27; align = 1; signed = false; } := uint27_t;

trace {
	major = 1;
	minor = 8;
	uuid = "76f996dc-d9c7-f144-bb4d-f90ca898cbe7";
	byte_order = le;
	packet.header := struct {
		uint32_t magic;
		uint8_t  uuid[16];
		uint32_t stream_id;
		uint64_t stream_instance_id;
	};
};

env {
	hostname = "ubuntu";
	domain = "kernel";
	sysname = "Linux";
	kernel_release = "5.4.128-rt61";
	kernel_version = "#1 SMP PREEMPT_RT Tue Sep 7 17:24:07 UTC 2021";
	tracer_name = "lttng-modules";
	tracer_major = 2;
	tracer_minor = 12;
	tracer_patchlevel = 5;
	trace_buffering_scheme = "global";
	trace_name = "my-tracing-session";
	trace_creation_datetime = "20210907T194707+0000";
};

clock {
	name = "lttng_test_clock_override";
	uuid = "83c63deb-7aa4-48fb-abda-946f400d76e6";
	description = "Freeze time with 1KHz for regression test";
	freq = 1000; /* Frequency, in Hz */
	/* clock value offset from Epoch is: offset * (1/freq) */
	offset = 1631044028667;
};

typealias integer {
	size = 27; align = 1; signed = false;
	map = clock.lttng_test_clock_override.value;
} := uint27_clock_monotonic_t;

typealias integer {
	size = 32; align = 8; signed = false;
	map = clock.lttng_test_clock_override.value;
} := uint32_clock_monotonic_t;

typealias integer {
	size = 64; align = 8; signed = false;
	map = clock.lttng_test_clock_override.value;
} := uint64_clock_monotonic_t;

struct packet_context {
	uint64_clock_monotonic_t timestamp_begin;
	uint64_clock_monotonic_t timestamp_end;
	uint64_t content_size;
	uint64_t packet_size;
	uint64_t packet_seq_num;
	unsigned long events_discarded;
	uint32_t cpu_id;
};

struct event_header_compact {
	enum : uint5_t { compact = 0 ... 30, extended = 31 } id;
	variant <id> {
		struct {
			uint27_clock_monotonic_t timestamp;
		} compact;
		struct {
			uint32_t id;
			uint64_clock_monotonic_t timestamp;
		} extended;
	} v;
} align(8);

struct event_header_large {
	enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id;
	variant <id> {
		struct {
			uint32_clock_monotonic_t timestamp;
		} compact;
		struct {
			uint32_t id;
			uint64_clock_monotonic_t timestamp;
		} extended;
	} v;
} align(8);

stream {
	id = 0;
	event.header := struct event_header_compact;
	packet.context := struct packet_context;
};

event {
	name = "power_cpu_frequency";
	id = 3;
	stream_id = 0;
	fields := struct {
		integer { size = 32; align = 8; signed = 0; encoding = none; base = 10; } _state;
		integer { size = 32; align = 8; signed = 0; encoding = none; base = 10; } _cpu_id;
	};
};

event {
	name = "kmem_mm_page_free";
	id = 2;
	stream_id = 0;
	fields := struct {
		integer { size = 64; align = 8; signed = 0; encoding = none; base = 16; } _page;
		integer { size = 64; align = 8; signed = 0; encoding = none; base = 10; } _pfn;
		integer { size = 32; align = 8; signed = 0; encoding = none; base = 10; } _order;
	};
};

event {
	name = "kmem_mm_page_alloc";
	id = 1;
	stream_id = 0;
	fields := struct {
		integer { size = 64; align = 8; signed = 0; encoding = none; base = 16; } _page;
		integer { size = 64; align = 8; signed = 0; encoding = none; base = 10; } _pfn;
		integer { size = 32; align = 8; signed = 0; encoding = none; base = 10; } _order;
		integer { size = 32; align = 8; signed = 0; encoding = none; base = 10; } _gfp_flags;
		integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _migratetype;
	};
};

event {
	name = "sched_switch";
	id = 0;
	stream_id = 0;
	fields := struct {
		integer { size = 8; align = 8; signed = 0; encoding = UTF8; base = 10; } _prev_comm[16];
		integer { size = 32; align = 8; signed = 1; encoding = none; base = 10W�uv������D�M�
                                                                                                    ������; } _prev_tid;
		integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _prev_prio;
		enum : integer { size = 64; align = 8; signed = 1; encoding = none; base = 10; } {
			"TASK_RUNNING" = 0,
			"TASK_INTERRUPTIBLE" = 1,
			"TASK_UNINTERRUPTIBLE" = 2,
			"TASK_STOPPED" = 4,
			"TASK_TRACED" = 8,
			"EXIT_DEAD" = 16,
			"EXIT_ZOMBIE" = 32,
			"TASK_PARKED" = 64,
			"TASK_DEAD" = 128,
			"TASK_WAKEKILL" = 256,
			"TASK_WAKING" = 512,
			"TASK_NOLOAD" = 1024,
			"TASK_NEW" = 2048,
			"TASK_STATE_MAX" = 4096,
		} _prev_state;
		integer { size = 8; align = 8; signed = 0; encoding = UTF8; base = 10; } _next_comm[16];
		integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _next_tid;
		integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _next_prio;
	};
};

razr

Does it make sense to add a few words in the README describing why do we need to add lttng support?

razr · 2021-09-07T21:34:52Z

Dockerfile

+# download lttng source for use later
+# TODO(flynneva): make script to auto-determine which version to get?
+RUN cd $HOME \
+  && wget https://lttng.org/files/lttng-modules/lttng-modules-${LTTNG}.tar.bz2 \


it is not better to get sources from Ubuntu directly, like

$ sudo apt-add-repository ppa:lttng/stable-2.12 $ sudo apt-get update $ apt-get source lttng-modules-dkms

they are patched afaics.

@razr sure that sounds great, I was only following the build instructions from the lttng-module website

I don't think it really changes anything in practice, but ROS 2 (well, ros2_tracing) currently targets the versions of LTTng available from the usual Ubuntu packages. So if we want to stick to that, we shouldn't use the PPA here. You can still call apt-get source without the PPA of course.

Dockerfile

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

flynneva · 2021-09-09T17:30:50Z

@razr i implemented your recommendations and am building the kernel again now just to confirm everything works as expected. ill follow up here once its done building and tested.

flynneva · 2021-09-10T18:22:59Z

would probably be good to get @christophebedard input / review here as well

README.md

razr

@flynneva thanks much, it looks good

Co-authored-by: Christophe Bedard <bedard.christophe@gmail.com>

LanderU

LGTM!

flynneva added 2 commits September 1, 2021 12:48

add lttng to kernel build

8ef7fe3

Signed-off-by: Evan Flynn <evanflynn.msu@gmail.com>

fix kernel src path for lttng script

511ac2f

Signed-off-by: Evan Flynn <evanflynn.msu@gmail.com>

flynneva mentioned this pull request Sep 1, 2021

unable to install lttng on rt linux kernel #16

Closed

LanderU requested review from LanderU and razr September 2, 2021 11:51

flynneva marked this pull request as draft September 2, 2021 16:23

update config fragment, use arg for lttng version

cb93bf6

reorder dockerfile, update config fragment settings for lttng

7b47e89

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

specify entire LTTNG version via ARG

94f737b

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

flynneva marked this pull request as ready for review September 7, 2021 19:45

razr reviewed Sep 7, 2021

View reviewed changes

use ubuntu ppa source for lttng, add lttng section to readme

7454731

Signed-off-by: Evan Flynn <evan.flynn@apex.ai>

flynneva requested a review from razr September 9, 2021 17:29

Merge branch 'master' into feature/lttng

f326aa4

christophebedard reviewed Sep 10, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

Update README.md

0c4436b

razr approved these changes Sep 10, 2021

View reviewed changes

Update README.md LTTng section

2539890

Co-authored-by: Christophe Bedard <bedard.christophe@gmail.com>

flynneva requested a review from christophebedard September 10, 2021 20:38

christophebedard approved these changes Sep 12, 2021

View reviewed changes

LanderU approved these changes Sep 12, 2021

View reviewed changes

razr merged commit 4e20866 into ros-realtime:master Sep 12, 2021

Conversation

flynneva commented Sep 1, 2021

Uh oh!

flynneva commented Sep 2, 2021

Uh oh!

flynneva commented Sep 2, 2021

Uh oh!

LanderU commented Sep 3, 2021

Uh oh!

flynneva commented Sep 7, 2021

Uh oh!

flynneva commented Sep 7, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flynneva commented Sep 7, 2021

Uh oh!

razr left a comment

Choose a reason for hiding this comment

Uh oh!

razr Sep 7, 2021

Choose a reason for hiding this comment

Uh oh!

flynneva Sep 7, 2021

Choose a reason for hiding this comment

Uh oh!

christophebedard Sep 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

flynneva commented Sep 9, 2021

Uh oh!

flynneva commented Sep 10, 2021

Uh oh!

Uh oh!

razr left a comment

Choose a reason for hiding this comment

Uh oh!

LanderU left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

flynneva commented Sep 7, 2021 •

edited

Loading

christophebedard Sep 10, 2021 •

edited

Loading