Skip to content

[WIP] journal: support match negation#12592

Open
mrc0mmand wants to merge 1 commit intosystemd:mainfrom
mrc0mmand:journal-introduce-match-negation
Open

[WIP] journal: support match negation#12592
mrc0mmand wants to merge 1 commit intosystemd:mainfrom
mrc0mmand:journal-introduce-match-negation

Conversation

@mrc0mmand
Copy link
Copy Markdown
Member

@mrc0mmand mrc0mmand commented May 16, 2019 •
edited
Loading

To get myself familiar further with the systemd codebase (and to fix an issue which has been nagging me for a while), I implemented (or at least tried to) a fix for #2720.

First of all, this is just a (working, so far) mockup, as I'd like to get a feedback if this is even the right approach.

To avoid collisions with bash operators and such (i.e. using ! or - for negation) I chose ~ as a prefix for a field user wants to NOT match.

For example:
journalctl -b ~MESSAGE="this is sparta"
shows all messages for this boot which are not "this is sparta".

$ build/journalctl --no-pager --file system.journal --since "2019-01-14 14:21:13" | tail -n 4
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty2.
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty5.
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty3.
Jan 14 14:21:13 pyrelight kernel: audit: type=1131 audit(1547472073.653:1033): pid=8784 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='unit=getty@tty3 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

$ build/journalctl --no-pager --file system.journal --since "2019-01-14 14:21:13" MESSAGE="Stopped Getty on tty3." | tail -n 4
-- Logs begin at Mon 2019-01-14 14:15:24 CET, end at Mon 2019-01-14 14:21:13 CET. --
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty3.

$ build/journalctl --no-pager --file system.journal --since "2019-01-14 14:21:13" ~MESSAGE="Stopped Getty on tty3." | tail -n 4
Jan 14 14:21:13 pyrelight kernel: audit: type=1131 audit(1547472073.653:1032): pid=8784 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='unit=getty@tty5 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty2.
Jan 14 14:21:13 pyrelight systemd[1]: Stopped Getty on tty5.
Jan 14 14:21:13 pyrelight kernel: audit: type=1131 audit(1547472073.653:1033): pid=8784 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:spc_t:s0 msg='unit=getty@tty3 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

TODO: tests

@mrc0mmand mrc0mmand force-pushed the journal-introduce-match-negation branch 3 times, most recently from 7e49208 to bad6229 Compare May 16, 2019 20:04
@mrc0mmand mrc0mmand changed the title journal: support match negation [WIP] journal: support match negation May 17, 2019
@mrc0mmand
journal: support match negation
48010ef 
As a pretty common use case is to filter a particular service from the
journactl output, let's add a NOT "operator" to the matching machinery.

TODO: description, examples
@mrc0mmand mrc0mmand force-pushed the journal-introduce-match-negation branch from bad6229 to 48010ef Compare January 10, 2020 19:31
dnicolodi
dnicolodi reviewed Jan 10, 2020
View reviewed changes
Copy link
Copy Markdown

@dnicolodi dnicolodi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ~ prefix is also not optimal because it expands to the home directory of the user whose user name is the string that follows. Thus your proposed syntax works without quoting only as long as there are not clashes between user names and journal field names.

@mrc0mmand
Copy link
Copy Markdown
Member Author

mrc0mmand commented Jan 10, 2020

The ~ prefix is also not optimal because it expands to the home directory of the user whose user name is the string that follows. Thus your proposed syntax works without quoting only as long as there are not clashes between user names and journal field names.

Hah, that completely slipped my mind, thank you. I need to take a closer look at this, especially after letting this PR lay dormant for more than a half an year.

@keszybz
Copy link
Copy Markdown
Member

keszybz commented Jul 24, 2020

@mrc0mmand update?

@poettering
Copy link
Copy Markdown
Member

poettering commented Aug 18, 2020

So, the journal file format actionally contains a bisection table (i.e. a strictly ordered array) to find all entry records for a specific data record. i.e. for each assignment as in FOO=BAR (i.e. each "data record") we actually have a list of entry records by their offset that contain them. We can use this to speed things up a bit, i.e. we don't actually have to go to an entry record to check if it lacks a specific data record. We instead can go to just go to the data record that shall not match, and then just check if the offset of the next potentially matching entry is in there. or in other words:

figure out the offset of the next potential entry, check it against any of the negation data record entry arrays, and skip them immediately if they appear there.

@poettering poettering added the reviewed/needs-rework 🔨 PR has been reviewed and needs another round of reworks label Aug 18, 2020
Base automatically changed from master to main January 21, 2021 11:54
@YHNdnzj YHNdnzj mentioned this pull request Jan 3, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@dnicolodi dnicolodi dnicolodi left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

journal reviewed/needs-rework 🔨 PR has been reviewed and needs another round of reworks

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@mrc0mmand @keszybz @poettering @dnicolodi