I use the tail -f <file> | grep <pattern> all the time.

It will wait till grep flushes, not till it finishes (I'm using Ubuntu).

I think that your problem is that grep uses some output buffering. Try

tail -f file | stdbuf -o0 grep my_pattern

it will set output buffering mode of grep to unbuffered.

If you want to find matches in the entire file (not just the tail), and you want it to sit and wait for any new matches, this works nicely:

tail -c +0 -f <file> | grep --line-buffered <pattern>

The -c +0 flag says that the output should start 0 bytes (-c) from the beginning (+) of the file.

In most cases, you can tail -f /var/log/some.log |grep foo and it will work just fine.

If you need to use multiple greps on a running log file and you find that you get no output, you may need to stick the --line-buffered switch into your middle grep(s), like so:

tail -f /var/log/some.log | grep --line-buffered foo | grep bar

you may consider this answer as enhancement .. usually I am using

tail -F <fileName> | grep --line-buffered  <pattern> -A 3 -B 5

-F is better in case of file rotate (-f will not work properly if file rotated)

-A and -B is useful to get lines just before and after the pattern occurrence .. these blocks will appeared between dashed line separators

But For me I prefer doing the following

tail -F <file> | less

this is very useful if you want to search inside streamed logs. I mean go back and forward and look deeply

Didn't see anyone offer my usual go-to for this:

less +F <file>
ctrl + c
/<search term>
<enter>
shift + f

I prefer this, because you can use ctrl + c to stop and navigate through the file whenever, and then just hit shift + f to return to the live, streaming search.

Coming some late on this question, considering this kind of work as an important part of monitoring job, here is my (not so short) answer...

Following logs using bash

1. Command tail

This command is a little more porewfull than read on already published answer

1. Difference between follow option tail -f and tail -F, from manpage:

      -f, --follow[={name|descriptor}]
             output appended data as the file grows;
   ...
      -F     same as --follow=name --retry
   ...
      --retry
             keep trying to open a file if it is inaccessible
   

   This mean: by using -F instead of -f, tail will re-open file(s) when removed (on log rotation, for sample).
   This is usefull for watching logfile over many days.

2. Ability of following more than one file simultaneously
   I've already used:

   tail -F /var/www/clients/client*/web*/log/{error,access}.log /var/log/{mail,auth}.log \
              /var/log/apache2/{,ssl_,other_vhosts_}access.log \
              /var/log/pure-ftpd/transfer.log
   

   For following events through hundreds of files... (consider rest of this answer to understand how to make it readable... ;)

3. Using switches -n (Don't use -c for line buffering!).
   By default tail will show 10 last lines. This can be tunned:

   tail -n 0 -F file
   

   Will follow file, but only new lines will be printed

   tail -n +0 -F file
   

   Will print whole file before following his progression.

2. Buffer issues when piping:

If you plan to filter ouptuts, consider buffering! See -u option for sed, --line-buffered option for grep, or stdbuf command:

tail -F /some/files | sed -une '/Regular Expression/p'

Is (a lot more efficient than using grep) a lot more reactive than if you does'nt use -u switch in sed command.

tail -F /some/files |
    sed -une '/Regular Expression/p' |
    stdbuf -i0 -o0 tee /some/resultfile

3. Recent journaling system

On recent system, instead of tail -f /var/log/syslog you have to run journalctl -xf, in near same way...

journalctl -axf | sed -une '/Regular Expression/p'

But read man page, this tool was built for log analyses: man journalctl said:

  -g, --grep=
      Filter output to entries where the MESSAGE= field matches the
      specified regular expression. PERL-compatible regular expressions
      are used, see pcre2pattern(3) for a detailed description of the
      syntax.

      If the pattern is all lowercase, matching is case insensitive.
      Otherwise, matching is case sensitive. This can be overridden with
      the --case-sensitive option, see below.

      When used with --lines=, --reverse is implied.

So for this, you could:

journalctl -axfg 'Regular Expression'

4. Integrating this in a bash script

1. Colored output of two files (or more)

   Here is a sample of script watching for many files, coloring ouptut differently for 1st file than others:

   #!/bin/bash
   
   tail -F "$@" |
       sed -une "
           /^==> /{h;};
           //!{
               G;
               s/^\\(.*\\)\\n==>.*${1//\//\\\/}.*<==/\\o33[47m\\1\\o33[0m/;
               s/^\\(.*\\)\\n==> .* <==/\\o33[47;31m\\1\\o33[0m/;
               p;}"
   

   They work fine on my host, running:

   sudo ./myColoredTail /var/log/{kern.,sys}log
   

2. Interactive script

   You may be watching logs for reacting on events?

   Here is a little script playing some sound when some USB device appear or disappear, but same script could send mail, or any other interaction, like powering on coffe machine...

   #!/bin/bash
   
   exec {tailF}< <(tail -F /var/log/kern.log)
   tailPid=$!
   
   while :;do
       read -rsn 1 -t .3 keyboard
       [ "${keyboard,}" = "q" ] && break
       if read -ru $tailF -t 0 _ ;then
           read -ru $tailF line
           case $line in
               *New\ USB\ device\ found* ) play /some/sound.ogg ;;
               *USB\ disconnect* ) play /some/othersound.ogg ;;
           esac
           printf "\r%s\e[K" "$line"
       fi
   done
   
   echo
   exec {tailF}<&-
   kill $tailPid
   

   You could quit by pressing Q key.

   Note: To use this with journalctl, you just have to replace line 3 by something like:

   exec {tailF}< <(journalctl -axfu someUnit -g 'Some pattern')
   

sed would be a better choice (stream editor)

tail -n0 -f <file> | sed -n '/search string/p'

and then if you wanted the tail command to exit once you found a particular string:

tail --pid=$(($BASHPID+1)) -n0 -f <file> | sed -n '/search string/{p; q}'

Obviously a bashism: $BASHPID will be the process id of the tail command. The sed command is next after tail in the pipe, so the sed process id will be $BASHPID+1.

Yes, this will actually work just fine. Grep and most Unix commands operate on streams one line at a time. Each line that comes out of tail will be analyzed and passed on if it matches.

This one command workes for me (Suse):

mail-srv:/var/log # tail -f /var/log/mail.info |grep --line-buffered LOGIN  >> logins_to_mail

collecting logins to mail service

you certainly won't succeed with

tail -f /var/log/foo.log |grep --line-buffered string2search

when you use "colortail" as an alias for tail, eg. in bash

alias tail='colortail -n 30'

you can check by type alias if this outputs something like tail isan alias of colortail -n 30. then you have your culprit :)

Solution:

remove the alias with

unalias tail

ensure that you're using the 'real' tail binary by this command

type tail

which should output something like:

tail is /usr/bin/tail

and then you can run your command

tail -f foo.log |grep --line-buffered something

Good luck.