Skip to content

Partial read triggers an event in Shadow, but not Linux #3274

@haxxpop

Description

@haxxpop

Describe the issue
When using epoll with EPOLLET flag set and the file under epoll is partially read (some is left in the buffer), the result of epoll_wait in the real Linux is different from the one in Shadow. This issue is a different one from #2673

To Reproduce
Add the following test to src/test/epoll/test_epoll.rs and run the test.

diff --git a/src/test/epoll/test_epoll.rs b/src/test/epoll/test_epoll.rs
index df0e67db2..f6b9849ce 100644
--- a/src/test/epoll/test_epoll.rs
+++ b/src/test/epoll/test_epoll.rs
@@ -4,6 +4,7 @@ use nix::errno::Errno;
 use nix::sys::epoll::{self, EpollFlags};
 use nix::unistd;
 
+use test_utils::socket_utils::{socket_init_helper, SocketInitMethod};
 use test_utils::{ensure_ord, set, ShadowTest, TestEnvironment};
 
 #[derive(Debug)]
@@ -329,6 +330,59 @@ fn test_ctl_invalid_op() -> anyhow::Result<()> {
     })
 }
 
+fn test_write_then_read() -> anyhow::Result<()> {
+    let (fd_client, fd_server) = socket_init_helper(
+        SocketInitMethod::Inet,
+        libc::SOCK_STREAM,
+        libc::SOCK_NONBLOCK,
+        /* bind_client = */ false,
+    );
+    let epollfd = epoll::epoll_create()?;
+
+    test_utils::run_and_close_fds(&[epollfd, fd_client, fd_server], || {
+        let mut event = epoll::EpollEvent::new(EpollFlags::EPOLLET | EpollFlags::EPOLLIN, 0);
+        epoll::epoll_ctl(
+            epollfd,
+            epoll::EpollOp::EpollCtlAdd,
+            fd_server,
+            Some(&mut event),
+        )?;
+
+        let timeout = Duration::from_millis(100);
+
+        let thread = std::thread::spawn(move || {
+            vec![
+                do_epoll_wait(epollfd, timeout, /* do_read= */ false),
+                // The second one is supposed to timeout.
+                do_epoll_wait(epollfd, timeout, /* do_read= */ false),
+            ]
+        });
+
+        // Wait for readers to block.
+        std::thread::sleep(timeout / 3);
+
+        // Make the read-end readable.
+        unistd::write(fd_client, &[0, 0])?;
+
+        // Wait and read some, but not all, from the buffer.
+        std::thread::sleep(timeout / 3);
+        unistd::read(fd_server, &mut [0])?;
+
+        let results = thread.join().unwrap();
+
+        // The first wait should have received the event
+        ensure_ord!(results[0].epoll_res, ==, Ok(1));
+        ensure_ord!(results[0].duration, <, timeout);
+        ensure_ord!(results[0].events[0], ==, epoll::EpollEvent::new(EpollFlags::EPOLLIN, 0));
+
+        // The second wait should have timed out with no events received.
+        ensure_ord!(results[1].epoll_res, ==, Ok(0));
+        ensure_ord!(results[1].duration, >=, timeout);
+
+        Ok(())
+    })
+}
+
 fn main() -> anyhow::Result<()> {
     // should we restrict the tests we run?
     let filter_shadow_passing = std::env::args().any(|x| x == "--shadow-passing");
@@ -340,6 +394,7 @@ fn main() -> anyhow::Result<()> {
     let mut tests: Vec<test_utils::ShadowTest<(), anyhow::Error>> = vec![
         ShadowTest::new("threads-edge", test_threads_edge, all_envs.clone()),
         ShadowTest::new("threads-level", test_threads_level, all_envs.clone()),
+        ShadowTest::new("write-then-read", test_write_then_read, all_envs.clone()),
         // in Linux these two tests have a race condition and don't always pass
         ShadowTest::new(
             "threads-level-with-late-read",

In Shadow, the second epoll_wait receives an event, but, in Linux, it doesn't.

Operating System (please complete the following information):

  • OS and version: Ubuntu 22.04.3 LTS
  • Kernel version: Linux thinkpad-t14 6.2.0-39-generic # 40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

Shadow (please complete the following information):

  • Version: commit c133925
  • Which processes you are trying to run inside the Shadow simulation: the tests

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: BugError or flaw producing unexpected results

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions