Hunting the Copy Fail Exploit With Detection Lattices

So the Copy Fail exploit (https://copy.fail/) is the hunt of the week. If you’re one of the few who have not heard, it is a Linux privilege escalation vuln, with a working exploit, that advertises elevation to root on any Linux distro from the last nine years which would include most fleets. I can’t confirm that but I can confirm it works very reliably on the Ubuntu instances I have seen it on. I know there is one school of thought that local privesc exploits are of lesser importance because they require starting from an existing shell. Many of the flags I have seen taken were the result of combining low-privilege remote code execution, or command injection, with local privilege escalations. These threat models are usually not visible from either a vulnerability management tool, or an appsec tool, because neither of those usually show users this kind of attack path, because neither one knows much about the other. So this one is worth hunting or at least threat modeling; the odds are this attack path exists somewhere in your environment.

This is an excellent case study for the PROTOSTAR project because, as we are seeing, there isn’t really a good conventional alert rule that can be written for this, there are simply too many moving parts and shades of grey. None of the events involved are clearly and distinctly bad enough to fire an important alert. It does, however, produce an alpha detection signal set. Here is what it looks like in what we call the tactical view which uses a table structure for packaging lattices as questions sent to an AI agent along with the appropriate prompts and knowledge:

The exploit creates what we call an alpha signal with a perfectly matched set of conventional and machine learning detections. As you can see, there are three types of regular alerts, for execution and privilege escalation, and two machine learning detections. These cannot be joined by a single process ID or username because there are at least two of each involved. What’s more important than any of those conventional joins or correlations is the fact that the conventional and machine learning detections fired on the same events. The “detection lattice” element was created when the PROTOSTAR engine noticed that pattern.

This is what a Copy Fail detection element set looks like in what we call the visual view:

This is the detail page, in the visual view, for a Copy Fail detection element. The trinary node structure is created by the relationships between the detection elements in the set. In the northern quadrant, there are two different machine learning detections for the Python activity and some of the syscalls used by this exploit payload – both are unusual, but as most of us know, many ML anomaly detections don’t have stronger signal than the conventional alerts they try to replace. There are simply too many normal outliers. These Python events, and the associated syscalls, are unusual, but Python scripts and syscalls are not inherently suspicious enough to turn them into alerts by themselves. In the southeast quadrant there are three types of conventional rule based detections for the Python shell activity and the associated syscalls. These again are not clearly malicious, when considered individually. None of these, so far, are decidable enough to fire alerts on individually.

What we call a detection lattice is in the southwest quadrant. What this indicates is that the engine saw multiple agreements, on the very same events, by the different detection pipelines. The conventional alerts tell us these events could be threat activity, possibly even something like Copy Fail. The machine learning detections tell us that these events are extremely unusual for this compute instance. One detection lattice is enough to create what we call a beta signal – essentially meaning suspicious activity is happening that is far outside what normal looks like. Multiple detection lattices is enough to create what we call an alpha signal. An alpha signal is usually a true positive detection because this many strange things, that resemble potential threat activity, happening in the same place, at the same time, by pure chance, is simply improbable.

The syscall events and associated detections here are something new we just added to our endpoint project. We’re going to start logging more types of syscalls using something like bpftrace (very carefully, as we generally don’t want to log every single syscall, particularly the one that are already evented.) The process detection elements, both conventional and machine learning, can be created today without eventing the syscalls used by Copy Fail.

Show and tell: agent smith


So we are seeing more and more cases of malicious code injected into shared models, skills, tool, agents, and projects. I had a case where a shared model had a number of malicious code blocks that the users had not noticed due to the size of the codebase. At the time of this writing, we are seeing proliferation of malicious skills and prompts with a variety of payloads. When run in an IDE with AI tools, much of these produce execution and C2 activity that is detectable if something was watching closely enough. The challenge is that these events will be in a haystack of benign process and network events from tasks the AI tools were given permission for by the user. One of the purposes of the ODR (open DR) project is to hunt for this class of threat activity using anomaly detection techniques and a learning informed detection pipeline that is continuously updated.

One reason to run these hunts at the local level, in addition to conventional SOC and SIEM operations, is that the definition of what normal looks like, for a particular IDE, is largely in the head of the developer. Another reason is that much of the context is on the endpoint where the activity took place, and all of that state cannot generally be logged to a SIEM due to data volume and cost. At the same time, devs cannot monitor every action taken by their tools, and they are not trained in what to look for. This feels like a job for an autonomous agent pack.

Smith is an autonomous agent pack that will process most alerts and anomaly detections generated by ODR (open DR, also in this GitHub org.) ODR mainly looks for strange things happening in your AI dev tools at present but Smith will process most alerts generated by ODR. There is a show and tell video here: https://youtu.be/lsh3JRne9sg and the project lives in a repo here: https://github.com/opendr-io/agentic-park/tree/main/smith

Smith processes raw event and alert data from the ODR (open DR) project in order to investigate alerts and anomaly detections. It outputs an analysis of each alert. It lets you know when it thinks it has a high confidence detection and engages you in a collaborative analysis conversation in order to work out whether the detected activity is benign and expected or unexpected and potentially malicious.

Some sample alert data is provided so that it can do something out of the box. Sample event data is not provided due to its size, and the need to sanitize it, but can be generated by running openDR. It has a filter layer to try to stop prompt injections from reaching the agents and one filter test case alert is included in the sample data; you will see it get “intercepted” by the filter. That is an interesting area of research and we would like to hear from both offensive and defensive researchers as we add more filtering techniques and more detections.