Fixed a perf regression by removing system calls and improving the reachability graph and the callee lookup by corneliuhoffman · Pull Request #556 · opengrep/opengrep

corneliuhoffman · 2026-01-29T16:25:53Z

This PR introduces 4 changes

removes the Unix.realpath that slowed everything down
Simplifies the reachability graph to compute in batch using OcamlGraph primitives, faster computation.
Improves the lookuop_callee_from_graph by not searching over all edges but only those going int the caller
adds a fix for elixir shortLambdas

…llee_from_graph

dimitris-m · 2026-01-29T22:26:43Z

src/tainting/Graph_from_AST.ml

+  (* Elixir: &func/n - ShortLambda wrapping a call to the named function.
+     Structure: OtherExpr("ShortLambda", [Params[&1,...]; S(ExprStmt(Call(func, args)))])
+     Create a _tmp node to match what AST_to_IL creates for the anonymous wrapper. *)
+  | G.OtherExpr (("ShortLambda", shortlambda_tok), [G.Params _; G.S { G.s = G.ExprStmt (inner_e, _); _ }]) ->


In places like AST_to_IL we have a way to specify the language, for example:

G.OtherExpr (("ShortLambda", _), _) when env.lang =*= Lang.Elixir

We also have short lambdas in Clojure, are these covered?

will write a clojure test

I think Clojure is not yet supported

dimitris-m · 2026-01-29T22:33:27Z

src/call_graph/Call_graph.ml

          |> List.find_opt (fun edge ->
              let label = G.E.label edge in
-              (* Implicit edges have line 0 - match by callee name if call is to same function *)
-              Int.equal label.call_site.Pos.line 0)
-          |> Option.map G.E.src
+              Pos.equal label.call_site call_pos)
+        in
+        match exact_match with
+        | Some edge ->
+            Some (G.E.src edge)
+        | None ->
+            (* Fallback: check for implicit/HOF edges by matching line 0 *)
+            incoming_edges
+            |> List.find_opt (fun edge ->
+                let label = G.E.label edge in
+                Int.equal label.call_site.Pos.line 0)
+            |> Option.map G.E.src


In fact one can do find_opt only once right?

And return a type Explicit _ | Fallback _ or whatever.

Not so important for now.

yes but then you need to worry about whioch came first I agreee it might be mildly cheaper though I still have to convince myself wether the fallback ever happens

dimitris-m · 2026-01-29T22:35:10Z

src/tainting/Graph_from_AST.ml

  in
-  (* Combine and deduplicate *)
-  all_callback_args @ configured_callbacks |> dedup_fn_ids
+  (* Combine - dedup would need to handle the tmp_opt too, skip for now *)


Why do we skip dedup? Because _tmp are identified?

bc the dedup is already done by the graph

dimitris-m · 2026-01-30T01:47:25Z

src/tainting/Graph_from_AST.ml

+          let tmp_tok = Tok.fake_tok shortlambda_tok "_tmp" in
+          let tmp_name = IL.{
+            ident = ("_tmp", tmp_tok);
+            sid = G.SId.unsafe_default;


But AST_to_IL.fresh_var has a proper sid:

let i = G.SId.mk () in

which I guess does not matter here?

yes the sids are irrelevant here

dimitris-m · 2026-01-30T04:03:06Z

Gentle reminder to change PR title to something more informative like:

Fix performance regression in intrafile (system calls)

maciejpirog · 2026-01-30T07:47:02Z

src/call_graph/Graph_reachability.ml

+      let from_sources = reachable_vertices_batch graph sources in
+      let from_sinks = reachable_vertices_batch graph sinks in
+      (* Fast set intersection *)
+      let common = VSet.inter from_sources from_sinks in


How about the situation when the input graph is:

src -> foo <- snk | v bar

My understanding is that it will add bar to the relevant graph, although it is not really necessary

true, this my plan for today, the graph lib has some trick for that, you need to look at SCC's that have no predecesors.

we agreed we will keep the filtering for now, In principle things like this will be a problem but our HOF technology is not yet sophisticated enough to catch them anyway now:

def f(): return source() def g(x): # ruleid: test-tuple-unpacking-taint sink(x) return x def h(x): return (f(), lambda y: g(y)) def k(x): a, b = h(x) z, w = b(a)

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [opengrep/opengrep](https://github.com/opengrep/opengrep) | minor | `v1.15.1` → `v1.16.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>opengrep/opengrep (opengrep/opengrep)</summary> ### [`v1.16.0`](https://github.com/opengrep/opengrep/releases/tag/v1.16.0): Opengrep 1.16.0 [Compare Source](opengrep/opengrep@v1.15.1...v1.16.0) #### Improvements - Dart: Add typed metavariabless by [@maciejpirog](https://github.com/maciejpirog) in [#551](opengrep/opengrep#551) - Dart: Use case of identifier to guess call vs new by [@maciejpirog](https://github.com/maciejpirog) in [#555](opengrep/opengrep#555) - Go: Enable goroutines in taint tracking by [@maciejpirog](https://github.com/maciejpirog) in [#559](opengrep/opengrep#559) - Add taint propagation via "for" comprehensions by [@maciejpirog](https://github.com/maciejpirog) in [#564](opengrep/opengrep#564) #### Bug Fixes - Rust: Missing Rust type alias translation by [@smith-xyz](https://github.com/smith-xyz) in [#549](opengrep/opengrep#549) - Fix: Ensure that linux binaries have 8mb stack size (musl) by [@dimitris-m](https://github.com/dimitris-m) in [#563](opengrep/opengrep#563) - Fixed a perf regression by removing system calls and improving the reachability graph and the callee lookup by [@corneliuhoffman](https://github.com/corneliuhoffman) in [#556](opengrep/opengrep#556) - Fixed intrafile bug introduced by a superfluous fallback by [@corneliuhoffman](https://github.com/corneliuhoffman) in [#567](opengrep/opengrep#567) - Ruby: Always translate `or` and `and` to expression by [@maciejpirog](https://github.com/maciejpirog) in [#562](opengrep/opengrep#562) - Bash: Allow redirects before command arguments by [@maciejpirog](https://github.com/maciejpirog) in [#548](opengrep/opengrep#548) #### Internal Improvements - Add `show dump-intrafile-graph` and `show dump-taint-signatures` commands by [@corneliuhoffman](https://github.com/corneliuhoffman) in [#552](opengrep/opengrep#552) - Improve tainting code by [@maciejpirog](https://github.com/maciejpirog) in [#546](opengrep/opengrep#546) - Graph refactoring by [@corneliuhoffman](https://github.com/corneliuhoffman) in [#553](opengrep/opengrep#553) #### New Contributors - [@smith-xyz](https://github.com/smith-xyz) made their first contribution in [#549](opengrep/opengrep#549) **Full Changelog**: <opengrep/opengrep@v1.15.1...v1.16.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

removed the system call, batch graph reachability, improved lookup_ca…

b8af8ef

…llee_from_graph

corneliuhoffman requested review from dimitris-m, maciejpirog and willem-delbare as code owners January 29, 2026 16:25

fixed elixir short lambdas

be14e09

dimitris-m reviewed Jan 29, 2026

View reviewed changes

dimitris-m reviewed Jan 30, 2026

View reviewed changes

maciejpirog reviewed Jan 30, 2026

View reviewed changes

maciejpirog approved these changes Feb 2, 2026

View reviewed changes

corneliuhoffman changed the title ~~bug fix and improvements~~ Fixed a perf regression by removing system calls and improving the reachability graph and the callee lookup Feb 2, 2026

improved the reachability

ddf7d35

corneliuhoffman force-pushed the ch/graph_performance branch from ed4d71c to ddf7d35 Compare February 2, 2026 14:09

corneliuhoffman merged commit eb76031 into main Feb 2, 2026
11 of 12 checks passed

corneliuhoffman deleted the ch/graph_performance branch February 2, 2026 14:34

maciejpirog mentioned this pull request Feb 4, 2026

Release v1.16.0 #568

Merged

Conversation

corneliuhoffman commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dimitris-m commented Jan 30, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

corneliuhoffman commented Jan 29, 2026 •

edited

Loading