Skip to content

use same chromium path install method is invoked with for background install#15

Merged
tsullivan merged 1 commit intotsullivan:screenshotting/server-package-7.17from
eokoneyo:fix/screenshotting/server-package-7.17
Aug 13, 2024
Merged

use same chromium path install method is invoked with for background install#15
tsullivan merged 1 commit intotsullivan:screenshotting/server-package-7.17from
eokoneyo:fix/screenshotting/server-package-7.17

Conversation

@eokoneyo
Copy link
Copy Markdown

Summary

Took a look at the test failures from elastic#189326, the most recent failure pointed out that the executable for linux was being requested from the wrong path, see image below;

Screenshot 2024-08-12 at 09 28 47

I ran this fix, in https://buildkite.com/elastic/kibana-pull-request/builds/226872#0191380a-3ff3-49cb-ae34-73ee20eac596 and all tests pass besides the checks for console.log I had included to debug the value of the binary path in the eventuality of a failure which didn't happen.

@tsullivan tsullivan merged commit e6b28bc into tsullivan:screenshotting/server-package-7.17 Aug 13, 2024
@eokoneyo eokoneyo deleted the fix/screenshotting/server-package-7.17 branch August 13, 2024 15:46
@eokoneyo eokoneyo restored the fix/screenshotting/server-package-7.17 branch August 20, 2024 13:13
@eokoneyo eokoneyo deleted the fix/screenshotting/server-package-7.17 branch August 20, 2024 13:13
tsullivan pushed a commit that referenced this pull request Apr 30, 2026
## Summary

Closes elastic#254714 

- Adds Claude reviewer using [GH Agentic
Workflows](https://github.github.com/gh-aw/introduction/how-they-work/)
- Adds workflow handling of @/claude mentions in comments
- Adds a `code-reviewer` subagent for reuse in workflows.
- Proxies through LiteLLM for Opus 4.7 model

### Testing

https://github.com/elastic/kibana/actions/workflows/reviewer-claude.lock.yml

This branch is directly in `elastic/kibana` rather than my fork and has
been running the workflow for reviews during development with
`pull_request` as the event, rather than the final state of
`pull_request_target` events. You can see comments Claude has posted and
responses to being mentioned. "No issues found" comments have been moved
to a `no-op` instead, the status checks reflect when the agent runs
anyways.

Will need to follow up with a proper `pull_request_target` run after
merge. There are some security settings automatically applied during
compilation that we cannot verify yet. Since we're gated behind the
label, there shouldn't be any issues immediately post merge with the
workflow running unnecessarily.

### Note for Reviewers
- It isn't necessary to review the two `lock` files. They are compiled
and generated files, but must be committed for the workflow to run.
- Open to different model versions, don't have a strong preference
towards Opus 4.7

## Cache and cost analysis

Tested ten Opus 4.7 (`llm-gateway/claude-opus-4-7`) reviewer runs with
`ENABLE_PROMPT_CACHING_1H` enabled. The cache is active, but the data
includes dirty cases: prompt changes, long gaps outside the useful cache
window, regular PR review runs, and comment-reply runs.

| Run | Event | Head | Cost | Turns | Cache read | Cache write | Output
| Effective tokens |
| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| [#15](https://github.com/elastic/kibana/actions/runs/25011054153) |
`pull_request` | `2076d4f` | $1.04 | 14 | 809,672 | 80,897 | 5,169 |
182,559 |
| [#16](https://github.com/elastic/kibana/actions/runs/25011557405) |
`pull_request` | `382357b` | $1.32 | 21 | 1,495,919 | 70,406 | 5,226 |
240,928 |
| [#17](https://github.com/elastic/kibana/actions/runs/25012064350) |
`pull_request` | `e93fe0b` | $1.33 | 23 | 1,378,836 | 50,413 | 13,022 |
240,413 |
| [#18](https://github.com/elastic/kibana/actions/runs/25013588252) |
`pull_request` | `04ded1d` | $1.20 | 24 | 1,453,092 | 39,648 | 8,876 |
220,490 |
| [#19](https://github.com/elastic/kibana/actions/runs/25030187214) |
`pull_request` | `88a2c7e` | $1.41 | 16 | 1,165,558 | 99,949 | 8,171 |
249,210 |
| [#21](https://github.com/elastic/kibana/actions/runs/25030484848) |
`pr_review_comment` | `88a2c7e` | $0.45 | 9 | 512,383 | 19,390 | 3,035 |
82,782 |
| [elastic#23](https://github.com/elastic/kibana/actions/runs/25031202052) |
`pull_request` | `66b198f` | $1.39 | 18 | 1,230,107 | 78,104 | 11,420 |
246,818 |
| [elastic#26](https://github.com/elastic/kibana/actions/runs/25033088024) |
`pull_request` | `d59d8c0` | $1.50 | 22 | 1,428,302 | 90,868 | 8,885 |
269,265 |
| [elastic#27](https://github.com/elastic/kibana/actions/runs/25034181833) |
`pull_request` | `d1052ef` | $1.73 | 24 | 2,016,691 | 81,113 | 8,737 |
317,759 |
| [elastic#28](https://github.com/elastic/kibana/actions/runs/25034226594) |
`pr_review_comment` | `d1052ef` | $1.18 | 23 | 1,453,250 | 36,528 |
7,788 | 213,038 |

| Run | First request cache read | First request cache write | Notes |
| --- | ---: | ---: | --- |
| 15 | 0 | 51,751 | Cold 1-hour cache write. |
| 16 | 35,886 | 15,905 | Stable prefix reused. |
| 17 | 35,886 | 15,907 | Same prefix hit pattern. |
| 18 | 35,886 | 15,911 | Same prefix hit pattern. |
| 19 | 0 | 53,676 | Cold again after long gap + prompt/head changes. |
| 21 | 45,471 | 8,375 | Comment-reply flow reused 19 prefix and wrote a
smaller suffix. |
| 23 | 37,245 | 16,587 | New commit/prompt still reused part of the
1-hour prefix. |
| 26 | 0 | 53,761 | Cold again after another gap + prompt/head changes.
|
| 27 | 37,245 | 16,511 | Warm PR run on a new head/prompt hash. |
| 28 | 45,465 | 8,461 | Comment-reply flow reused 27 prefix, but still
ran long. |

Summary:
- All observed cache creation was 1-hour (`ephemeral_1h_input_tokens`);
no 5-minute writes were observed.
- Average observed cost is about $1.26/run across all ten runs, or about
$1.37 for normal `pull_request` runs excluding comment replies.
- Comment replies are not inherently cheap: run 21 was $0.45 because it
was narrow and 9 turns, while run 28 was $1.18 because it took 23 turns.
- Cold starts appear after long gaps or prompt churn (runs 15, 19, 26);
warm runs still reuse partial prefixes even when the commit/prompt hash
changes.
- Next tuning is comparing 5-minute vs 1-hour caching, testing cheaper
models against review quality, and auditing trace data for unnecessary
output or artifact reads.

---------

Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants