💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Steps errors

Thanks @estolfo for adding this shared spec.

I feel that we might get better adoption for shared specs by keeping them as short and as concise as possible.
Do you mind if we simplify a bit the structure, for example by using scenario outlines and examples ?

@SylvainJuge Sure, simpler is always better 😃
Can you give me an example of what you mean by an outline and example?

Hi @estolfo , I finally managed to make progress on this.

What I meant by using Scenario outline is doing something like this to test the HTTP codes mapping:

  @http
  Scenario Outline: HTTP transaction and span outcome
    Given an HTTP transaction with <status> response code
    Then transaction outcome is "<server>"
    Given an HTTP span with <status> response code
    Then span outcome is "<client>"
    Examples:
      | status | client  | server  |
      | 100    | success | success |
      | 200    | success | success |
      | 300    | success | success |
      | 400    | failure | success |
      | 404    | failure | success |
      | 500    | failure | failure |
      | -1     | failure | failure |
      # last row with negative status represents the case where the status is not available
      # for example when an exception/error is thrown without status (IO error, redirect loop, ...)

Here is the full file I'm using to test the Java agent implementation.
There are a few gaps in the spec that were not covered that I thought would be relevant to add:

• db spans (was already covered by the dotnet agent)
• gRPC status codes mapping with something close to what we have with HTTP client/server (will likely require some sync with other agents).

Also, the usage of tags like @http and @grpc allow to filter which parts to include or exclude, which makes it possible to only test for gRPC related features when testing gRPC.

One notable thing that might require a bit of discussion is that we think that having outcome = failure whenever an error is reported (and success otherwise) for a given span/transaction seems a fair default when there is no protocol-specific mapping.

For example, that would cover db spans and messaging spans & transactions without much effort.

Also, there is one minor thing that we might change in future specs, the Given: an agent statement is currently used to provide a hook to initialize the agent in every test scenario.

In Java, unlike in Ruby, we don't test the whole agent as doing this would require proper packaging and would move those tests in the integration test category, which run later in the build pipeline and are thus a high-latency feedback loop. Thus, to make those specs more relevant we only test the relevant parts of the agent like a regular unit-test.

As a result, we have different instructions to setup the agent for each set of features we want to test (currently API token/secret auth & outcome), and we can't have a common implementation for Given: an agent, we use an before hook instead with a no-op implementation for Given: an agent spec statement.

Updated with changes after initial review by @eyalkoren :

• improved gRPC mapping for gRPC transactions
• simpler definition of span outcome similar to transactions.

@axw Given your knowledge of gRPC, do you have any opinion on the mapping proposal here ? I remember the Go agent current implementation maps gRPC UNKNOWN status to unknown outcome (which does not fit this change proposal).

@axw Given your knowledge of gRPC, do you have any opinion on the mapping proposal here ? I remember the Go agent current implementation maps gRPC UNKNOWN status to unknown outcome (which does not fit this change proposal).

I think that was a mistake, and should be treated as a bug in the Go agent. The mapping here makes sense to me.