Skip to content

Conversation

@acoulton
Copy link
Contributor

@acoulton acoulton commented Nov 7, 2025

Brings the 4.x branch up to date with current master.

I AM ACTUALLY NOT SURE ABOUT MERGING THIS.

The merge essentially reverts everything that is in the current 4.x (master...4.x). Most of these are workflow / dependency changes that have already been superseded (other than enabling CI on the 4.x branch).

The only functional change in the 4.x branch is #1397, which was targeted to 4.x due to a concern about BC. The 4.x branch stalled shortly afterwards (in 2023) and that PR was never mentioned in the issue for relating to running with multiple paths (#834)

Therefore, we actually merged a similar-but-different contribution from #1611 to the 3.x series and released it in 3.23.0. It looks like we didn't pick up on the potential BC concern about changing that CLI argument to an array, but we have not had reports of any issues. I've reviewed #1397 and there's nothing there that isn't already in 3.x.

Given all that, and that the 4.x branch has never been released and has been stale for several years, I think we might be better off just deleting and recreating it from the current master.

That would give us significantly clearer git history and ensure we can more easily see what has actually gone into the 4.x series as we work towards releases on that branch.

Opinions welcome @carlos-granados @stof and anyone else!

acoulton and others added 30 commits December 4, 2024 11:13
dx: add composer scripts for testing tools
Signed-off-by: Franck Matsos <franck.matsos@akawaka.fr>
Signed-off-by: Franck Matsos <franck.matsos@akawaka.fr>
Signed-off-by: Franck Matsos <franck.matsos@akawaka.fr>
…lter-on-cli

fix: allow using the narrative filter on the command line
change: disable Xdebug by default and provide a CLI option to activate it
…ppets

Add possibility to use attributes for Context snippets
Co-authored-by: Andrew Coulton <andrew@ingenerator.com>
carlos-granados and others added 24 commits September 20, 2025 09:17
- Update rector version
- Convert string class names to class constants
- Closure to arrow function
- New string functions
- Skip StringableForToStringRector rule
- Use ::class instead of get_class
- Convert class properties into constructor promoted properties
- Mark properties as readonly
- Several small miscellaneous changes
- Final Rector php set config
So that it's easier to run individual examples and see the actual
output, and to have completion etc for them.
Because of the stage where it is thrown, the exception is treated as
an overall fatal error. This meant that Behat does not output any detail
on the feature file / step it was parsing when it encountered the
unexpected argument. As a result, users had no information on the cause
of the error and would have to check their features / steps manually.

Instead, include the name of the function and (where possible) the
feature and step within the exception message. This will still mean that
users have to fix these one at a time (Behat will exit on the first such
exception) but they should at least now be able to locate the issue.
…on-context

fix: Provide more context when failing with unexpected multiline args
Adds and AGENTS.md file which AI agents can use to work in our codebase
Lately JSON has become the lingua franca of tools, which use this format
to exchange information everywhere. This PR introduces a JSON formatter
which is able to save Behat test results as a JSON file. One possible
application for this format is to be able to create an MCP server for
Behat, using the generated JSON to return information to the client

This formatter is heavily inspired in the existing JUnit formatter and
the exported output is similar but, since it is not constrained by the
existing JUnit specification, it follows more closely the structure of
our project. The generated JSON basically will contain a list of Suite
nodes, which will contain Feature nodes, which will contain Scenario
nodes. Each node will have properties indicating some info like the
status, the number of tests for each status, file and line and execution
time. Also, the JUnit formatter produced one file per suite, this
formatter produces just a single JSON file for all suites so the
`output_path` needs to be a file name, not a folder name

The PR also defines a JSON schema which can be used by the tools that
will use this information
The latest version of php-cs-fixer showed a warning saying
"Unable to determine minimum supported PHP version from composer.json:
Invalid version string "8.1.*""

It seems that, though composer was able to understand our constraint,
php-cs-fixer wasn't

I updated it to use a new constraint that should be understood by
everyone
In the `ConfigConverterTools` we were using `assert` to check a couple
of conditions that should be checked at run time. The problem is that
php assertions can be disabled and are not guaranteed to be available.
This PR modifies the code to use if statements and exceptions instead
The JSON and JUnit formatters were using a very simple management of the
printed file paths. This PR replaces this with the
ConfigurablePathPrinter that the other formatter use. This brings a
couple of advantages:
- It uses the `%paths.base%` to calculate relative paths, this is more
correct than using cwd as those formatters were doing
- It allows us to use the options offered by that printer

The editorUrl option of that printer is not appropriate for these
formatters, which cannot use the link added to the paths, so I added a
flag to mark if this option needs to be applied or not

Adds tests that check the correct working of the printer options in
these formatters
When the error reporting level was set (either by using
`withErrorReporting()` or by using the default value of E_ALL), we were
calling `set_error_handler()` with this level but were not calling
`error_reporting()`. This meant that event though we had set a handler
to be called on certain error levels, this handler might not be called
if the error_reporting level set in php.ini was more restrictive. I saw
this in a system where error_reporting was set to `E_ALL & ~E_DEPRECATED
& ~E_STRICT` and the test `With default error reporting reports all PHP
errors and deprecations` failed because the deprecation was not raised

This PR fixes this problem by actually setting the error_reporting
level, not just using it in the set_error_handler call

Fixes Behat#1663
Behat#1680)

Adds the "file" attribute in feature nodes in the JSON and JUnit
formatters

Replaces Behat#1601
The information that shows the time spent to run each test in the JSON
and JUnit formatters is interesting if you are trying to optimise the
performance of your tests but in many other cases you may only be
interested in whether the tests pass or not and this timer information
will be of no interest to you. This PR adds a "timer" option to these
formatters (similar to the one already available in the pretty and
progress formatters) that can be used to disable the output of this
data. It defaults to true to preserve existing functionality
…it (Behat#1682)

Previously, when a feature file contained scenarios with `Example` nodes
the junit report would append an index to each testcase name to
differentiate them (e.g. `<testcase name="Some scenario outline #1"`,
`<testcase name="Some scenario outline Behat#2`, etc).

These indices were dynamically generated based on the examples that
actually ran. As a result, on a `--rerun` the names would start from
`#1` in sequence and be inconsistent with the numbering of the first
execution. This makes it difficult to compare reports between each test
run, for example to link a passing rerun to the previous failure.

With this change, examples are numbered consistently based on the order
they appear in the feature file. This fixes Behat#1448.

Additionally, we also now support placeholders in the Scenario Outline
title. This allows users to include some or all of the values from the
Examples: table in each `<testcase` name for additional clarity. This
matches the behaviour of the official cucumber runners.

This PR builds on work done by @uuf6429 in Behat#1459 and in particular the
implementation of the numbering & placeholder replacement that he
contributed in Behat/Gherkin#270 and Behat/Gherkin#271.
With the current implementation, the checks for the JSON file (check
that the output path had been passed, that it was not a directory, that
we could create the file) would not happen until after all the tests had
been run, which could be very inconvenient, run all tests only to find
that you cannot write the file.

In this PR we change this to follow what the JUnit formatter does, which
is to try to create the file at the beginning of the test run.
Updates the hook failures tests to use fixtures instead of files created
on the fly. Uses a single fixture folder for all hook failures tests.
Removes hook failure tests for annotations
…t#1690)

The legacy mode of the gherkin parser was trimming the `@` sign from
tags when building the AST, so the pretty printer was prepending it
back.

As the upstream cucumber gherkin AST preserve the `@` sign in tags,
future versions of the behat/gherkin parser will preserve it as well.

Refs Behat/Gherkin#336

The pretty printer can handle that case by detecting the format of the
tag, without any need of knowing the compatibility mode used by the
parser.
Note that the filtering by tags is not updated in this PR because the
implementation of the filter is done in behat/gherkin, not in this repo.
Adds PHP 8.5 to our build matrix and supported versions.
`FixedContextIdentifier` is always created and registered by the
`ContextSnippetsController` with an injected `contextClass` based on the
 `--snippets-for` CLI option - see
https://github.com/Behat/Behat/blob/0d4162bbf8401bfba8085a25f427eabe6d00cead/src/Behat/Behat/Context/Cli/ContextSnippetsController.php#L65.

If this option has not been specified on the CLI then the `contextClass`
property will be `null`.

This was causing us to call `ContextEnvironment::hasContextClass` with
a `null` despite that method being typed as taking only
`class-string<Context>` (the method currently only has a phpdoc
typehint). This means `InitializedContextEnvironment::hasContextClass()`
then attempts to use `null` as an array key.

Up to PHP 8.4 this worked as expected, but from 8.5.0 it triggers a
deprecation.

Therefore, we now explicitly check if the value is null in
`FixedContextIdentifier` to ensure that we only call the
`->hasContextClass()` with a valid string value.
@carlos-granados
Copy link
Contributor

Agree that probably the best option will be to start fresh from scratch

@acoulton
Copy link
Contributor Author

Closed in favour of #1693 ahead of starting a new 4.x branch.

@acoulton acoulton closed this Nov 12, 2025
@acoulton acoulton deleted the merge-up-3.x-4.x branch November 12, 2025 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.