-
-
Notifications
You must be signed in to change notification settings - Fork 95
Improve title handling, especially for example scenarios #271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve title handling, especially for example scenarios #271
Conversation
acoulton
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the slow review, been a busy week.
Urgh, I hadn't appreciated how complex & nested the interfaces for these classes are. Considering that, I'd be nervous about adding yet another.
As I see it, the problem we're fighting is:
- Arguably,
getTitle()has always returned the wrong thing for an ExampleNode - the idea implied by the interface is it looks and behaves like a Scenario, in which case thetitleshould be the user-supplied title of the Scenario Outline, not the example values in text form. - However, we can't change the result of
getTitle()because there are existing valid uses - in Behat and probably elsewhere - that need the text example and rely on getting it from that method. - And these are implementations of interfaces, not simple DTOs, so as you say, there's no easy way to provide a BC path to deprecate ambiguous method and add a new one.
So I think my suggestion for now is:
- Add the index to
ExampleNodeas planned. - Add
getName()toExampleNode(instead of calling itgetScenarioTitle()) - this would align with the naming in cucumber's AST. - Add the
getExampleText()as planned. - Deprecate
getTitle()only onExampleNodebut only with a docblock tag pointing togetName()orgetExampleText(), not a runtime deprecation. - Open an issue for a future major version to consider changing the interfaces so that anything "scenario-like" has a
getName()instead of agetTitle(). I would probably do this alongside bumping Behat's major, so that we can be sure any extensions etc only get the Gherkin objects they were expecting even if they don't specify abehat/gherkinconstraint of their own.
This means that:
- The change is BC for all existing users
- Only users who are writing formatters or similar and explicitly considering ExampleNode will see the deprecation (in their IDE or e.g. phpstan etc). But these are really the only cases at the moment where it's necessary to encourage people to be explicit about which one they want.
- If / when we release a new major renaming
getTitle()togetName()then code that has already been updated forExampleNodewill be fine, anything that is only using the scenario interfaces will by default get the new (/expected) value instead of the table text, and we can highlight the difference in the upgrade notes.
This feels a bit messy, and I'm not sure it's right, but I think it probably is the least-worst compromise solution to adding this feature in a BC way.
Thoughts welcome!
|
@acoulton your points are most fine to me except this:
Why wait? If we do that later, we won't have a chance to deprecate it - it will suddenly be renamed from getTitle to getName (unless we have to wait for 2 major versions). I would define getName in both scenario and example nodes. Additionally, I would consider creating a new The point is that it would nice (for users) to code against Another interesting thing to note: I was considering deprecating getTitle at the interface level (it is defined in |
|
@uuf6429 my reluctance to introduce new interfaces/deprecate things at interface level at this stage is exactly because I think it needs more thought about what that structure should be. ISTR there are nodes currently defining I think part of the problem is methods have been defined at the wrong level of the hierarchy because they appear to be similar. For example You can see from the existing Behat formatter implementations that actually it very rarely makes sense to call But to deprecate it on the interface now, we need to find a well-designed and BC way to replace it across all the nodes that implement it : even though some nodes might actually be fine with Really for this usecase what we want to capture is that So if we had to introduce an interface now I would call it I wouldn't use |
|
Oh and on this specific:
I don't see us going immediately from this to a new major. Indeed with current resources a new major could still be a way off, especially if we're going to look at switching parser/moving to Pickles etc. And even when we release a major there will be a period where we are maintaining both version series. So there will still be an opportunity to deprecate methods between now and then - I just think we should minimise that until we can give users a clear overall explanation of what will be changing. Deprecating the method on ExampleNode where it is currently ambiguous feels fine to do right now as there is an identified problem with that specific object and we have an easy solution. |
|
@acoulton I agree with all points.
That makes sense to me, especially since most probably we'd want consistency over those two classes. What do you think about Anyway, I'll implement this interface and open this PR for review, I think we're in a good state now for that. |
|
I think Outline is a higher level grouping of scenarios, like a feature or a suite (or a Rule if/when we support them). It's not the same as a Scenario/Example which represents a specific set of executable steps. So I would leave Outline alone, for the moment at least. |
|
@acoulton side-note: there is a |
|
@acoulton while adjusting tests, I found out that outline represents the "scenario outline" node and that, for example, iterating over scenarios in a feature file will actually yield instance of that object. Apparently it is up to caller to iterate over the "example [child] nodes" of that object while ignoring the rest of the object. In short, a naive |
|
Hi @carlos-granados! I guess @acoulton might be a bit occupied at the moment, so maybe you could give the task a look? A second (well, third) set of eyes wouldn't hurt. 😄 |
|
Apart from my comments about a couple of comments that need to be updated, everything else looks OK to me |
acoulton
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@uuf6429 I'm so sorry for the slow review - I hadn't originally realised you'd done more on it (it seems github doesn't send a notification when a draft PR is marked ready for review, only if you specifically request a review or add a comment). And then I was off sick and then trying to catch up.
This looks good to me with a couple of very small comments - @carlos-granados the getScenarioTitle() references in phpdocs are just leftover from a previous iteration, they can be updated to getName()
acoulton
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks :)
carlos-granados
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just found a tiny thing that needs changing
carlos-granados
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good now, thanks @uuf6429
|
@acoulton now that his is finally merged, should we create a release so that this can be used on the Behat side? |
|
@uuf6429 released now, thank you! |
|
First of all, thanks again for the effort. One quick point, can we somehow make sure to squash PRs? It's just that I'm not typically working on branches with the understanding that the commits (and more importantly, their messages) will show up verbatim in the release branch. |
|
@uuf6429 we can certainly try, the challenge is if there are unrelated changes - for example the fixes for the file cache tests on windows in this PR - in the same branch. I don't think they should be just squashed into a single merge commit for the overall feature, and there's no way to partially squash on GitHub. I think it's probably better if PR authors squash and force-push as appropriate before the final code review (apart from simple changes / ones that can easily be squash-merged). But you're right that we should probably be looking at the commits & messages as well as the diff when reviewing PRs. |
…it (#1682) Previously, when a feature file contained scenarios with `Example` nodes the junit report would append an index to each testcase name to differentiate them (e.g. `<testcase name="Some scenario outline #1"`, `<testcase name="Some scenario outline #2`, etc). These indices were dynamically generated based on the examples that actually ran. As a result, on a `--rerun` the names would start from `#1` in sequence and be inconsistent with the numbering of the first execution. This makes it difficult to compare reports between each test run, for example to link a passing rerun to the previous failure. With this change, examples are numbered consistently based on the order they appear in the feature file. This fixes #1448. Additionally, we also now support placeholders in the Scenario Outline title. This allows users to include some or all of the values from the Examples: table in each `<testcase` name for additional clarity. This matches the behaviour of the official cucumber runners. This PR builds on work done by @uuf6429 in #1459 and in particular the implementation of the numbering & placeholder replacement that he contributed in Behat/Gherkin#270 and Behat/Gherkin#271.
Fixes #270
Remaining points
getTitleis defined inKeywordNodeInterface- there's alsoScenarioLikeInterfaceandScenarioInterface).We could add a new interface on top, though - no BC break there.
getTitle()togetScenarioTitle()getName().\Behat\Gherkin\Node\ExampleNode::replaceTextTokenspublic? It feels less important to do so with this PR, plus it's a BC break (sinceExampleNodeis not final)