Skip to content

Allow usage of the generate action in generators when testing#34418

Closed
tubbo wants to merge 1 commit intorails:masterfrom
tubbo:fix-generate-command-in-generator-tests
Closed

Allow usage of the generate action in generators when testing#34418
tubbo wants to merge 1 commit intorails:masterfrom
tubbo:fix-generate-command-in-generator-tests

Conversation

@tubbo
Copy link

@tubbo tubbo commented Nov 9, 2018

Summary

It was previously not possible to test whether a generator could
invoke another generator using the generate command, because this
shells out and can potentially be mocked or its errors can be hidden
from the user. The command always runs, but its effects cannot be
observed. For example, if a file is supposed to be generated using
generate :model, the file will not exist in the destination_root of
the test when executed. To remedy this, instead of shelling out with
#run_ruby_script, use Generator.invoke directly and pass in all of
the arguments given to the command.

Other Information

(none)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried rails instead bin/rails? It always shellout and it was working and for the report you have it was failing because bin/rails doesn't exist in the execution directory but rails will work in any directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't investigate that much but changing the run_ruby_script("bin/rails ...") to run_ruby_script("rails ...") produces the same problem (i.e. No such file or directory -- rails).

On the other hand, relying on system("rails generate ...") properly triggers the generation but the file are placed inside the application's app and test directories rather than the specified source_destination in the test.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rafaelfranca yes, @robin850 is correct...I tried your suggestion of changing bin/rails to rails, but I got the same issue in tests. Additionally, when actually trying to run it in the real world, I got an additional "No such file or directory" error printed to STDOUT. That said, I don't believe the ability to test the results of a generator called from another generator ever worked, at least I can't remember a time when it did...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, using Generators.invoke rather than shelling out removes support for code doing something like:

generate("scaffold articles --no-timestamps")

While it looks like the test suite isn't relying on such case and the documentation doesn't mention it, people might rely on this feature. But I don't know whether we should have something like:

unless args.present?
  what, args = a.split(" ", 2)
end

Or deprecate this ability.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to test this we should not be stubbing anymore, this test give no guarantees this will work.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rafaelfranca agreed, I don't believe this assertion is every useful...would you be OK with me replacing the test to be this?

action :generate, "model", "MyModel"
assert_file('app/models/my_model.rb')

That way we're actually testing the results of the operation rather than whether some command was called...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@tubbo
Copy link
Author

tubbo commented Nov 9, 2018

also...I targeted this for 5-2-stable, felt like it was safe for a patch but I'd be OK with moving it to master if y'all don't feel the same way...

@rafaelfranca
Copy link
Member

oh, right, we don't merge things on stable branches unless the bug doesn't exist on master. It doesn't seems to be the case.

It was previously not possible to test whether a generator could
invoke another generator using the `generate` command, because this
shells out and can potentially be mocked or its errors can be hidden
from the user. The command always runs, but its effects cannot be
observed. For example, if a file is supposed to be generated using
`generate :model`, the file will not exist in the `destination_root` of
the test when executed. To remedy this, instead of shelling out with
`#run_ruby_script`, use `Generator.invoke` directly and pass in all of
the arguments given to the command.

Fixes rails#33558
@tubbo tubbo force-pushed the fix-generate-command-in-generator-tests branch from d6f2aba to 6a8f5d2 Compare November 9, 2018 19:03
@tubbo tubbo changed the base branch from 5-2-stable to master November 9, 2018 19:03
@rails-bot rails-bot bot added the railties label Nov 9, 2018
@tubbo
Copy link
Author

tubbo commented Nov 9, 2018

@rafaelfranca ahh ok, thanks for clarifying! I re-based it to 'master'

@deivid-rodriguez
Copy link
Contributor

@tubbo Does #34420 also fix the problem you're trying to solve here? I didn't fully understand your problem, that's why I'm asking. Is it an internal rails testing issue or a final end user problem?

@tubbo
Copy link
Author

tubbo commented Nov 14, 2018

@deivid-rodriguez It would seem so! And your approach seems less "breaking" than mine, since I'm changing the entire way generate is executed rather than shelling out with proper environment configuration. Assuming that all calls to in_root { run("some command") } suffer from this problem, we might be avoiding bugs elsewhere by going with your solution.

@rafaelfranca you mind if I close this in favor of #34420? it seems to solve a couple more problems than just the generate action.

@deivid-rodriguez
Copy link
Contributor

@tubbo Great! Yeah, my PR should basically keep things as they are now, but you can opt-in to a more strict behavior via the new abort_on_failure option.

@rafaelfranca
Copy link
Member

No at all. I'll review #34420.

@tefi-anbessa
Copy link

I fail to see how aborting the generator solves the original problem, which still exists to this day as far as I can tell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants