Upgrading from 9.5.1 to 10+ adds a sorbet dependency that affects unit tests unrelated to Shopify #1109
Description
Issue summary
Our organization is experimenting with an upgrade from v9.5.1 to v12.4.0. When we pulled in all the dependencies, sorbet caused many of our specs, completely unrelated to Shopify, to fail. I was able to get the unit tests to pass by adding a sorbet-runtime dependency to one of our spec helper classes, but we have concerns that this could cause errors in unrelated parts of our codebase after deployment.
I have set the default_check_level to :never and overridden the runtime error handlers in the main application class, but it's hard to be verify the impact across the entire application. Please advise: is there anything we should be doing to ensure that sorbet will not perform type checking on our code during runtime?
shopify_apiversion: 12.4.0- Ruby version: 2.7.0
- Operating system: MacOS 12.4
1) {Spec expectation}
Failure/Error: allow_any_instance_of(action_klass).to receive(:perform_action).and_return(nil)
NameError:
undefined method `perform_action' for class `Organization::Activity'
Did you mean? perform_other_action
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/sorbet-runtime-0.5.10626/lib/types/compatibility_patches.rb:30:in `instance_method'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/sorbet-runtime-0.5.10626/lib/types/compatibility_patches.rb:30:in `observe!'
# ./spec/controllers/controller_actions/apps/tested_class_spec.rb:6153:in `block (5 levels) in <main>'
# ./spec/controllers/controller_actions/apps/tested_class_spec.rb:6152:in `each'
# ./spec/controllers/controller_actions/apps/tested_class_spec.rb:6152:in `block (4 levels) in <main>'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:124:in `block in run'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `loop'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:110:in `run'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/rspec-retry-0.6.2/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/rspec-retry-0.6.2/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
# /Users/raymondweidner/workspace/platform_gems/ruby/2.7.0/gems/webmock-3.14.0/lib/webmock/rspec.rb:37:in `block (2 levels) in <top (required)>'
Expected behavior
The shopify_api gem upgrade shouldn't affect anything unrelated to Shopify functionality, including specs and the rest of the codebase.
Actual behavior
Many unrelated unit tests fail on buildkite once the shopify_api gem is upgraded and its dependencies (especially sorbet-runtime) are pulled into our Gemfile.lock.
Steps to reproduce the problem
- Upgrade shopify_api from 9.5.1 to 12.4.0 in the Gemfile
- Apply dependency changes (e.g.
bundle install). - Perform unit tests with
rspec.