solidus_starter_frontend compatibility with extension dummy apps

[benjaminwil]

We are developing a Solidus extension that needs to test the customer-facing storefront of a Solidus application. Right now we do that by installing solidus_frontend and using its storefront.

But as of Solidus 3.2.0, solidus_frontend has been deprecated. We can continue to use it to test our extension, but it seems like at some point this will become inconvenient.

We tried to follow the instructions in the solidus_starter_frontend README to install it in our extension's dummy app. We even saw the git repo get cloned as it should. Unfortunately, when we inspected the dummy app no views or controllers it provides were present.

Reproduction instructions

1. Prepare a new extension:

   gem install solidus_dev_support
   solidus extension hello_world
   cd solidus_hello_world
   git init .
   

2. Add a install generator task to install solidus_starter_frontend.

   You might not do this in a real extension. But this reproduces the issue: you can't run the generator inside of a dummy app.

   # lib/generators/solidus_hello_world/intall/install_generator.rb
   # ...
   def install_solidus_starter_frontend
     run "bin/rails generate solidus:install --frontend=solidus_starter_frontend"
   end
   

   If you install using the "For existing stores" instructions, bin/rails app:template LOCATION="https://github.com/solidusio/solidus_starter_frontend/raw/main/template.rb you get even weirder errors.

3. Run tests:

   bundle exec rake
   

Result

The dummy app does not get solidus_starter_frontend views:

Terminal output

To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
Generating dummy Rails application...
      create  config/master.key
Copied migration 20220921220359_create_active_storage_tables.active_storage.rb from active_storage
Created database 'db/solidus_test.sqlite3'
      create  config/initializers/solidus_frontend.rb
       exist  app/assets/images
      create  vendor/assets/javascripts/spree/frontend
      create  vendor/assets/stylesheets/spree/frontend
      create  vendor/assets/images/spree/frontend
      create  vendor/assets/javascripts/spree/frontend/all.js
      create  vendor/assets/stylesheets/spree/frontend/all.css
Setting up dummy database...
bin/rails db:environment:set RAILS_ENV=test
bin/rails db:drop db:create db:migrate VERBOSE=false RAILS_ENV=test
Running extension installation generator...
bin/rails generate solidus_hello_world:install --auto-run-migrations
      create  config/initializers/solidus_hello_world.rb
      append  vendor/assets/javascripts/spree/frontend/all.js
      append  vendor/assets/javascripts/spree/backend/all.js
      insert  vendor/assets/stylesheets/spree/frontend/all.css
      insert  vendor/assets/stylesheets/spree/backend/all.css
         run  bin/rails railties:install:migrations FROM=solidus_hello_world from "."
         run  bin/rails generate solidus:install --frontend=solidus_starter_frontend from "."
    conflict  config/initializers/spree.rb
Overwrite /Users/bw/Projects/solidus_hello_world/spec/dummy/config/initializers/spree.rb? (enter "h" for help) [Ynaqdhm] Y
       force  config/initializers/spree.rb
Installing Active Storage
        rake  active_storage:install
       exist  app/assets/images
       exist  vendor/assets/javascripts/spree/backend
       exist  vendor/assets/stylesheets/spree/backend
       exist  vendor/assets/images/spree/backend
    conflict  vendor/assets/javascripts/spree/backend/all.js
Overwrite /Users/bw/Projects/solidus_hello_world/spec/dummy/vendor/assets/javascripts/spree/backend/all.js? (enter "h" for help) [Ynaqdhm] Y
       force  vendor/assets/javascripts/spree/backend/all.js
    conflict  vendor/assets/stylesheets/spree/backend/all.css
Overwrite /Users/bw/Projects/solidus_hello_world/spec/dummy/vendor/assets/stylesheets/spree/backend/all.css? (enter "h" for help) [Ynaqdhm] Y
       force  vendor/assets/stylesheets/spree/backend/all.css
       exist  app/overrides
    Solidus has a default authentication extension that uses Devise.
    You can find more info at https://github.com/solidusio/solidus_auth_devise.

    Regardless of what you answer here, it'll be installed if you choose
    solidus_starter_frontend as your storefront in a later step.

    Would you like to install it? (Y/n) Y

File unchanged! The supplied flag value not found!  db/seeds.rb

copying  migrations

creating  database

rake  db:create

Database 'db/solidus_test.sqlite3' already exists

gemfile  solidus_auth_devise

/Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions/file_manipulation.rb:274:in binread': No such file or directory @ rb_sysopen - /Users/bw/Projects/solidus_hello_world/spec/dummy/Gemfile (Errno::ENOENT) from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions/file_manipulation.rb:274:in gsub_file'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/generators/actions.rb:389:in append_file_with_newline' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/generators/actions.rb:57:in block i

n gem'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions.rb:201:in block in in_root' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions.rb:190:in block in inside'

from /Users/bw/.asdf/installs/ruby/3.0.2/lib/ruby/3.0.0/fileutils.rb:139:in chdir' from /Users/bw/.asdf/installs/ruby/3.0.2/lib/ruby/3.0.0/fileutils.rb:139:in cd'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions.rb:190:in inside' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/actions.rb:201:in in_root'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/generators/actions.rb:46:in gem' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/bundler/gems/solidus-c0a35d703265/core/lib/generators/solidus/install/install_generator.rb:155:in block in run_bundle_install_if_needed_by_plugins'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/bundler/gems/solidus-c0a35d703265/core/lib/generators/solidus/install/install_generator.rb:154:in each' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/bundler/gems/solidus-c0a35d703265/core/lib/generators/solidus/install/install_generator.rb:154:in run_bundle_install_if_needed_by_plugins'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/command.rb:27:in run' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in invoke_command'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:134:in block in invoke_all' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:134:in each'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:134:in map' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:134:in invoke_all'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/group.rb:232:in dispatch' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/base.rb:485:in start'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/generators.rb:263:in invoke' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/commands/generate/generate_command.rb:26:in perform'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/command.rb:27:in run' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in invoke_command'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/thor-1.2.1/lib/thor.rb:392:in dispatch' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/command/base.rb:87:in perform'

from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/command.rb:48:in invoke' from /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/railties-7.0.4/lib/rails/commands.rb:18:in <top (required)>'

from bin/rails:4:in require' from bin/rails:4:in 
'

run  bin/rails db:migrate from "."

cd /Users/bw/Projects/solidus_hello_world

/Users/bw/.asdf/installs/ruby/3.0.2/bin/ruby -I/Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/rspec-core-3.11.0/lib:/Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/rspec-support-3.11.1/lib /Users/bw/Projects/solidus_hello_world/vendor/bundle/ruby/3.0.0/gems/rspec-core-3.11.0/exe/rspec --pattern spec/**{,/*/**}/*_spec.rb

Provide a CODECOV_TOKEN environment variable to enable Codecov uploads

We are transitioning to a new way of loading factories for extensions.

Be sure this extension does not load factories using require but uses

the load_for() method in its spec_helper.rb, eg:

SolidusDevSupport::TestingSupport::Factories.load_for(ExtensionName1::Engine, ExtensionName2::Engine)
This will load Solidus Core factories right before the ones defined in

lib/extension_name/testing_support/factories/*_factory.rb or

lib/extension_name/testing_support/factories.rb
This message will be removed when all extensions are updated.

Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}

No examples found.
Randomized with seed 20806
Finished in 1.06 seconds (files took 4.99 seconds to load)

0 examples, 0 failures
Randomized with seed 20806
Coverage report generated for RSpec to /Users/bw/Projects/solidus_hello_world/coverage. 20 / 22 LOC (90.91%) covered.

Potential solutions

• Similar to how we generate a dummy app with --user-class args, we can also provide the option to specify --frontend. Link to source.
• Extensions come with a extension:specs default Rake task. Maybe we should also provide a extension:specs:with_frontend task.

[waiting-for-dev]

Hi @benjaminwil, thanks for opening this discussion.

You're right. It's impossible to generate the dummy application with a frontend other than solidus_frontend. There're two reasons for that:

• Technical: the installation of solidus_starter_frontend modifies the Gemfile of the application. The dummy application uses the extension's Gemfile. We can't change that one. That's true both when installing on new or existing stores.
• Conceptual: Since the release of solidus_starter_frontend, Solidus recommends not adding storefront code to extensions. The reason is that storefronts get customized, and it's impossible (as we experienced in the past) to be flexible enough to accommodate everything. Instead, we now recommend encapsulating the core domain and exposing it through a clear API, plus documenting how to integrate it in a storefront.

However, we're still in the process of evaluating and understanding all possible consequences of that. Do you think developing your extension in those terms would be viable?

[benjaminwil]

While I appreciate the recommendation that extensions should not add storefront code, this is not acceptable for all types of extensions. In this case: we're developing a payment method integration and we want to write end-to-end checkout feature tests to a) ensure that our payment method is compatible with Solidus and b) give confidence to users of the extension that the integration is battle-tested against a "real" Solidus checkout.

Another possible approach is to still provide some generic frontend code that is installed with a generator (like we do with the starter-frontend) that can then be customized in the store.

Thoughts on this angles to the problem?

Yes, I thought that maybe we could just use solidus_starter_frontend for this, but because the starter copies over tests and Ruby dependencies into the Gemfile–that means it can't reuse solidus_starter_frontend. We'd have to create something like solidus_dummy_frontend: the starter frontend views without all of the other stuff. Perhaps this could be a flag that solidus_starter_frontend provides: then, we could reuse all of the views and not have to maintain a second frontend.

Similarly, I think that solidus_starter_frontend providing a flag that only copies views could be useful in other circumstances. For example, if I do not want to use rspec-rails or some other dependency and only want starter views.

[waiting-for-dev]

I guess the main problem is that even if you target solidus_starter_frontend, what you really would be supporting is vanilla solidus_starter_frontend, but that's not what any serious store uses after some development.

I wonder if the fact that you feel the need to test all the checkout flow is a smell that we need extension points. I mean, for a payment extension, we shouldn't worry about addresses, shipments, etc. It'd be nice if Solidus offered a better way for extensions to hook into the checkout flow. Testing an extension against the minimal granularity it needs would be enough as we should well test the whole process in Solidus itself.

Definitely, this is something we still need to figure out, so all ideas are more than welcome.

[jarednorman]

Yeah, we're going to have to find a solution here. We need a way to run automated integrations tests for these extensions, even if the installation method we use for them isn't what most stores will end up using if they've customized things.

[benjaminwil]

It would be great to not have to test payment integrations against a vanilla storefront. But, in the case of payment method integrations:

• Payment service providers often require customer browser data to be sent back to them during the checkout process.
• Payment service providers almost always provide a JavaScript library so that the store's servers never have to store customer payment-related PII.

I also think it's important to end-to-end test multi-step checkout steps, like 3DS2 challenges.

I mean, for a payment extension, we shouldn't worry about addresses, shipments, etc.

This is not always true:

• Some payment methods require a billing address.
• We want to test "capture on ship" functionality.

[elia]

Since this is a common problem among payment-related extensions we could simply have an official checkout extension, and offer that to stores that aren't interested in taking the burden of customizing it, not far from what is provided by Shopify. If that was a thing it would be much easier to provide an out-of-the-box solution and at the same time leave the ability to fully customize everything else.

That is also one finding that came out of the extensions survey, the fact that it's been the go to solution for multiple services (not just Shopify, but also some payment providers) to me is a signal that's the right direction.

If there's consensus around this we keep testing on solidus_frontend until solidus_checkout is available.