[feature]: Publish develop branch on npm

[paales]

Is your feature request related to a problem? Please describe.
I'm trying to install @magento/venia-concept via npm but my node version is 12 which isn't allowed by the 6.0.1 version, but is already in the develop branch.

Describe the solution you'd like
Publish the latest branch of pwa-studio after each commit on npm with a @develop tag.
https://www.npmjs.com/package/@magento/venia-concept

So I can do yarn add @magento/venia-concept@develop to always have the latest version available.

Describe alternatives you've considered
I'm now trying to use npm link to do it that way, but would be nice if other developer in the team don't have to do that.

Please let us know what packages this feature is in regards to:

• venia-concept
• venia-ui
• pwa-buildpack
• peregrine
• pwa-devdocs
• upward-js
• upward-spec
• create-pwa

[m2-assistant]

Hi @paales. Thank you for your report.
To help us process this issue please make sure that you provided sufficient information.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[tjwiebell]

@paales - This is an interesting idea, and we have most of our publication automated, so let me check with the team if this is something that could be done easily. I assume we could just follow React's style of publishing version 0.0.0-<latest-commit-hash> to avoid worrying about real versioning. Ideally I would like to see us releasing more often, but I won't get into why that's not possible ATM.

I'm also wondering if this fits better into scaffolding. We have an existing command that lets you simulate an install using the latest code in the monorepo; its not well documented, but have you experimented with doing DEBUG_PROJECT_CREATION=1 pwa-studio/packages/create-pwa/bin/create-pwa? I can see how scaffolding every time to receive updates would be cumbersome, but I think there's a long term vision to make a scaffolding update utility to ease that pain.

If I'm being honest, this will likely fall low in priority since we don't have a full time CI/CD engineer, but I'll see if there's anything we could do with minimal effort that makes it easier to get latest develop without interacting with the monorepo.

[paales]

Thanks for your response, very much appreciated! I've got a few questions / points:

1. Are the publication steps open source in this repo, or is that private somewhere, if so, can that be made public? In the spirit of 'open development'?

2. Some form of release would be nice. I do understand that we can directly work with the latest GitHub version with npm link, but that's not always what we are looking for (especially team members might not be looking for that).

3. As for releasing more often, that would probably be already be a major step in the right direction. Sorry to bring up NextJS, but they do their releases very organized: Create a new canary every few days, release every few weeks.

   To achieve that I'd say that the releases should be automated as much as possible, else it'll become to much of a chore and nobody wants to do it.

4. We're not using the create-pwa as a basis, so that probably doesn't work for us? Will have to take a look at it though.

5. About low priority; What does that mean exactly? Maybe communication could be a little bit more clear here. Do you mean:

   • "We consider this out of scope for the project at the time, you are welcome to provide us with more information for us to re-evaluate in the future."?
   • "We are excited about the proposed feature, but we don't have the bandwidth right now to do this ourselves. If you can provide us with a pull request, we are willing to merge this (if the quality is on par)"

Automation solution:

We are using https://semantic-release.gitbook.io/semantic-release/ a lot ourselves which allows you to create development releases easily on the develop branch + create release notes + publish on NPM + reply release information to related PR's. It can be configured in such a way

To implement such a thing, there are two steps:

• Release configuration: https://github.com/ho-nl/docker-development-box/blob/master/package.json#L25-L84
• GitHub release action: https://github.com/ho-nl/docker-development-box/blob/master/.github/workflows/release.yml

[tjwiebell]

1. Are the publication steps open source in this repo, or is that private somewhere, if so, can that be made public? In the spirit of 'open development'?

This is now private. There's a secret exposure vector that we have to protect if we expose the repository that's actually used, but perhaps we could expose a public fork that's kept in sync. Will review with the team.

5. About low priority; What does that mean exactly? Maybe communication could be a little bit more clear here. Do you mean:

Our internal priorities align with our roadmap, which is heavily in favor of feature parity work; so I would say more of the latter. There's a bigger team conversation that needs to happen internally about versioning before we can even think about establishing a release cadence, or fully automating that process, but all this information you've provided will help kick off the conversation, thank you.

Will provide the next update after grooming tomorrow.

[awilcoxa]

Created PWA-662

[fooman]

@paales I am currently experimenting with something like this

diff --git packages/pwa-buildpack/lib/cli/create-project.js packages/pwa-buildpack/lib/cli/create-project.js
index 2e2c9963..6750b4c1 100644
--- packages/pwa-buildpack/lib/cli/create-project.js
+++ packages/pwa-buildpack/lib/cli/create-project.js
@@ -220,11 +220,11 @@ module.exports.handler = async function buildpackCli(argv) {
     }
 
     if (process.env.DEBUG_PROJECT_CREATION) {
-        prettyLogger.info('Debug: Removing generated tarballs');
+        prettyLogger.info('Moving generated tarballs');
         const pkgDir = require('pkg-dir');
         const monorepoDir = resolve(pkgDir.sync(__dirname), '../../');
         prettyLogger.info(
-            execa.shellSync('rm -v packages/*/*.tgz', { cwd: monorepoDir })
+            execa.shellSync('mv packages/*/*.tgz ..', { cwd: monorepoDir })
                 .stdout
         );
     }

And then running something like
DEBUG_PROJECT_CREATION=true ./pwa-studio-dev-packages/packages/pwa-buildpack/bin/buildpack create-project tmp-to-delete --template "venia-concept" --name "tmp-to-delete" --author "Kristof Ringleff, Fooman <kristof@example.com>" --backend-url "https://master-7rqtwti-mfwmkrjfqvbjk.us-4.magentosite.cloud/" --braintree-token "sandbox_8yrzsvtm_s2bg8fs563crhqzk" --npm-client "yarn" --install 0. You then end up with a bunch of tgz files which you can reference like "@magento/venia-ui": "file:path-to-packages/magento-venia-ui-3.0.0.tgz", or possibly even a hosted version via url. That way this only needs to be done once centrally and the rest of the team doesn't have to perform any of those steps.

Agreed that ultimately some sort of npm repository based canary packages would be easier.

[paales]

Our current process:

1. Clone this repo to some folder on your system.
2. Run yarn link in each package directory.
3. It will now output the command you can use in your project to link them.

This works fine in our case, but again, this is not always what we want.

[fooman]

fyi just built something interim https://github.com/fooman/pwa-studio-canary/actions which all going well should build and publish packages weekly from the develop branch of pwa-studio.

Current week (24) outputs:https://canary-packages.fooman.com/pwa-create/package-2020-24.json and includes the references to install from url.

[zetlen]

Publishing nightly edge releases should be managed by Support and CICD (if they decide it should be done). Tip @dpatil-magento :)