Is there an easy way to preview the new article?

[f-hollow]

Originally posted by @tore-espressif in #37 (comment)

As of now the only way to preview a new article in a PR is:

1. Check out the pull request locally.
2. Make sure you have Hugo installed locally and, in the repo folder, run hugo server.

Automate the preview process by building the PR branch with Hugo and deploying it using GitHub Actions:

• How to automate previews for Github Pages
• Using GitHub Actions to build automagic Hugo previews of draft articles

[f-hollow]

Summary

The standard page_url for a repo is <username>.github.io/<reponame>. Changing this page_url is not supported, which means that you cannot publish more than one branch from the same repo to githuab pages. I tried two workarounds without luck. Probably, there are more, but these are still workarounds for now. Also, there is a feature request: Deploy preview option for PRs that seems to be dormant.

Details

To make preview useful, we should be able to change page_url, so that we can have multiple versions of the portal for different PRs.

The standard deployment action actions/deploy-pages, can only publish to a standard URL <username>.github.io/<reponame>. The proposed workaround does not work.

# Sample workflow for building and deploying a Hugo site to GitHub Pages
name: Deploy Hugo site to Pages

on:
  # Runs on pushes targeting the default branch
  push:
    branches:
      - main

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
  contents: read
  pages: write
  id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
  group: "pages"
  cancel-in-progress: false

# Default to bash
defaults:
  run:
    shell: bash

jobs:
  # Build job
  build:
    runs-on: ubuntu-latest
    env:
      HUGO_VERSION: 0.128.0
    steps:
      - name: Install Hugo CLI
        run: |
          wget -O ${{ runner.temp }}/hugo.deb https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_linux-amd64.deb \
          && sudo dpkg -i ${{ runner.temp }}/hugo.deb
      - name: Checkout
        uses: actions/checkout@v4
        with:
          submodules: recursive
          fetch-depth: 0

      - name: Setup Pages
        id: pages
        uses: actions/configure-pages@v5
      - name: Install Node.js dependencies
        run: "[[ -f package-lock.json || -f npm-shrinkwrap.json ]] && npm ci || true"
      - name: Build with Hugo
        env:
          HUGO_CACHEDIR: ${{ runner.temp }}/hugo_cache
          HUGO_ENVIRONMENT: production
        run: |
          hugo \
            --gc \
            --minify \
            --baseURL "${{ steps.pages.outputs.base_url }}/"
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v3
        with:
          path: ./public

  # Deployment job
  deploy:
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v4

The alternative action actions-gh-pages, does not work. I tried changing branches, and I always get remote: Permission to <username>/developer-portal.git denied to github-actions[bot].

      - name: Deploy
        uses: peaceiris/actions-gh-pages@v4
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: ./public

[igrr]

We can probably use preview-docs.espressif.com for this. For a PR, an action would build the site and SCP it over to preview-docs.

This would probably be limited to authors who can push branches directly to this repo. It wouldn't work if a PR is done from a fork, because the forked repository wouldn't have access to secrets (e.g. SSH keys) required for deployment. Still, we could add all the regular authors here as contributors and at least for them, the workflow would be simplified.

[f-hollow]

@hfudev also shared this:

Worth checking:

• GitHub action: Deploy PR Preview
  • Downside of this project it that, it only support "branch pr", not "fork pr"
• Netlify's Deploy Preview
  • To enable "full preview" for fork pr, use netlify instead

Accepting fork PRs can pose security threats:

• Github Actions and the threat of malicious pull requests
• Why doesn't PR Preview Action support preview from forks?

---

We can try Netlify and see if it introduces any vulnerabilities. If yes, we can go with either Deploy PR Preview or preview-docs.espressif.com suggested by @igrr

[f-hollow]

Another interesting approach:

https://stackoverflow.com/questions/70620223/deploying-hugo-onto-staging-website-from-feature-branch

[f-hollow]

Summary of work done

Out of the implementation ideas mentioned in the comments above, all had their drawbacks or downsides. I tried registering with Netlify. This service opened my account and immediately blocked it with a request to share my ID and other wild things.

Eventually, I came to realize that our best bet is to use the S3 bucket for previews.

Implement preview on the S3 bucket

This was my initial list of things to consider

• (done) Deploy on preview-developer.com/pr<PR-number>
• (done) Make sure that repo secrets do not get leaked
  • (done) On PR, include the button to verify by maintainer before running the preview-deploy-site workflow
• (done) Set robots.txt to request crawlers against indexing the website
• (done) Remove pr<PR-number> upon closing or merging PR
• (done) Introduce configs for develop and production
  • (done) Set up a testing environment
• (done) Make sure that CloudFront is set up to deploy separate websites from each pr directory
  • (done) The pr folders should be mapped to preview-developer.espressif.com

All this has been largely implemented in the merged PRs. Along the way a number of issues were fixed and additional features implemented.

#348

• Added the workflow pr-deploy-preview
• Added the workflow pr-remove-preview
• Introduced separate configs for staging and products
• Cleaned up Hugo configs

After merge, I still had issues previewing changes from fork PRs. The workflow didn't have access to GitHub secrets because it is triggered with pull_request. The easy fix would be to share repo secrets with fork repos by using pull_request_target, which was unacceptable.

#359

• Split pr-deploy-preview into pr-build-preview and pr-deploy-preview: pr-build-preview only built the Hugo website, while pr-build-preview deployed the files to S3 bucket in the content of the main repo. This secured the repo secrets.

Still the previews didn't work the way I expected.

#347

• Fixed the issue of saving artifacts and sharing them between fork-side pr-build-preview and main-repo-side pr-deploy-preview
• Added a proper baseURL

After merging this PR, a few PR previews appeared just fine, but #178 showed under pr (instead of pr178) after rebasing on main. The PR number wasn't passed correctly due to a known issue.

Testing environment

The PRs #348, #359, and #347 were implemented without a proper testing environment, and the tests were done after merge. After merging #347, it was clear that the remaining issues required a testing environment to avoid merging more half-tested PRs.

In espressif/developer-portral, I created a develop branch. This approach didn't work well for my issues, because the event workflow_run only works on the default branch (in our case main). Eventually, I resorted to doing all tests in my fork f-hollow/developer-portal by adding secrets there and merging to main. It appeared to be a fully-functional testing environment!

#361

• Implemented passing PR number between pr-build-preview and pr-deploy-preview via a text file in artifacts. This proved to be much more reliable that using built-in GitHub Actions variables.

#367

This PR dealt with less urgent but nevertheless important task of cleaning up after PR merge or close.

• The workflow pr-remove-preview failed to run correctly on fork PRs due to unavailable secrets. This workflow was split into pr-close-cleanup and prw-remove-preview, the latter is run in the main repo environment.
• Added a step to remove all comments with PR preview link upon PR close or merge, because the previews get deleted and the link will only yield the 404 error.

#374

This PR dealt with some residual issues:

• So far, a new comment with PR preview link was added upon each push or new commits. To avoid polluting the PR page, now only one such comment is kept at the end of the timeline.
• A link back to the PR is added in the header of each PR preview site.
• The buttons on the home page didn't work. For them to work with a baseURL containing a subdir, there was the need to remove leading slashes in href of the button shortcode.
• As part of this PR, there was an attempt to fix the issue with the article shortcode (I would say this is a bug in the Blowfish theme). It fails to work if the baseURL contains a subdir. The go templating code appeared to be a bit overwhelming, so I didn't manage to fix it yet.

[f-hollow]

@pedrominatel @georgik All the merged PRs have added the necessary functionality to create previews, notify about them, and remove previews. This issue can be closed.

Feel free to open new issues for bug reports and new feature requests.