Browsertrix Cloud 1.5.1

What's Changed

• Removes webkit <summary> element triangle by @Shrinks99 in #852
• Fix workflow edit back button by @SuaYoo in #857

Full Changelog: v1.5.0...v1.5.1

Browsertrix Cloud 1.5.0

Notes

This release introduces some crawling challenges. Crawls should be stopped before migrating due to completely different crawl setup.
CronJobs are migrated from crawlers namespace to main. If upgrading from previous version, may need to uninstall and reinstall helm chart (data will remain intact as long as volumes are not touched).

Major Changes

• Switch to new operator controller for CrawlJobs and ProfileJobs.
• New integrated workflow / crawling UI, grouping crawls by crawl config
• Improved error reporting via logs

What's Changed

• Add crawl /log API endpoint to stream crawler logs by @tw4l in #682
• Remove CI by @stavares843 in #760
• Upgrade to mongo 6 and use sortArray for workflow crawls (#764) by @ikreymer in #765
• Add crawl timeout nightly test by @tw4l in #762
• Docs: Font additions, style updates, and icon implementation by @Shrinks99 in #758
• Update collections backend API by @tw4l in #759
• backend: add 'lastCrawlStartTime' and 'lastStartedByName' fields to c… by @ikreymer in #753
• Add crawl errors endpoint by @tw4l in #757
• crawlconfig api: add 'currCrawlState' and 'currCrawlTimeStart' to crawlconfig list api (already queried on backend) by @ikreymer in #770
• Make btrix helper work with microk8s by @tw4l in #768
• Fix workflow total size by @tw4l in #783
• nginx: enable worker processes autotune to correctly set the number o… by @ikreymer in #785
• Revert docs body font to Inter by @Shrinks99 in #790
• Refactor to use new operator on backend by @ikreymer in #789
• Backend: App Startup Fixes by @ikreymer in #793
• Frontend crawl workflows rework by @SuaYoo in #775
• Parse JSON-l errors before returning by @tw4l in #799
• Removes extra padding around replay window & small file list style update by @Shrinks99 in #801
• Change copy text button → icon button by @Shrinks99 in #806
• Minor Frontend Visual Updates by @Shrinks99 in #810
• Improves icon button aria labels by @Shrinks99 in #809
• Crawling page control bar layout improvements by @Shrinks99 in #802
• Fix workflow limit empty values being set to 0 by @SuaYoo in #795
• Refactor screencast IDs by @SuaYoo in #800
• Fix crawl /errors API endpoint by @tw4l in #813
• Improve Recursive font declaration by @Shrinks99 in #791
• Crawl detail page update by @Shrinks99 in #808
• Adds bug report issue template by @Shrinks99 in #797
• Show critical errors in Crawl detail logs by @SuaYoo in #811
• add authsign block to microk8s playbook by @ikreymer in #776
• Set stop-crawl in redis from operator when crawl is stopped by @tw4l in #815
• Update scale in workflow when crawl scale is updated by @tw4l in #820
• Add option to list workflows API endpoint to filter by schedule by @tw4l in #822
• Ensure Volumes are deleted when crawl is canceled by @ikreymer in #828
• Crawl workflow detail page improvements by @SuaYoo in #823
• Improve crawl detail files list truncation by @Shrinks99 in #830
• Allow users to set max depth/hop out within scope by @SuaYoo in #816
• Pre-compute workflow last crawl information by @tw4l in #812
• Update precompute migration to only update active workflows by @tw4l in #833
• operator and config update fixes: by @ikreymer in #834
• stopping fix: backend fixes for #836 + prep for additional status fields by @ikreymer in #837
• crawlconfig: fix default filename template, make configurable by @ikreymer in #835
• Add Waiting state on the backend and frontend by @ikreymer in #839
• frontend crawl stopping improvements (#836) by @ikreymer in #838
• Add stop crawl confirmation dialog by @SuaYoo in #841
• backend: ensure cid is a UUID, remove unneeded inactive check on crawls by @ikreymer in #842
• backend: fixes to 0005 migration: by @ikreymer in #843
• Improve Workflow Detail tab and button UX by @SuaYoo in #840
• Fix superadmin running crawls views by @SuaYoo in #846
• Fix workflow edit page not loading by @SuaYoo in #848

Full Changelog: v1.4.1...v1.5.0

Browsertrix Cloud 1.5.0 Beta 4

What's Changed

• Fix workflow edit page not loading by @SuaYoo in #848

Full Changelog: v1.5.0-beta.3...v1.5.0-beta.4

Browsertrix Cloud 1.5.0 Beta 3

What's Changed

• Improve Workflow Detail tab and button UX by @SuaYoo in #840
• Fix superadmin running crawls views by @SuaYoo in #846

Full Changelog: v1.5.0-beta.2...v1.5.0-beta.3

Browsertrix Cloud 1.5.0 Beta 2

What's Changed

• operator and config update fixes: by @ikreymer in #834
• stopping fix: backend fixes for #836 + prep for additional status fields by @ikreymer in #837
• crawlconfig: fix default filename template, make configurable by @ikreymer in #835
• Add Waiting state on the backend and frontend by @ikreymer in #839
• frontend crawl stopping improvements (#836) by @ikreymer in #838
• Add stop crawl confirmation dialog by @SuaYoo in #841
• backend: ensure cid is a UUID, remove unneeded inactive check on crawls by @ikreymer in #842
• backend: fixes to 0005 migration: by @ikreymer in #843

Full Changelog: v1.5.0-beta.1...v1.5.0-beta.2

Browsertrix Cloud 1.5.0 Beta 1

What's Changed

• Removes extra padding around replay window & small file list style update by @Shrinks99 in #801
• Change copy text button → icon button by @Shrinks99 in #806
• Minor Frontend Visual Updates by @Shrinks99 in #810
• Improves icon button aria labels by @Shrinks99 in #809
• Crawling page control bar layout improvements by @Shrinks99 in #802
• Fix workflow limit empty values being set to 0 by @SuaYoo in #795
• Refactor screencast IDs by @SuaYoo in #800
• Fix crawl /errors API endpoint by @tw4l in #813
• Improve Recursive font declaration by @Shrinks99 in #791
• Crawl detail page update by @Shrinks99 in #808
• Adds bug report issue template by @Shrinks99 in #797
• Show critical errors in Crawl detail logs by @SuaYoo in #811
• add authsign block to microk8s playbook by @ikreymer in #776
• Set stop-crawl in redis from operator when crawl is stopped by @tw4l in #815
• Update scale in workflow when crawl scale is updated by @tw4l in #820
• Add option to list workflows API endpoint to filter by schedule by @tw4l in #822
• Ensure Volumes are deleted when crawl is canceled by @ikreymer in #828
• Crawl workflow detail page improvements by @SuaYoo in #823
• Improve crawl detail files list truncation by @Shrinks99 in #830
• Allow users to set max depth/hop out within scope by @SuaYoo in #816
• Pre-compute workflow last crawl information by @tw4l in #812
• Update precompute migration to only update active workflows by @tw4l in #833

Full Changelog: v1.5.0-beta.0...v1.5.0-beta.1

Browsertrix Cloud 1.5.0 Beta 0

Major Changes

• Switch to new operator controller for CrawlJobs and ProfileJobs
• New integrated workflow / crawling UI, grouping crawls by crawl config

What's Changed

• Add crawl /log API endpoint to stream crawler logs by @tw4l in #682
• Remove CI by @stavares843 in #760
• Upgrade to mongo 6 and use sortArray for workflow crawls (#764) by @ikreymer in #765
• Add crawl timeout nightly test by @tw4l in #762
• Docs: Font additions, style updates, and icon implementation by @Shrinks99 in #758
• Update collections backend API by @tw4l in #759
• backend: add 'lastCrawlStartTime' and 'lastStartedByName' fields to c… by @ikreymer in #753
• Add crawl errors endpoint by @tw4l in #757
• crawlconfig api: add 'currCrawlState' and 'currCrawlTimeStart' to crawlconfig list api (already queried on backend) by @ikreymer in #770
• Make btrix helper work with microk8s by @tw4l in #768
• Fix workflow total size by @tw4l in #783
• nginx: enable worker processes autotune to correctly set the number o… by @ikreymer in #785
• Revert docs body font to Inter by @Shrinks99 in #790
• Refactor to use new operator on backend by @ikreymer in #789
• Backend: App Startup Fixes by @ikreymer in #793
• Frontend crawl workflows rework by @SuaYoo in #775
• Parse JSON-l errors before returning by @tw4l in #799

Full Changelog: v1.4.0...v1.5.0-beta.0

Browsertrix Cloud 1.4.1

Bug fix release: Update to mongo 6 to support $sortArray operation, don't use JS in mongo due to lack of support by DigitalOcean.

What's Changed

• Upgrade to mongo 6 and use sortArray for workflow crawls by @tw4l in #764

Full Changelog: v1.4.0...v1.4.1

Browsertrix Cloud 1.4.0

Major Features

• Crawl Configs are now mutable, instead of creating a duplicate config
• Crawl Configs are now renamed to Workflows. (workflow revisions are tracked internally but not yet displayed)
• Support for latest Browsertrix Crawler 0.9.0 which sorts crawl queue by depth, uses playwright for crawling

What's Changed

• Add icons to crawl details navigation buttons by @Shrinks99 in #666
• Paginate API list endpoints by @tw4l in #659
• Fix crawl config name in "run now" alert by @SuaYoo in #673
• Fix missing crawl config name by @SuaYoo in #683
• backend: make crawlconfigs mutable! (#656) by @ikreymer in #662
• Limit organization name length by @SuaYoo in #671
• permissions: allow user with 'viewer' permissions to access read-only… by @ikreymer in #687
• Improve crawl queue pagination UX by @SuaYoo in #680
• Migrate crawl config frontend -> workflow by @SuaYoo in #686
• backend: Fix for total crawl time limit. by @ikreymer in #665
• Add lightweight logging mode by @leepro in #668
• exclusions editor fix: by @ikreymer in #692
• Fix saving config limit and browser setting fields by @SuaYoo in #704
• Disable copy tags menu item if no tags by @SuaYoo in #709
• Combine watch crawl with crawl queue by @SuaYoo in #710
• backend: update queue apis to work with new sorted queue apis (also b… by @ikreymer in #712
• Add proofread action CI by @stavares843 in #714
• Add optional description to crawl configs by @tw4l in #707
• Allow users to set workflow description by @SuaYoo in #708
• Remove new issue project automation by @SuaYoo in #718
• Add Playwright UI tests + CI by @stavares843 in #614
• chore(playwright): fix version by @stavares843 in #725
• Fix migration to avoid jobType KeyError by @tw4l in #727
• Leave trailing slash in seed URLs by @SuaYoo in #731
• fix version related to @playwright/test by @stavares843 in #729
• Filter and sort crawl and workflow list API endpoints in backend by @tw4l in #724
• Add README.md related to run playwright tests locally by @stavares843 in #722
• Allow configurable max pages per crawl in deployment settings by @ikreymer in #717
• Add pageSize to pagination output format by @tw4l in #736
• Update nightly test fixtures to use Seed objects by @tw4l in #734
• Max page limit override by @ikreymer in #737
• misc frontend build fixes: playwright version + chunking by @ikreymer in #740
• Set max pages to API default by @SuaYoo in #739
• No More Workflowuration by @ikreymer in #741
• fix: only include finished crawls in crawlCount value for /api/crawlconfigs by @ikreymer in #746
• config: add 'pageLoadTimeout' and 'pageExtraDelay' options to backend… by @ikreymer in #742
• Crawls list backend pagination by @SuaYoo in #735
• Frontend Docker build improvements by @SuaYoo in #749
• Allow users to set additional page time limits by @SuaYoo in #744
• Fix additional URLs by @SuaYoo in #752
• Add btrix CLI dev helper by @tw4l in #732
• Configure crawler disk utilization threshold via helm chart by @tw4l in #748
• Docs: adds mkdocs features, adds theming by @Shrinks99 in #728
• Inverts the autoscroll behavior setting to true by default by @Shrinks99 in #756
• Adds inputmode attributes to workflow config fields by @Shrinks99 in #755

Full Changelog: v1.3.1...v1.4.0

Browsertrix Cloud 1.4.0 Beta 2

What's Changed

• Add proofread action CI by @stavares843 in #714
• Add optional description to crawl configs by @tw4l in #707
• Allow users to set workflow description by @SuaYoo in #708
• Remove new issue project automation by @SuaYoo in #718
• Add Playwright UI tests + CI by @stavares843 in #614
• chore(playwright): fix version by @stavares843 in #725
• Fix migration to avoid jobType KeyError by @tw4l in #727
• Leave trailing slash in seed URLs by @SuaYoo in #731
• fix version related to @playwright/test by @stavares843 in #729
• Filter and sort crawl and workflow list API endpoints in backend by @tw4l in #724
• Add README.md related to run playwright tests locally by @stavares843 in #722
• Allow configurable max pages per crawl in deployment settings by @ikreymer in #717
• Add pageSize to pagination output format by @tw4l in #736
• Update nightly test fixtures to use Seed objects by @tw4l in #734
• Max page limit override by @ikreymer in #737
• misc frontend build fixes: playwright version + chunking by @ikreymer in #740
• Set max pages to API default by @SuaYoo in #739
• No More Workflowuration by @ikreymer in #741
• fix: only include finished crawls in crawlCount value for /api/crawlconfigs by @ikreymer in #746
• config: add 'pageLoadTimeout' and 'pageExtraDelay' options to backend… by @ikreymer in #742
• Crawls list backend pagination by @SuaYoo in #735
• Frontend Docker build improvements by @SuaYoo in #749
• Allow users to set additional page time limits by @SuaYoo in #744
• Fix additional URLs by @SuaYoo in #752
• Add btrix CLI dev helper by @tw4l in #732
• Configure crawler disk utilization threshold via helm chart by @tw4l in #748
• Docs: adds mkdocs features, adds theming by @Shrinks99 in #728

Full Changelog: v1.4.0-beta.1...v1.4.0-beta.2