Merge branch 'add-retroarch-frontend' of https://github.com/SineSwiper/skyscraper into SineSwiper--add-retroarch-frontend

Author: Gemba

VERSION.ini

@@ -1 +1 @@
-VERSION="3.19.0-dev"
+VERSION="3.19.0"

docs/CHANGELOG.md

@@ -3,16 +3,73 @@
 This page summarizes the changes of each Skyscraper release, a [changlog for
 humans](https://keepachangelog.com).
 
-### Version 3.19.0 (2026-TBA)
-
+### Version 3.19.0 (2026-03-16)
+
+- Added: BBC Micro is now supported out-of-the-box as
+  [platform](https://github.com/Gemba/skyscraper/blob/23ca6fda9402513df45febe0e21e6041b116b067/peas.json#L369).
+  Spin up your BBC emulator, like
+  [b-em](https://github.com/FollyMaddy/RetroPie-Share/blob/main/00-scriptmodules-00/emulators/b-em-a5.sh)
+  or [libretro
+  b2](https://github.com/Gemba/RetroPie-BSides/blob/main/docs/Libretro_bbcmicro.md)
+  (since 3.18.4)
+- Added: Ability to [disable colored terminal
+  output](CLIHELP.md#disable-terminal-colors) by setting the environment
+  variable `NO_COLOR=1`. Coloring will now automagically disabled when using
+  file redirection of stdout or running Skyscraper via a serial link.
+- Added: [JSON
+  Schema](https://github.com/Gemba/skyscraper/blob/master/supplementary/scraperdata/peas-schema.json)
+  file for platform definition file of Skyscraper and a [Python
+  script](https://github.com/Gemba/skyscraper/blob/master/supplementary/scraperdata/peas_validate_with_json_schema.py)
+  to validate your `peas_local.json` file against it. _RetroPie users_: Please
+  update the skyscraper scriptmodule in the RetroPie-Setup first before updating
+  Skyscraper.
+- Added: Workaround for ES-DE's potential invalid gamelist: ES-DE places in some
+  conditions an `<alternativeEmulator/>` as second root element to a
+  `gamelist.xml` file. Any serious XML parser will refuse this file as non
+  standard-conformant XML, so did Skyscraper/Qt. Thanks for reporting in the
+  first place and testing on macOS @DFelten
+- Added: Mitigation of Screenscraper.fr long running responses on some games
+  causing games not to be scraped (since 3.18.4), thanks for reporting
+  @saitamasahil, @ZacharyFoxx and @therealpxc
+- Added: CLI option [`--stderr`](CLIHELP.md#--stderr) which explicitly prints a
+  one liner to stderr when Skyscraper runs into an error condition in addition
+  to the stdout output. Parsing this single line makes integrations into other
+  programs more convenient (leaning towards
+  [Scappy](https://github.com/gabrielfvale/scrappy)).
+- Added: Build evaluates
+  [`SYSCONFDIR`](https://github.com/Gemba/skyscraper/blob/23ca6fda9402513df45febe0e21e6041b116b067/skyscraper.pro#L33)
+  environment variable to allow [filesystem hierarchy
+  standard](https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard)
+  compliant builds and packaging (since 3.18.3). Thanks @Newchair2644.
+- Changed: Add ellipsis (...) on truncated text description, when
+  [`maxlength`](CONFIGINI.md#maxlength) is reached.
+- Changed: Initially implemented logic from 2019 which removes cache files when
+either parameter (m _or_ t) of `--cache purge:m=...,t=...` are matched. Updated
+implementation removes resource if and only if both parameters (m _and_ t) are
+matched, cf. thread in [RetroPie Forum](https://retropie.org.uk/forum/post/306255) (since 3.18.3)
+- Changed: More pleasant replacement for colons (:) in texts for Pegasus frontend.
+  Now uses the Unicode counterpart (Modifier Letter Colon, 0xa789) (since 3.18.3). Thanks
+  @LeeBigelow
+- Changed: Changes in The GamesDB API implied some re-write of Skyscraper's TGDB
+  scraper module. See paragraphs on changed [API limits and private key
+  usage](https://gemba.github.io/skyscraper/SCRAPINGMODULES/#thegamesdb-tgdb)
+  (since 3.18.5), thanks to the reporters @rudism, @saitamasahil and
+  @BrandonShega!
+- Updated: Docker container can now digest a host provided `config.ini` and
+  [several improvements](https://github.com/Gemba/skyscraper/pull/221). Kudos to
+  @rudism
+- Updated: Stricter `--cache` command validation to avoid invalid combinations.
 - Updated: ZXInfo scraper now checks first for file hash match, allow overriding
   with query parameter or hinting with release year in parenthesis in filename or
   via `aliasMap.csv` (since 3.18.2)
-- Changed: More pleasant replacement for colons in texts for Pegasus frontend.
-  Now uses the Unicode counterpart (Modifier Letter Colon, 0xa789). Thanks
-  @LeeBigelow. 
-- Fixed: Filename output name of cache operations, thanks @SineSwiper (since
-  3.18.2)
+- Fixed: Some glitches by cache commands which can operate without a platform
+  given (since 3.18.4). These cache features were added in 3.15.0.
+- Fixed: Reverted logic to ignore media during esgamelist ingest, unless they
+are explicitly set (e.g., `--flags video`) (since 3.18.3); regression from
+3.18.0, cf. thread in [RetroPie Forum](https://retropie.org.uk/forum/post/306218), thanks for reporting
+@s1eve-mcdichae1
+- Fixed: Filename output name of cache operations (since
+  3.18.2), thanks @SineSwiper
 - Fixed: A set of edgecase bugs, thanks to all reporters.
 
 ### Version 3.18.1 (2025-12-23)
@@ -153,8 +210,7 @@ the place to put ideas or to table Skyscraper puzzles.
   report:missing](CLIHELP.md#-cache-reportmissingall-textual-artwork-media-or-resource1resource2),
   [cache validate](CLIHELP.md#-cache-validate), [cache
   vacuum](CLIHELP.md#-cache-vacuum) and [cache
-  purge](CLIHELP.md#-cache-purgekeywordmodule-andor-type). Thanks #2, @Leukhos
-  the addition is very appreciated.
+  purge](CLIHELP.md#-cache-purgekeywordmodule-andor-type). Thanks #2, @Leukhos.
 - Fixed: Improvements to the Pegasus frontend, to avoid inconsistency with
   existing aliases in the frontend output for the keywords `command/launch` and
   `workdir/cwd`. Thanks, @Leukhos for your third submission.

docs/CLIHELP.md

@@ -23,6 +23,20 @@ On _non_ RaspiOS-based RetroPie-Installments put the file from
 
 In either case: Open a new bash and press ++tab++ key twice to see it in action.
 
+### Disable Terminal Colors
+
+By default Skyscraper will use ANSI terminal colors. However, colors will be not
+be used in the output when you run Skyscraper via a serial console or you apply
+file redirection e.g., `Skyscraper ... > out.log`. In any case you can manually
+enforce no color output by setting the environment variable `NO_COLOR` to a
+non-zero value (a value of zero will show colors). For example:
+
+```bash
+NO_COLOR=1 Skyscraper ...
+```
+
+The [no-color.org](https://no-color.org) website has additional information.
+
 ## Short Options
 
 The most prevalent short options you will use are most likely [`-s`](#-s-module)

docs/FRONTENDS.md

@@ -25,10 +25,10 @@ Skyscraper will preserve the following metadata when re-generating a game list f
 
     Folder data is not cached by Skyscraper, thus if you delete your `gamelist.xml`, Skyscraper cannot restore the edited folder elements from cache.
 
-Automatic addition of folder elements if [`addFolder`](CONFIGINI.md#addfolders) is true:  
+Automatic addition of folder elements if [`addFolder`](CONFIGINI.md#addfolders) is true:
 If at least one ROM is within a subfolder and this subfolder is not yet part of the `gamelist.xml` file, it will be added with two mandatory subelements:
 
--  `<path/>` reflects the relative subpath from the system folder and 
+-  `<path/>` reflects the relative subpath from the system folder and
 -  `<name/>`, which represents the direct parent folder of a ROM by default. However, you may edit this to any name which should be shown in EmulationStation.
 
 !!! example
@@ -81,7 +81,7 @@ files into the `downloaded_media` folder (e.g.
 `~/ES-DE/downloaded_media/<PLATFORM>/screenshots/` for screenshots) from where
 ES-DE will pick them up. ES-DE does not support textures currently. Any manual
 or fanart data present in the cache will be put automagically into
-`~/ES-DE/downloaded_media/<PLATFORM>` where ES-DE will load it.  
+`~/ES-DE/downloaded_media/<PLATFORM>` where ES-DE will load it.
 This frontend automatically enables the output of backcovers, fanart and manuals
 during gamelist creation, whenever cached data is present for a game.
 
@@ -116,7 +116,7 @@ Over time the fork of EmulationStation in Batocera has diverted stark from the
 initial EmulationStation. Thus, the gamelist format is somewhat different, but
 the textual scraping elements are still the same. Most notably Batocera
 EmulationStation stores a lot of additional information in the gamelist, and the
-set of information may differ for each game.  
+set of information may differ for each game.
 Skyscraper aims to cover the most frequent used elements for scraping, currently
 fanarts, manuals and videos. See the [scraping
 modules](SCRAPINGMODULES.md#capabilities-of-scrapers) for support of these
@@ -143,7 +143,7 @@ every `<PLATFORM>` you scrape.
 -   Default media file location: `/userdata/roms/<PLATFORM>/{images,videos,manuals}`
 
 If you set a game list location and do not specifiy the ROM folder (input
-folder) and media folder, then these are set relatively to the game list folder.  
+folder) and media folder, then these are set relatively to the game list folder.
 This frontend automatically enables the output of backcovers, fanart and manuals
 whenever cached data is present for a game.
 
@@ -217,7 +217,7 @@ Windows desktop users can use SMB shares and can adapt the following steps.
    ```
    You may also set the default [frontend in the
    configuration](CONFIGINI.md#frontend), then you can even drop the `-f` CLI
-   parameter.  
+   parameter.
    If you want to use your existing Batocera gamelist and mediafiles (to save
    some online scrape cycles), you can do so by using the
    [esgamelist](SCRAPINGMODULES.md#emulationstation-style-gamelists) scraping
@@ -283,9 +283,46 @@ You need to add the individual platform rom directories to Pegasus (if they are
 
 !!! info
 
-    If you are generating game lists for Pegasus, it is highly recommended to disable third-party game list data sources! Otherwise you will have a mish-mash or different sources showing up in Pegasus. Start the Pegasus frontend, press ESC on the keyboard and choose _Settings_ -> _Enable/disable data sources_ and disable everything in that submenu.  
+    If you are generating game lists for Pegasus, it is highly recommended to disable third-party game list data sources! Otherwise you will have a mish-mash or different sources showing up in Pegasus. Start the Pegasus frontend, press ESC on the keyboard and choose _Settings_ -> _Enable/disable data sources_ and disable everything in that submenu.
     Then reload the game lists or restart Pegasus, and all of the platforms should show up with media and game information generated by Skyscraper.
 
 #### Metadata preservation
 
-Skyscraper will preserve any metadata key-value pairs added to the header and / or individual game list entries.
+Skyscraper will preserve any metadata key-value pairs added to the header and / or individual game list entries.
+
+### RetroArch
+
+-   Default game list location: `/opt/retropie/configs/all/retroarch/playlists`
+-   Default game list filename: `<DB_NAME>.lpl` (using its own platform name format)
+-   Default media dir location: `/opt/retropie/configs/all/retroarch/thumbnails/<DB_NAME>/Named_*`
+
+Despite the presence of all of these other fancy frontends, the lower-level RetroArch layer that manages most of your emulator cores has a perfectly-capable GUI in the form of a playlist creator/viewer.  The playlists contain each platform's game list, and the playlist viewer has support for covers, screenshots, and logos for each ROM.
+
+#### Configuration
+
+When generating for RetroArch, Skyscraper uses a built-in mapping of platform names to RetroArch database names (e.g., `nes` → `Nintendo - Nintendo Entertainment System`).  If we don't have it in our system, this can be easily added in `platforms_idmap.csv`.
+
+You can optionally use the `-e` parameter with `frontendExtra="<CORE_PATH>;<CORE_NAME>"` to set a default core path/name for the playlist.  Or set it in `config.ini`:
+
+```ini
+[retroarch]
+frontendExtra="<CORE_PATH>;<CORE_NAME>"
+```
+
+#### Media Support
+
+RetroArch supports the following media types:
+
+| Media Type | RetroArch Directory |
+| :---------- | :-------------- |
+| Covers (Box Art) | `Named_Boxarts` |
+| Screenshots | `Named_Snaps` |
+| Marquees/Wheels (Logos) | `Named_Logos` |
+
+All media files are matched to games by their sanitized game title, not by ROM filename.  Title screenshots (`Named_Titles`) are not currently supported.
+
+Media may not all be in PNG formats.  You should turn on the _Settings_ -> _Playlists_ -> _Allow All Supported Image Types for Thumbnails_ option in your RetroArch config to support JPEG image displays.
+
+#### Metadata preservation
+
+Skyscraper will preserve existing game titles and paths when re-generating a game list for RetroArch. If an existing playlist file is found and you choose to skip existing entries, Skyscraper will use the old game list as a reference.

hints.xml

@@ -33,14 +33,17 @@
   <hint>Skyscraper supports the ES-DE frontend gamelist format.</hint>
   <hint>You can add new platforms (i.e., systems to scrape) by editing configuration files. See https://gemba.github.io/skyscraper/PLATFORMS</hint>
   <hint>You can scrape data from GameBase DB files with Skyscraper. GameBase DBs are a well of information especially for indie or homebrew games for retro systems.</hint>
-  <hint>Skyscraper supports ten scraping modules: 7 for online sources and 3 for local sources.</hint>
+  <hint>Skyscraper supports ten scraping modules: Seven for online sources and three for local sources.</hint>
   <hint>Pizza and beer go well together.</hint>
   <hint>Skyscraper debuted at June, 11th 2017.</hint>
   <hint>Skyscraper runs on Linux, macOS and Windows.</hint>
   <hint>You can use relative paths for almost every path related option. See https://gemba.github.io/skyscraper/PATHHANDLING</hint>
   <hint>For a file extension/suffix option value like '*.ext', you can use the shorter forms '.ext' or even only 'ext'.</hint>
-  <hint>The supplementary/ folder contains helper scripts to support you in the advanced Skyscraper configuration.</hint>
+  <hint>The supplementary/scraperdata folder contains helper scripts to support you in the advanced Skyscraper configuration.</hint>
   <hint>You can scrape fanart and backcovers with Skyscraper, check the respective options.</hint>
   <hint>The flag --buildinfo provides useful information when reporting an issue.</hint>
   <hint>Default built-in extensions for game files are: *.m3u, *.ml, *.zip and *.7z</hint>
+  <hint>You can disable the colored terminal output by setting NO_COLOR=1 as environment variable.</hint>
+  <hint>You can validate your platforms definition file (peas_local.json) with the script peas_validate_with_json_schema.py of Skyscraper.</hint>
+  <hint>Albet Einstein's birthdate was on the pi day (3/14/1879), Karl Marx died on the pi day of 1883.</hint>
 </hints>