Changes for relative path handling of configuration options:

- Path functions added to config.cpp
- Pegasus frontend refactored
- Enhanced Unit-Tests
- Documentation PATHHANDLING.md
- Minor (unrelated to rel path) fixes for edgecases

Author: Gemba

.gitignore

@@ -55,6 +55,7 @@ compile_commands.json
 
 moc_*
 *.tmp
+*.bkp
 *.bk
 .qmake*
 Skyscraper
@@ -64,6 +65,7 @@ packages
 target
 build
 peas_local.json
+platforms_idmap_local.csv
 
 # macOS
 .DS_Store

config.ini.example

@@ -22,15 +22,17 @@
 ; Also remember to uncomment or create the sections they belong to such as '[snes]'.
 
 [main]
-;inputFolder="/home/pi/RetroPie/roms"
+;gameListFolder="/home/pi/RetroPie/roms"
+# next two are relative to 'gameListFolder'
+;inputFolder="."
+;mediaFolder="./media"
 ;excludePattern="*[BIOS]*"
 ;includePattern="Super*"
-;excludeFrom="/home/pi/.skyscraper/excludes.txt"
-;includeFrom="/home/pi/.skyscraper/includes.txt"
-;gameListFolder="/home/pi/RetroPie/roms"
+# relative to this config.ini path, also for other path parameters
+;excludeFrom="./excludes.txt"
+;includeFrom="./includes.txt"
 ;gameListBackup="false"
-;mediaFolder="/home/pi/RetroPie/roms"
-;cacheFolder="/home/pi/.skyscraper/cache"
+;cacheFolder="./cache"
 ;cacheResize="false"
 ;nameTemplate="%t [%f], %P player(s)"
 ;jpgQuality="95"
@@ -39,7 +41,7 @@
 ;cacheWheels="true"
 ;cacheMarquees="true"
 ;cacheTextures="true"
-;importFolder="/home/pi/.skyscraper/import"
+;importFolder="./import"
 ;unpack="false"
 ;frontend="emulationstation"
 ;emulator=""
@@ -73,6 +75,7 @@
 ;hints="false"
 ;subdirs="true"
 ;spaceCheck="true"
+# only absolute path allowed
 ;scummIni="/full/path/to/scummvm.ini"
 ;tidyDesc="true"
 ;onlyMissing="false"

docs/CHANGELOG.md

@@ -5,6 +5,11 @@ humans](https://keepachangelog.com).
 
 ### Version 3.17.0 (2025-TBA)
 
+- Changed: Handling of relative path in configuration options adapted to enable
+  that gamelist, ROM folder and media folder can be moved easier around or can
+  be moved to other devices. Also, the config INI file can be provided with a
+  relative path, as all other path options. See details [here](PATHHANDLING.md).
+  Thanks to @cameronhimself for nudging me.
 - Added: Artwork finetuning, added option to [control aspect ratio when
   scaling images](ARTWORK.md#aspect-attribute-o_1). Thanks to
   @joyrider3774 for the suggestion.
@@ -32,7 +37,7 @@ humans](https://keepachangelog.com).
   RetroPie-Setup first before updating Skyscraper.
 - Added: Documentation on scraper modules supplied with [scraper
   capabilities](SCRAPINGMODULES.md#capabilities-of-scrapers).
-- Added: Documentation on options for --query parameter per scraper modules. See
+- Added: Documentation on options for `--query` parameter per scraper modules. See
   [here](SCRAPINGMODULES.md#recognized-keywords-in-query).
 - Added: IGDB scraper now supports Screenshot and Cover scraping, plus it
   allows querying with game ID `--query="id=..."`.

docs/CONFIGINI.md

@@ -18,7 +18,7 @@ Settings in the `[main]` section will always be set regardless of selected platf
 
 `<SCRAPER>` can be any of the supported scraping modules (check list with `--help` under the `-s` option), in which case the settings will only be applied while scraping with that particular scraping module.
 
-Each section can have overlapping parameters. In case where a certain option exists in several sections they are prioritized as scraping module first, then frontend, then platform and lastly main. Any commandline option which relates to an configuration setting in `config.ini` has highest precedence, regardless of the other four levels respective sections.
+Each section can have overlapping parameters. In case where a certain option exists in several sections they are prioritized as scraping module first, then frontend, then platform and lastly main. Any commandline (CLI) option which relates to an configuration setting in `config.ini` has highest precedence, regardless of the other four levels respective sections.
 
 You can find an example config file at `/home/<USER>/.skyscraper/config.ini.example`. This file contains all available options. Just copy the file to `config.ini` and uncomment and edit the ones you wish to use by removing the `#` or `;` in front of the variables. Remember to also uncomment the section the option relates to such as `[main]` or `[amiga]`.
 
@@ -812,7 +812,7 @@ Allowed in sections: `[main]`, `[<PLATFORM>]`, `[<FRONTEND>]`
 
 #### relativePaths
 
-Enabling this forces the rom and any media paths inside the game list to be relative to the rom input folder. Currently only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list (see also [frontend](#frontend) option).
+Enabling this forces the rom and any media paths inside the game list to be relative to the path of the gamelist file. Currently only relevant when generating an EmulationStation, a Retrobat or a Pegasus game list (see also [frontend](#frontend) option).
 
 !!! info
 

docs/PATHHANDLING.md

@@ -0,0 +1,109 @@
+## Path Logic in Skyscraper
+
+This page describes how Skyscraper processes the different paths and especially
+how the absolute path is calculated when a you provide a relative path.
+
+Do not get confused by the lenghty flow diagram below. It covers gamelist folder,
+input folder and media folder handling. You wiil notice that input folder and
+media folder are processed in the same manner.
+
+<figure markdown style="width:100%">
+  <img style="width:100%" src="../resources/path_handling_flow.svg"/>
+  <figcaption>Path processing flowchart</figcaption>
+</figure>
+
+Ah, good you made it to the end of the flow.
+
+Pay attention to the connectors _A-A_, _B-B_ and _C-C_ in the diagram.
+
+_Normalization_ means any surplus `./` and `../` are removed from the path.
+However, at this stage it is not verified if the filepath does exists. Also, any
+symbolic links are not resolved.
+
+Remember the precedence of [CLI and configuration
+options](CONFIGINI.md#configini-options) when you read through this document.
+
+!!! tip "In a Nutshell"
+
+    Key takeaway is that a relative input folder and relative media folder are
+    always calculated to an absolute folder in relation to the absolute gamelist
+    folder. When you apply [`relativePaths="true"`](CONFIGINI.md#relativepaths),
+    they will be generated with the relative path in respect to the gamelist-
+    or metadata-output (in Pegasus terms) path.
+
+Now, let's see how Skyscraper handles relative path configuration options.
+
+### Computing the Absolute Path from ...
+
+a relative path provided. The next subsections are summarizing the absolute path
+calculation of the different path and file options.
+
+#### Current Working Directory
+
+The current working directory (CWD) is the directory from where you run
+Skyscraper. The absolute path is computed as `<CWD>/<option-value>` for these
+CLI options and their values:
+
+`-a <artwork.xml>`  
+`-c <configfile>`  
+`-g <gamelistfolder>`  
+`-d <cachefolder>`
+
+This means the first part of the flow diagram (Gamelist folder) also applies to
+`-a` and `-d` options and their configfile counterparts.
+
+In contrast, when you provide these three (`<artwork.xml>`,`<gamelistfolder>`
+and `<cachefolder>`) in a configuration INI-file the absolute path it determined
+from the absolute path of the config file. See
+[below](#an-option-in-a-configuration-ini-file).
+
+#### Skyscraper Built-in Config Directory
+
+This is usually `/home/<USER>/.skyscraper/` (base). With [XDG](XDG.md) it is
+slightly different. The files provided with these options
+
+`--excludeFrom <excludes.txt>`  
+`--includeFrom <includes.txt>`
+
+are searched in that directory by concatenating the base and for example the
+exclude file. If not found, Skyscraper tries to access them with the current
+working directory as base.
+
+#### the Gamelist Folder
+
+If you define a Gamelist folder either via `-g` or via `gameListFolder=`
+(INI-file) this is the base for the relative paths of
+
+`-i <folder>` or `inputFolder=<folder>`  
+`-o <folder>` or `mediaFolder=<folder>`
+
+This is also depicted in the diagram above.
+
+#### Input Folder (ROM-/gamefile-path)
+
+The files provided with these options
+
+`--startAt <game-file-A>`  
+`--endAr <game-file-B>`
+
+are first searched in the current working directory. If not found,
+Skyscraper tries to access them in the input folder.
+
+#### an Option in a Configuration INI File
+
+When you have set one of the following parameters in the default configuration
+file (`/home/<USER>/.skyscraper/config.ini`) or in a custom config file defined
+with `-c <configfile>` the path is calculated from the absolute path of
+the location of the config INI-file.
+
+`artworkXml=<artwork.xml>`  
+`gameListFolder=<gamelistfolder>`  
+`cacheFolder<cachefolder>`
+`importFolder<importfolder>`
+
+!!! note
+
+    The `importFolder=` parameter has no counterpart on the command line.
+
+Remember, that this path calculation is different than the logic when using [CLI
+options](#current-working-directory).

docs/USECASE.md

@@ -16,7 +16,10 @@ Now, let's dig in!
 
 You probably installed Skyscraper in order to get some data and artwork on your frontend game lists. So let's say that you have a bunch of Super Nintendo files you wish to populate with data. You'd first gather some data into the Skyscraper resource cache and then generate a game list using the cached data afterwards. So it's a two-step process.
 
-![Skyscraper flowchart](resources/skyscraper_overview_chart.png)
+<figure markdown>
+  ![Skyscraper flowchart](resources/skyscraper_overview_chart.png)
+  <figcaption>Skyscraper flowchart</figcaption>
+</figure>
 
 #### The Gathering Phase
 

docs/XDG.md

@@ -17,18 +17,20 @@ has a brief section on the topic.
 
 ### File Locations: Genuine- and XDG-Skyscraper
 
-|   Type   | Genuine Skyscraper Loation                                            | XDG BDS Location                       |
-| :------: | --------------------------------------------------------------------- | -------------------------------------- |
-|  CONFIG  | All files, but no subfolders of `~/.skyscraper/*`                     | `$XDG_CONFIG_HOME/skyscraper/`         |
-|  CACHE   | All of `~/.skyscraper/cache/`                                         | `$XDG_CACHE_HOME/skyscraper/`          |
-|  IMPORT  | All of `~/.skyscraper/import/`                                        | `$XDG_DATA_HOME/skyscraper/import/`    |
-| RESOURCE | All of `~/.skyscraper/resources/`                                     | `$XDG_DATA_HOME/skyscraper/resources/` |
-|   LOG    | Skipped files/games during processing `~/.skyscraper/skipped*.txt`    | `$XDG_STATE_HOME/skyscraper/`          |
-|  REPORT  | Cache reports `~/.skyscraper/reports/*`                               | `$XDG_STATE_HOME/skyscraper/reports/`  |
-|   LOG    | Rare Screenscraper Error Log `~/.skyscraper/screenscraper_error.json` | `$XDG_STATE_HOME/skyscraper/`          |
-
-As the per specifications: All paths must be absolute, relative paths will be
-ignored by Skyscraper. Non-existing paths will be created by Skyscraper.
+|   Type   | Genuine Skyscraper Location                                          | XDG BDS Location                       |
+| :------: | -------------------------------------------------------------------- | -------------------------------------- |
+|  CONFIG  | All files, but no subfolders of `~/.skyscraper/*`                    | `$XDG_CONFIG_HOME/skyscraper/`         |
+|  CACHE   | All of `~/.skyscraper/cache/`                                        | `$XDG_CACHE_HOME/skyscraper/`          |
+|  IMPORT  | All of `~/.skyscraper/import/`                                       | `$XDG_DATA_HOME/skyscraper/import/`    |
+| RESOURCE | All of `~/.skyscraper/resources/`                                    | `$XDG_DATA_HOME/skyscraper/resources/` |
+|   LOG    | Skipped files/games during processing `~/.skyscraper/skipped*.txt`   | `$XDG_STATE_HOME/skyscraper/`          |
+|  REPORT  | Cache reports `~/.skyscraper/reports/*`                              | `$XDG_STATE_HOME/skyscraper/reports/`  |
+|   LOG    | Rare Screenscraper error log `~/.skyscraper/screenscraper_error.txt` | `$XDG_STATE_HOME/skyscraper/`          |
+
+!!! warning "Skyscraper is strict"
+
+    As per specification: All paths must be absolute, relative paths will be
+    ignored by Skyscraper. Non-existing paths will be created by Skyscraper.
 
 ### How to Enable XDG
 
@@ -43,8 +45,8 @@ Skyscraper will be deploy its configuration files to the XDG destinations (by
 default sourced from `/usr/local/etc/skyscraper`).
 
 If you want to verify if Skyscraper is compiled with XDG support run `Skyscraper
---version`. If you see a XDG label below the version it means XDG is enabled and
-supported.
+--version`. If you see a XDG label below the version number it means XDG is
+enabled and supported.
 
 To disable XDG support undo the steps above.
 

docs/resources/path_handling_flow.drawio

@@ -29,11 +29,13 @@
   <hint>You can pinpoint a game with the Mobygames scraper with "-s mobygames --query &lt;gameid&gt;", where the &lt;gameid&gt; is the ID listed on the mobygames site at the game's detail view.</hint>
   <hint>Skyscraper on Linux supports bash-completion (aka. programmable completion): Check the documentation at https://gemba.github.io/skyscraper/CLIHELP/#programmable-completion for details. Once enabled press TAB twice to display completion options.</hint>
   <hint>You can scrape game manuals with Skyscraper and display them if your frontend supports this. Use manuals=true in config.ini.</hint>
-  <hint>You can make Skyscraper use the XDG Base Directories on Linux systems. See the XDG.md documentation.</hint>
+  <hint>You can make Skyscraper use the XDG Base Directories on Linux systems. See the https://gemba.github.io/skyscraper/XDG documentation.</hint>
   <hint>Skyscraper supports the ES-DE frontend gamelist format.</hint>
-  <hint>You can add new platforms (i.e., systems to scrape) with configuration file edits. See PLATFORMS.md.</hint>
+  <hint>You can add new platforms (i.e., systems to scrape) with configuration file edits. See https://gemba.github.io/skyscraper/PLATFORMS</hint>
   <hint>You can scrape data from GameBase DB files with Skyscraper. GameBase DBs are a well of information especially for indie or homebrew games for retro systems.</hint>
   <hint>Skyscraper supports ten scraping modules: 7 for online sources and 3 for local sources.</hint>
   <hint>Pizza and beer go well together.</hint>
   <hint>Skyscraper debuted at June, 11th 2017.</hint>
+  <hint>Skyscraper runs on Linux, macOS and Windows.</hint>
+  <hint>You can use relative paths for almost every path related option. See https://gemba.github.io/skyscraper/PATHHANDLING</hint>
 </hints>