Installation Issue on Arch-Based Distros: Workaround

[Aezore]

Description

Issue:

I encountered a problem installing CaringCaribou on Arch-based Linux distributions like Artix or Manjaro following the provided installation instructions. The primary issue appears to be related to how system packages are managed externally in Arch-based systems, which conflicts with the installation process described in the documentation.

Environment:

• Operating System: Arch Linux / Manjaro (or any other Arch-based distribution)
• Python Version: 3.11
• CaringCaribou Version: Current master branch at a38e1bc commit

Expected Behavior:

The installation process should complete without errors, allowing CaringCaribou to be installed and run on Arch-based systems as described in the documentation.

Actual Behavior:

The installation fails due to conflicts arising from externally managed system packages.

Steps to Reproduce:

1. Follow the installation instructions provided in the CaringCaribou documentation for Arch-based systems.
2. Observe the failure in the installation process, particularly related to conflicts with system package management.

Workaround:

I managed to install i by modifying the pyproject.toml file and creating a bespoke PKGBUILD file. Here are the steps I followed for the workaround:

1. Modify pyproject.toml: pyproject.toml

2. Create a Bespoke PKGBUILD:

pkgname=caringcaribou
pkgver=0.5
pkgrel=1
pkgdesc="A friendly automotive security exploration tool"
arch=('any')
url="https://github.com/Aezore/caringcaribou"
license=('GPL3')
depends=('python' 'python-can' 'python-pip' 'python-build')
branch="archlinux"
source=("https://github.com/Aezore/${pkgname}/archive/${branch}.tar.gz")
sha256sums=('SKIP')

build() {
  cd "$srcdir/${pkgname}-${branch}"
  python -m build
}

package() {
  cd "$srcdir/${pkgname}-${branch}"
  # Find the generated wheel file
  local wheel=$(ls dist/*.whl)
  # Install the wheel using pip
  python -m pip install "${wheel}" --root="${pkgdir}" --ignore-installed --no-deps
}

4. Build and Install: After making these changes, I was able to build and install it locally by running makepkg -si in the directory containing the modified PKGBUILD file. I forked the repo to do the changes locally in my machine thus the source url is my repo and the branch is "archlinux"

Suggested Solution:

While the workaround mentioned above is effective, it would be beneficial to support Arch-based distributions more seamlessly. Potential solutions could include:

• Adjusting the installation process to accommodate the package management peculiarities of Arch-based systems.
• Providing an official PKGBUILD or instructions for creating one, facilitating easier installation for users of these distributions.

DISCLAIMER:

I'm by no means any skilled programmer or software engineer nor work for any company so i'm far from best practices and I used GPT to aid me in the "learning process" so take this with a grain of salt. I thought it might come in handy for someone else with the same issue. I also used chatGPT to aid me in formatting this issue report as it's my very first time "fixing" something like this and I don't have any past experience or anyone else near to ask. Apologies if this bothers any of you.

[kasperkarlsson]

Hi there @Aezore,

I am not an Arch user myself (and hence unable to provide any support), but I am glad to hear that you managed to get CC up and running at last 😅 I will leave this issue open and hope that it

1. helps other Arch users who need to perform a similar dance before hacking ECUs
2. attracts a skilled Arch magician who knows what to do, and is able to submit a PR either fixing the issue or adding suitable documentation

Thanks for reporting and good luck with your research!

[Aezore]

Thanks for your hasty reply. For the sake of completion I'll try my best to get it properly sorted by looking at how the main repositories handle Python scripts and then make my very first PR. We can discuss then how to make it fit within code guidelines and behavior. I noticed too that while compiling it threw some deprecation warnings about the pkg_resources lib but I'll have to take a look at that in a different branch so it still conforms to Python2 aswell. Thanks again

[Aezore]

Hi @kasperkarlsson,

After thoroughly investigating the issue, I've come up with a proposed solution that involves the architectural separation of package management files from the main project repository. My suggestion is to create a dedicated repository specifically for Arch package manager files. This "recipe" would detail the process for fetching data, preparing it, do testing, building, and installing, etc. This approach is motivated by the observation that this code does not directly pertain to the core functionality of the project and would therefore be better managed separately. The intention is to then publish these files to the Arch User Repository (AUR), which is the community-managed repository.

For installing a package from the AUR, users can leverage AUR helpers like yay or paru. These tools automate the installation process (e.g., yay -S caringcaribou).

Additionally, to accommodate this change and inadvertently enhance the package, I propose a minor modification to the pyproject.toml file. This modification is essential for ensuring a successful build process. The change involves specifying the build backend, which is supported by setuptools for both setup.py and pyproject.toml configuration files. Although the use of setup.py is generally discouraged, it remains supported. The proposed change is in my updated archlinux branched pyproject.toml, not the previously discussed one.

It's important to note a minor breaking change: the TOML file parser encounters issues with dots in variable names. Consequently, I've dropped the cc.py entry point with caringcaribou. An alternative, such as renaming it to ccpy or another variant, is possible as long as it avoids using dots.

I have conducted tests across three distinct docker containers: Python 2.7, Python 3.12.2, and the latest ArchLinux image along with its Python 3.11.8 package from the official Arch repository at time I'm writing this. The installation process completed successfully in all three environments. I can supply all three Dockerfiles for you to test aswell. Right now it's pulling from my forked repo.

The files are present in Dockerfiles

While I have no prior experience in publishing or maintaining a package in the AUR, I am willing to learn and take responsibility for this task.

I look forward to your feedback on this proposal.

[kasperkarlsson]

Great! Thanks a lot for investigating this and coming up with a solution!

As for the dot in the installed name, this should really not be necessary anymore and was rather just kept for legacy reasons. I will have to come back to this (and, most likely, remove it from this repo as well). Hopefully I will have some time to look into this next week (and feel free to ping me if you feel that I am too slow 😅)

[kasperkarlsson]

Hi again @Aezore 👋

I just switched the preferred command from cc.py to caringcaribou in the 0.6 release, as it really makes no sense to include the file extension for an installed tool. It used to be different back when it was just a standalone script, of course, and I guess this is why I never removed it 🙂 The only reason I did not drop the cc.py completely right now was for legacy reasons, as other users may have shell scripts etc with hardcoded calls to cc.py rather than caringcaribou.

I am, however, considering some significant changes to the tool, including (finally) dropping the Python 2 support. If removing the cc.py mapping is strictly needed for Caring Caribou to work smoothly under Arch, I reckon this could be done at the same time as part of this year's spring-time cleaning 😄 Is there anything else you think I should keep in mind when doing this, for the sake of making the Arch magic easier to implement or maintain?

[Aezore]

Hi @kasperkarlsson ,

I reckon that the dot issue has more to do with the toml parser than a distribution problem. I added more metadata and the build backend option to the pyproject.toml to make it more conformant with the way it seems people builds Python code nowadays. It would also work in our favour in order to publish it in the pypi package index so for a non arch user would simply mean pip install caringcaribou and maybe set up a github action that builds and publish a new release on each push automagically.

The Arch PKGBUILD file is mostly a "grab this .tar source code, unpack it and run python -m build && pip install" but those instructions can be changed to whatever it needs to; sort of a "bash" script but what seems to be enforced to run under the package manager to keep track. It doesn't have any source code (it has the capability to get it on demand upon installation) so that's why I suggested to just have a separate git repo with that file. I guess it will complain too if I try to install a global system package from node although I haven't tried yet.

I think another solution would be if caringcaribou is installed in a virtual environment as that is also allowed (since it's not global anymore). I can take a deeper look at it and will report back. Perhaps we could set it up to let the user either run it under a venv with pipenv or poetry, or install it globally by releasing both .deb and PKGBUILD and that should cover pretty much everyone (I suppose that if a user is running something else then already knows the ropes).

As a final word I also support the choice to drop Python 2 support. I fully respect the decision to keep it all this time; it fully covers all user base regardless of technical constraints. I think cc already has a very handy set of tools and porting to Python 3 would mean a cleaner codebase. I suggest moving it to a separated py2 branch and if a tool is developed that serves a new important role it could be backported if it's just a "semantics" difference (eg. using existing variables in different ways)

Thanks for having my words into consideration. Being a "programmer for fun" and having this interaction means a lot for me.

[obbardc]

I included the pyproject.toml changes in #109 !

[kasperkarlsson]

#109 has now been merged (and it seems to work well in a venv setup, according to @obbardc).

Does this help resolve this issue in any way? Any idea what further changes are needed, if any?