Official Debian packaging

[Fantu]

Hi, I took a very quick look to assess the possibility of packaging it for the Debian and derivative repositories.
Debian requires that all dependencies be included in the repository (with rare exceptions); unfortunately, it seems there are missing dependencies (which need to be packaged first), and looking at the GitHub build workflow, it also downloads precompiled binaries (which is unacceptable in Debian).
There are also several parts that require different languages and multiple components, including Rust, which I’m not familiar with and which seems to require a separate procedure.
However, I saw that the Rust part has just been moved to its own separate repository, right? So that should simplify the packaging a bit.

More in details required thing seems:

• tinyxml2: already in debian but with one RC bug about an ABI break in latest version (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1100661), so need to check the compatibility
• mongoose added recently in Debian: https://tracker.debian.org/pkg/mongoose
• Miniz: missed in Debian but there is an ITP https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1124745
• check_nsclient: if required should be packaged first (not checked rust deps if all already present in Debian)
• about web frontend I have not checked all deps in web/package.json: should check whether the corresponding node-* packages are available in Debian and ensure there are no problematic components; otherwise, we can include all the necessary source files here (tolerated)
• python part: need to check if present in Debian the deps in build/python/requirements.txt

other thing:

• .net: seems needed only in windows or I'm wrong?
• Boost: from a fast look seems all already included with libboost-all-dev
• lua: liblua5.4-dev present and already used also here

[mickem]

General

I guess it would be possible to package a "stripped down" agent only version to reduce effort so for monitoring: you install the packages and if you want admin (UI) features and some exotic module you use the packages built here?

Tinyxml

This is used by core and NRDP module and is just an xml reader/writer if there other libs for that I guess one could explore moving to them or look into the ABI breakage, it is not used much so should not be hard either way.

Mongoose

This is used only the web server is something I am exploring swapping out as it is not fit for purpose.
The challenge here is that mongoose compiles easily on anything where as many other lib do not.
One candidate for this is to split so we use boost-beast for "modern windows" and linux and mongoose for XP.
In this case it would be swapped out for boost, but I have not explored this yet.

miniz

This is zip functionality so should be easy to swap for another library and there are numerous (use by core I think).

check_nsclient

This is "optional" and intended to be a separate module so the plan is to package that separately and not include it in the "nsclient-agent" packages. That said we could do a multi module builder wrapping the two repos with submodules which I assume would be acceptable.

Web frontend

Thats likely the hardest part it is react, MUI and vite so pretty much big contender so likely packaged already but a shortcut here would be to ship without web frontend and only package this as an "agent" Ii.e. monitoring only.

Python

Here we install some stuff required only by test packages. We also package and ship python and some modules "because python might be missing on windows" this is not required on linux so likely no packages would be required apart from protobuf (and if we want to build web-based docs mkdocs and mkdocs-material).
And arguably for linux it might be better to generate man pagaes instead.

.net

.net is currently not enabled so it is there but do not build. I have plans to fix this "at some point" one could use .net support on linux but not sure how meaningful it is.

Boost

Should be is included in the dist

Lua

Should be included in the dist

Also

We also have cryptopp which I currently think I ignore on linux this breaks NSCA an here I have been pondering giving up on all old and strange chipers and instead only do AES which I think we can do without cryptopp but alas not finished this yet...

[Fantu]

Thanks for the reply, maybe if I have time, I could try creating the base package by disabling certain components, such as the web server, documentation, tests, and check_nsclient. That way, I suppose I could the base packaging in a few hours.

Given that miniz’s packaging hasn’t seemed to have progressed in the last three months (based on the Salsa repository) and the stability of ABI is in question, replacement might be useful if quick and easy and you want. At a glance, libzip could be a possible candidate, but I haven’t done any in-depth research yet.

Disabling components in builds: At a quick glance, the ability to disable certain components appears to be missing or incomplete in the build:

• Web server: I don't see an option to completely disable it via a parameter or automatically if Mongoose isn't found
• Testing: It doesn't seem to support disabling it; I see something vague, perhaps is partial
• Documentation: This seems to be already skip it can't find any mkdocs

As for finding and using system packages, everything seems to be in order for lua, boost, protobuf, openssl, iconv, cryptopp, gtest, gmock.
Instead for tinyxml2, mongoose and miniz are missing (if miniz will be replaced, there will be no need to implement them for that part).

As for Python, it seems that for only the agent part is necessary: for the build python3-dev python3-protobuf libboost-python-dev; and runtime python3 python3-protobuf. So don't seems needed add or change for this part.

Supporting the disabling of components and the detection and use of system packages would likely also be useful for packaging in other distributions and for users who need quick custom builds of only specific components

[mickem]

Modules "should" degrade nicely i.e. not build if things are missing. This though only work for some things so feel free to improve that. I.e. if mongoose is not found it should not build the mongoose wrapper lib and the webserver.

Same thing for missing zlib we could likely disable the zipped modules support int hat case.
And so on and so forth.

All modules have a build.cmake (IIRC) that dictates if they should be included some of them have checks there (for instance windows only modules).
There are also feature that can be disabled like CRYPTOPP, LUA, and SSL so feel free to extend it that way.

Also feel free to add cmake options for "selecting which things to build" that would also be sensible and to be honest much of the cmake is "as is" rather then "because" so it has been evolving and mainly to solve my own issues rather then make it easy for others to build which is not a good strategy so improvements here.
All the cmake code is really due for a major refactoring work (just not fun and always a bother so it is always low priority :P)

I cant promise I can look into any of this this week (have a lot on my plate right now, but likely next weekend or week after that).
But feel free to take a stab at it, and create PR:s.. Id suggest doing them in iterations rather than as a big one, I'll try to comment/approve/merge quickly...

[Fantu]

@mickem I saw 2fb17b0 when you renamed the package nscp->nsclient, you will do also in linux packages and in Debian package I should do the same with the binary package? (on the source instead I'll keep nscp, same of repository)

[mickem]

Thats a good question... I think nscp is a bad name as it likely clashes with many things so in hindsight I regret naming the binary and packages nscp. So nsclient is probably a better name as thats the name of the "project" and website. But unfortunetly renaming things break things so sometimes it will not be possible/convienient.

I am not really sure about linux packages seems like renaming them nsclient "should" be straight forward but I honestly have no idea how upgrades handle that and such if we do "official packages" it is likely a good idea to name them nsclient (nsclint++ is also a bad name as ++ is not a-z)...

But if I wrer to restart I would likely name them:

• check-nsclient (nsclient-client?): For the rust client that can be used to check nsclient.
• nsclient (nsclient-agent?) for the monitoring agent
• nsclient-server (nsclient-admin) for a possible future managament UI to allow remote managament

But this is more of an idea... So I'd happily accept feedback here...

[Fantu]

Checking https://tracker.debian.org/ both nscp and nsclient are not taken as source, for the source I don't know if it can be useful to change, I was thinking of leaving nscp, unless you change it upstream.
The binary package name even if single can be different from source, but I would probably make multiple binary packages (in the Debian packaging) to do things better, for example a nsclient-common package to separate the independent arch files (unless there are almost none and insignificant size), to save space in the repositories (instead of duplicating for each supported arch). Then, when the documentation will be packaged, nsclient-docs package, max as recommended (and not dep.) avoiding installing it if not needed or make possible uninstalling it later. Plus any other separate parts such as the webserver part.
Let me know if the package names will be nsclient and nsclient-*, or if you'd like to do it differently. Currently, there are no packages with those names in Debian:
https://packages.debian.org/search?suite=sid&searchon=names&keywords=nsclient

However, I was thinking that, even if we stick with Debian packaging also here, separate packages complicate things (if they’re only included in the binaries available for download with the releases), so it might be better to keep a single package here. To better manage any migration to or from official packages, having a different name, and thus keeping nscp, might be the better.
Migration to different package name (or split) is supported and for example on rename in d/control should be added Breaks: and Replaces:
https://www.debian.org/doc/debian-policy/ch-relationships.html
However, I don't know if an old bug still present I found in some packages I maintain will create problems for the configuration management instead.
I don't even know if it will be good to leave persistent breaks/replaces and if they will accept it at the NEW revision (being related to packages external to Debian and for which normally the migration from/to unofficial packages is not supported)

Searching also in https://repology.org/ nscp and nsclient are not taken, it hadn't been packaged for distribution yet.

For Debian, however, a dfsg repack will definitely be necessary for it to be accepted (which should only affect the packaging, since you might need some of the precompiled files for Windows builds, I haven't checked that thoroughly yet)

[mickem]

First off the "multiple packages" I talked about above are not "this repo" it will be other repos (and standalone) so one would be the already moved out rust client: https://github.com/mickem/check_nsclient (this is intended to be installed on the "monitoring side" i.e. a way to check a remote nsclient agent this might make sense to package for debian but I guess rust might be hard to package for debian (given their affinity for offline build)...
And the other a completely separate project I have been pondering for a long time and a web project so likely not sensible to package for debian (given their affinity for offline build)...

That said it might make sense to have multiple packages for instance some of the clients like "CheckNSClient" (the old protocol) and OP5Client are likely not very interesting for most users. So it could make sense to have some of those broken into late say nsclient-rarely-used-modules out but at the same time they are not that big or harmful so here I would go with whats ideomatic and how "others" likes to package things.

Naming I would likely prefer nsclient over nscp but I will not rename the git repository (that would likely break links) I will probably rename the download binaries hosted here when I release 1.0 (which will likely happen in the next few months).
The plan is to make some long overdue breaking changes on Windows (i.e. move config file out of program files) and then it makes sense to rename the downloaded files at the same as people will have to make some manual changes anyway.

[Fantu]

Rust packaging on Debian is supported and seems to be growing steadily, so I don’t think it would be a problem. The only thing is that I have no experience with Rust or its packaging, so it would be better if someone else who’s an expert on the subject handled it. It would have been a problem if check_nsclient had stayed here, but since it’s separate and everything is in Rust, I suppose it’s not complicated.
As a general rule, all dependent packages must be included in Debian, not just Rust; therefore, building offline would not be a problem, in fact, it is the norm, since downloading during the build is prohibited, except in rare cases on some non-free packages, but is permitted only on non-free package if I remember good.

OT: About future changes in windows what do you mean "move config file out of program files"?

[mickem]

Great, I will look into that once I have finished all the changes I have here...

As for the windows changes... Today nsclient.ini is stored in the same folder as the binaries (i.e. c:\program files\nsclient++) this is painful (as msi does not really like it) so the idea is to move the config files, certs and such to %ProgramData% to make installer simpler and more reliable. I will also then enforce permissions on settings and certs to make things secure by default rather then secured by hand as well as teak the installer to make upgrades easier...
This is all a change that will likely require some manual touchups depending on how people have installed things so planning to do that for "1.0". But will fix up all the low hanging fruit first...

[Fantu]

now that I built successful I looked to continue the packaging but I saw that paths are strange and I think should be fixed improved (not only for Debian but also for major of distro and users using prefix /usr).
I ask claude to create a fix (I think can be improved):
Fantu@9e98b68

[mickem]

FIY main should no longer have dependecnies (except docs and web) now and I also just for fun asked AI to generate man pages in this branch: #1288 not sure if it is at all useful but that could be a wat out of "mkdocs".

[mickem]

Regarding paths: create a PR with that commit and I can look at it...

[Fantu]

In addition to the precompiled Windows-specific files that I'll exclude from the repack, I also saw this Lintian error:
E: nscp source: license-problem-old-unicode [include/simpleini/ConvertUTF.c]
ConvertUTF.c and ConvertUTF.h have a license incompatible with dsfg and seems used also on linux build, so that part should be replaced with another or package will be in "non-free" e not "main"

I tried to ask claude to found solutions, it showed 3 possible and I asked to try to create a patch for one that seems the "better":
Fantu@04c8fa2

I also tried to enable the build tests (for test_settings for the previous patch) and failed (not for the patch above), another patch:
Fantu@cbec5d9