Description
Hi!
I would be interested in handling the rewriting of the test suite for GSoC.
I saw the experimental start and would like to add some highlights for the questions raised in the PR, and in addition to the GSoC summary:
Is it worthwhile? There are a lot of tests to convert
In addition to make debugging easier, it could also lower the entry barrier for potential contributors/reviewers.
As mentioned in the summary, sh
is harder to debug but also to review, with sometimes surprising behavior.
Also, if we rewrite the tests, duplications due to the lack of parameterization and inheritance would probably not make the number of tests to convert a big deal.
Are the C bindings worth the trouble for all functions? I initially
planned to call the function-under-test directly with C, like the
original pjdfstest. But Python's os module is very close to C. For
example, os.symlink is basically the same thing as symlink(2) except
that it accepts Python paths as arguments instead of C strings, and it
raises exceptions on error rather than setting errno.
It depends on the language that will be finally chosen, but for example, Python implements almost all POSIX syscalls with calling conventions very similar to those of C.
From what I have checked, the only missing that are present in pdjfstest.c
are these ones:
- openat
- unlinkat
- mkdirat
- linkat
- symlinkat
- renameat
- mkfifoat
- mknodat
- bind
- bindat
- connectat
- fchmodat
- fchownat
- fchflags
- chflagsat
- fstatat
- lpathconf
- utimensat
(The bold functions have tests.)
Though, to be fair, most of those -at
functions are replaced by the dir_fd
parameter (except for bindat
and connectat
).
So, the only functions which would potentially need bindings are lpathconf
, bindat
and connectat
.
This would also allow us to improve the documentation, which for the moment (no offense!) is rather scarce.
Language choice
Python
Pros
- Easier to review
- Syscalls availability (
os
module) - Larger community
Cons
- Performance: Python is definitely not the best in terms of performance, but does it really matter for our case, since we mostly test IO?
- Setup
Rust
Pros
- Performance
- Binary: Tests could be compiled into one binary and distributed this way, instead of having to setup an environment
- Fearless concurrency
- Safety
Cons
- Rely on external dependency: not sure of how much this is a con, but to use syscalls, we cannot rely on the stdlib since it provides no guarantees (and wasn't built for this). But we can use for example nix or rustix.
- Community: Not really a big problem (I'm myself a huge Rust advocate!), but the community is rather small compared to the others
Go
Pros
- Syscalls availability (
syscall
module) - Performance
- Binary: Same as for Rust
Cons
- Not so fearless concurrency
- Not so safe (debatable?)
ATF integration
I maybe misunderstood something, but I'm not sure of why is it needed to support ATF, given the fact that kyua
supports TAP (and even plain) tests and that it seems to produce reports even for non-ATF tests.
A Kyuafile
would probably be sufficient to output reports.
Activity