build.7: Add an example for developing and testing userspace

[patmaddox]

Adding new functionality to userspace sometimes requires updating a lib and bin together. This example demonstrates how to build the lib, and use the build result for building the bin and running tests.

This example differs from "Example 3: Build and upgrade a single piece of userspace" because it demonstrates how a developer may quickly iterate on a piece of userspace before installing it to its final destination.

While here, update examples to consistently use /usr/src.

[github-actions]

Thank you for taking the time to contribute to FreeBSD!

Some of files have special handling:

Important

@concussious wants to review changes to share/man/

[concussious]

Hi Pat! Thanks so much for this, this is wonderful! I would be happy to take this with the following small tweaks:

[concussious]

Please put this after example 3 and increment subsequent example numbers

[patmaddox]

Done!

[concussious]

Suggested change

	cd /usr/src
	cd src

For consistency with the other examples

[concussious]

Oops, I'm wrong; looking closer I now think the ones that just say src should be aligned to how you wrote it.

[bsdimp]

yea, this is already consistent with the other examples. the 'src' in them is a shorthand for the top within the example, but they all start with /usr/src with the understanding that you can have the source anywhere.

[patmaddox]

Based on this feedback, I updated the last example to consistently use /usr/src - before the sequence was: src, /usr/src, src/bin/ls, src

[bsdimp]

why WITHOUT_TESTS if you are doing make check?

[patmaddox]

I did my best to explain in the commit comment, please let me know if I can word things more clearly. As far as I can tell, it only comes up when adding new symbols. Here's how it goes:

1. Add a symbol to foo.h and call it from the test program
2. make all now fails, because a) ${SYSROOT}/usr/include/foo.h does not have that symbol b) the test program uses it and c) the test program is included in make all
3. You can't call make install to install it

If you check out the code from #1975 you should be able to reproduce this (example below).

So the trick is we need to run make install to update the header file before we can build the test program. That's what make all install -DWITHOUT_TESTS provides, and then the test program gets built when you run make check.

Another way to do it is setting up CFLAGS and LDFLAGS (e.g. CFLAGS="-I$(make -C lib/libbe -V .OBJDIR)" LDFLAGS="-L$(make -C lib/libbe -V .OBJDIR)" make -s -C lib/libbe clean all) but that's pretty painful imo.

Perhaps instead of this, building the test program needs to be changed to use those when run in the source tree? Because my understanding is that tests can be run in two ways:

1. From /usr/tests against an installed system
2. From within the source tree, (presumably) against the built object files

root@bsdev:~/lab.jj/oss/freebsd-src/main.jj # make buildenv
Entering world for amd64:amd64
root@bsdev:~/lab.jj/oss/freebsd-src/main.jj # make -s -C lib/libbe clean all
===> tests (clean)
===> tests (all)
ld: error: undefined symbol: be_create_empty
>>> referenced by target_prog.c:51 (/root/lab.jj/oss/freebsd-src/main.jj/lib/libbe/tests/target_prog.c:51)
>>>               target_prog.pieo:(main)
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

Stop.
make[4]: stopped making "all" in /root/lab.jj/oss/freebsd-src/main.jj/lib/libbe/tests
*** Error code 1

Stop.
make[3]: stopped making "all" in /root/lab.jj/oss/freebsd-src/main.jj/lib/libbe/tests
*** Error code 1

Stop.
make[2]: stopped making "clean all" in /root/lab.jj/oss/freebsd-src/main.jj/lib/libbe

[patmaddox]

hrm... I think I jumped the gun on this. After some more testing, it seems that make check does require it to be installed to the system and does not actually look in SYSTEM_ROOT for the headers and obj. I just happened to have previously installed it and so was getting false positives.

Perhaps instead of this, building the test program needs to be changed to use those when run in the source tree?

I think this is really it... when I'm running make check within the source tree, I expect it to build against the obj files from the source tree, not from the system. I'll poke around a bit more.

[patmaddox]

I rebuilt my dev VM so I could walk through the steps from scratch that led me to the place where I needed this... and so far have not been able to reproduce it. So now I'm thinking I just did something dumb, and found a workaround in a busted environment.

Going to close this for now - and if I come across it again, I'll re-open and revise with new info.

[patmaddox]

Okay reopened. The -DWITHOUT_TESTS was a red herring caused by a line missing from bin/bectl/tests/Makefile.

My goal was to develop libbe(3) and bectl(1) together without installing files until they were ready. This example documents how to do that.