Skip to content

Commit 64da30d

Browse files
tallpsmithtallpsmith
committed
Add QA infrastructure guide for AI assistants
Document critical learnings about internal headers in test code, especially the libpcp.h handling differences between build tree and installed testsuite, and macOS vs Linux path differences.
1 parent 2df973a commit 64da30d

1 file changed

Lines changed: 208 additions & 0 deletions

File tree

‎qa/CLAUDE.md‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
# QA Testing Infrastructure - Guide for AI Assistants
2+
3+
This file provides critical context about PCP's Quality Assurance (QA) testing infrastructure for AI assistants working with this codebase.
4+
5+
## Critical Learnings
6+
7+
### Internal Headers in Test Code
8+
9+
**CRITICAL**: QA test PMDAs (in `qa/pmdas/`) and test programs (in `qa/src/`) legitimately use INTERNAL PCP headers that are marked `NOSHIP` and are NOT installed to the system include directories.
10+
11+
#### The libpcp.h Conundrum
12+
13+
`libpcp.h` is an internal header containing private symbols like `PDU_FLAG_AUTH`, `pmDebugOptions`, etc. It is:
14+
- Marked `NOSHIP` in `src/include/pcp/GNUmakefile`
15+
- **NOT installed** to `/usr/include/pcp/` or `/usr/local/include/pcp/`
16+
- **ONLY available** in the build tree and the installed testsuite directory
17+
18+
**Key Locations**:
19+
- Build tree: `$(TOPDIR)/src/include/pcp/libpcp.h`
20+
- Installed testsuite: `/var/lib/pcp/testsuite/src/libpcp.h` (note: flattened, no `pcp/` subdirectory)
21+
22+
#### How qa/src Handles This
23+
24+
Files in `qa/src/` use:
25+
```c
26+
#include "libpcp.h" // Quoted include, no pcp/ prefix
27+
```
28+
29+
The `qa/src/GNUmakefile` creates a symlink:
30+
```makefile
31+
libpcp.h: $(TOPDIR)/src/include/pcp/libpcp.h
32+
rm -f libpcp.h
33+
$(LN_S) $(TOPDIR)/src/include/pcp/libpcp.h libpcp.h
34+
```
35+
36+
Then uses `-I$(TOPDIR)/src/include/pcp` (build) or `-I.` (installed) to find it.
37+
38+
#### How qa/pmdas Should Handle This
39+
40+
**DO**:
41+
- Use `#include "libpcp.h"` (quoted, no `pcp/` prefix) to match `qa/src` pattern
42+
- Add `-I../../src` to CFLAGS in `GNUmakefile.install` for installed testsuite builds
43+
- Add `-I$(TOPDIR)/src/include/pcp` for source tree builds
44+
45+
**DON'T**:
46+
- Use `#include <pcp/libpcp.h>` - this requires a `pcp/` subdirectory that doesn't exist after installation
47+
- Try to remove `libpcp.h` includes from test code - they're legitimately needed
48+
- Assume libpcp.h will be in system include directories - it won't be
49+
50+
#### The macOS vs Linux Difference
51+
52+
**Why Linux "works"**:
53+
- On Linux, `PCP_INC_DIR=/usr/include/pcp`
54+
- The GNUmakefile condition `ifneq "$(PCP_INC_DIR)" "/usr/include/pcp"` is FALSE
55+
- So it uses `-I../../src` which finds the installed libpcp.h
56+
57+
**Why macOS initially failed**:
58+
- On macOS, `PCP_INC_DIR=/usr/local/include/pcp` (different prefix)
59+
- The condition is TRUE (not equal to `/usr/include/pcp`)
60+
- So it used `-I/usr/local/include/pcp` which doesn't have libpcp.h
61+
62+
**The Fix**:
63+
- Added Darwin-specific case to explicitly use `-I../../src`
64+
- Changed include style to match `qa/src` pattern
65+
66+
### Test PMDAs vs Production PMDAs
67+
68+
**Production PMDAs** (`src/pmdas/*`):
69+
- Use ONLY public API headers (`pmapi.h`, `pmda.h`)
70+
- Must work when installed system-wide
71+
- Follow strict API boundaries
72+
73+
**Test PMDAs** (`qa/pmdas/*`):
74+
- CAN use internal headers (`libpcp.h`)
75+
- Exist to test internal functionality and edge cases
76+
- More permissive - they're test code, not production code
77+
78+
**Don't "clean up" test code by removing internal headers without understanding why they're there!**
79+
80+
## Test Infrastructure
81+
82+
### Directory Structure
83+
84+
```
85+
qa/
86+
├── src/ # Test programs (binaries that exercise PCP)
87+
├── pmdas/ # Test PMDAs (for testing PMDA functionality)
88+
│ ├── dynamic/ # Dynamic indom test PMDA
89+
│ ├── broken/ # Deliberately broken PMDA for error testing
90+
│ ├── trivial/ # Minimal PMDA example
91+
│ └── ...
92+
├── common.rc # Common shell functions for tests
93+
├── common.check # Service management functions
94+
├── check # Main test runner
95+
└── [0-9]* # Individual test scripts
96+
```
97+
98+
### Build vs Install
99+
100+
**Two different makefile workflows**:
101+
102+
1. **Source tree build** (`GNUmakefile`):
103+
- Uses `$(TOPDIR)` relative paths
104+
- Include: `-I$(TOPDIR)/src/include`
105+
- Links against build tree libraries
106+
107+
2. **Installed testsuite** (`GNUmakefile.install`):
108+
- Installed to `/var/lib/pcp/testsuite/`
109+
- Gets renamed to just `GNUmakefile` when installed
110+
- Include: `-I../../src` (for internal headers), `-I$(PCP_INC_DIR)/..` (for public headers)
111+
- Links against installed libraries
112+
113+
**Critical**: The `GNUmakefile.install` file is what gets used when QA tests run `make setup`!
114+
115+
### Platform-Specific Paths
116+
117+
**Linux**:
118+
- Prefix: `/usr`
119+
- Includes: `/usr/include/pcp`
120+
- Libraries: `/usr/lib` or `/usr/lib64`
121+
122+
**macOS**:
123+
- Prefix: `/usr/local` (or `/opt/homebrew` for Homebrew)
124+
- Includes: `/usr/local/include/pcp`
125+
- Libraries: `/usr/local/lib`
126+
- Why: `/usr` is reserved for Apple system software; third-party software uses `/usr/local`
127+
128+
**BSD**:
129+
- Similar to macOS, typically uses `/usr/local`
130+
131+
## Common Pitfalls
132+
133+
### 1. Assuming Linux-only Behavior
134+
135+
**Problem**: Code that works on Linux might assume `/usr/include/pcp` paths.
136+
137+
**Solution**: Use `$(PCP_INC_DIR)` from `pcp.conf` and handle different prefixes properly.
138+
139+
### 2. Removing "Unnecessary" Includes
140+
141+
**Problem**: Seeing `#include <pcp/libpcp.h>` and thinking "this should use public APIs only".
142+
143+
**Solution**: Test code is ALLOWED to use internal headers. Check the Makefile before removing includes.
144+
145+
### 3. Not Handling Angle vs Quote Includes
146+
147+
**Problem**: Using `<pcp/libpcp.h>` style when libpcp.h is installed flat (no `pcp/` subdirectory).
148+
149+
**Solution**: Use `"libpcp.h"` style for internal headers in test code.
150+
151+
### 4. Forgetting About GNUmakefile.install
152+
153+
**Problem**: Fixing `GNUmakefile` but not `GNUmakefile.install`, which is what actually runs in the testsuite.
154+
155+
**Solution**: When fixing build issues, check BOTH makefiles. The `.install` version is what matters for QA.
156+
157+
## Testing Locally
158+
159+
### Running QA Tests
160+
161+
```bash
162+
# Full sanity suite
163+
cd /var/lib/pcp/testsuite
164+
sudo -u pcpqa ./check -g sanity
165+
166+
# Single test
167+
sudo -u pcpqa ./check 123
168+
169+
# Test range
170+
sudo -u pcpqa ./check 100-200
171+
172+
# Specific group
173+
sudo -u pcpqa ./check -g pmda
174+
```
175+
176+
### Debugging Test PMDA Build Issues
177+
178+
```bash
179+
# Check what's actually installed
180+
ls -la /var/lib/pcp/testsuite/pmdas/dynamic/
181+
182+
# Try building manually
183+
cd /var/lib/pcp/testsuite/pmdas/dynamic
184+
sudo -u pcpqa make -n setup # Dry run to see commands
185+
sudo -u pcpqa make setup # Actual build
186+
187+
# Check include paths
188+
grep CFLAGS GNUmakefile
189+
```
190+
191+
### Checking Header Locations
192+
193+
```bash
194+
. /etc/pcp.conf
195+
echo "PCP_INC_DIR=$PCP_INC_DIR"
196+
197+
# Check for libpcp.h in various locations
198+
ls -la $PCP_INC_DIR/libpcp.h # Won't exist (NOSHIP)
199+
ls -la $PCP_INC_DIR/../libpcp.h # Won't exist
200+
ls -la /var/lib/pcp/testsuite/src/libpcp.h # Should exist
201+
```
202+
203+
## Resources
204+
205+
- Main QA README: `qa/README`
206+
- Service management: `qa/common.check`
207+
- Test helpers: `qa/common.rc`
208+
- macOS QA plan: `build/mac/plans/MACOS_QA_IMPLEMENTATION_PLAN.md`

0 commit comments

Comments
 (0)