Skip to content

Commit 2f6b1ed

Browse files
jfb8856606jfb8856606
committed
Add unit test README.
1 parent 7ce7072 commit 2f6b1ed

2 files changed

Lines changed: 411 additions & 48 deletions

File tree

‎tests/README.md‎

Lines changed: 199 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,219 @@
1-
# F-Stack tests/
1+
# F-Stack `tests/`
22

3-
Unit-test framework for F-Stack `lib/` glue code, scoped to the host-side
4-
files declared in `lib/Makefile` `FF_HOST_SRCS`.
3+
Test harness for the F-Stack `lib/` glue layer. It contains two suites:
54

6-
## Quick start
5+
| Suite | Path | Boots DPDK EAL? | Purpose |
6+
|-------|------|-----------------|---------|
7+
| **Unit** | `tests/unit/` | No (rte_* stubbed/wrapped) | Fast, isolated per-file branch/line coverage |
8+
| **Integration** | `tests/integration/` | Yes (`--no-huge --no-pci`) | `ff_dpdk_if.c` paths that need a live EAL |
79

8-
```bash
9-
cd /data/workspace/f-stack/tests/unit
10+
Total: **189 unit test cases** across 11 binaries + **8 integration test cases**.
1011

11-
make help # list available targets
12-
make test # build + run sanity + P0 + P1 (~0.4s, 59 TC)
13-
make test_p0 # P0 only (ff_ini_parser + ff_log, 31 TC)
14-
make test_p1 # P1 only (ff_host_interface + ff_epoll + ff_config, 26 TC)
15-
make test_sanity # hello-world sanity check (2 TC)
16-
make clean # remove build artifacts (uses workspace rm_tmp_file.sh)
17-
```
12+
---
1813

19-
## Prerequisites
14+
## 1. Prerequisites
2015

2116
- `gcc` + GNU `make`
2217
- `pkg-config` reporting `cmocka >= 1.1.7`
23-
- On TencentOS 4.4: `dnf install -y libcmocka libcmocka-devel`
18+
- TencentOS 4.x: `dnf install -y libcmocka libcmocka-devel`
2419
- Verify: `pkg-config --modversion cmocka`
25-
- DPDK headers (`/usr/local/include/rte_config.h` etc.) — used when compiling
26-
`lib/ff_log.c` and `lib/ff_config.c` host-side
20+
- DPDK 23.11 / 24.11 headers + runtime libs under `/usr/local` (`rte_config.h`, `librte_*.so`)
21+
- `lcov` / `genhtml` (for coverage reports)
22+
- `valgrind` (for `make check` memcheck)
23+
24+
Coverage and integration binaries are linked against the DPDK shared libs, so they
25+
need the runtime path at execution time:
26+
27+
```bash
28+
export LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH
29+
```
30+
31+
The Makefiles already inject this via `RUN_ENV` for `make test` / `make check` /
32+
`make coverage`; you only need it when launching a binary by hand (e.g.
33+
`./test_ff_dpdk_kni`).
34+
35+
---
36+
37+
## 2. Quick start
38+
39+
```bash
40+
cd /data/workspace/f-stack/tests/unit
41+
42+
make help # list all targets
43+
make test # build + run every unit binary (189 TC)
44+
make check # re-run everything under valgrind memcheck (0 leak gate)
45+
./run_coverage.sh # build with gcov + run + lcov HTML + G8 threshold gate
46+
```
47+
48+
Whole-project (unit + integration merged) coverage:
49+
50+
```bash
51+
cd /data/workspace/f-stack/tests
52+
./run_full_coverage.sh # merges unit + integration into one report
53+
```
54+
55+
---
56+
57+
## 3. Unit suite — `make` targets
58+
59+
| Target | Action |
60+
|--------|--------|
61+
| `make all` | Build all test binaries |
62+
| `make test` | Build + run all suites (sanity + P0 + P1 + P2 + P3) |
63+
| `make test_sanity` | hello-world cmocka/pkg-config sanity (2 TC) |
64+
| `make test_p0` | P0: `ff_ini_parser` + `ff_log` |
65+
| `make test_p1` | P1: `ff_host_interface` + `ff_epoll` + `ff_config` |
66+
| `make test_p2` | P2: `ff_thread` + `ff_init` + `ff_dpdk_pcap` + `ff_dpdk_if` |
67+
| `make test_p3` | P3: `ff_dpdk_kni` |
68+
| `make check` | Run every binary under `valgrind --tool=memcheck` (definite leak => fail) |
69+
| `make coverage` | gcov build + run + emit `coverage_report/index.html` |
70+
| `make clean` | Remove build artifacts (via `rm_tmp_file.sh` wrapper) |
71+
| `make coverage_clean`| Remove `.gcda/.gcno` + `coverage_report/` |
72+
73+
You can also build/run a single binary:
74+
75+
```bash
76+
make test_ff_config
77+
LD_LIBRARY_PATH=/usr/local/lib64:$LD_LIBRARY_PATH ./test_ff_config
78+
```
79+
80+
### Test-binary → lib-file map
2781

28-
## Layout
82+
| Binary | Group | lib file under test | TC | Notes |
83+
|--------|-------|---------------------|----|-------|
84+
| `test_hello` | sanity | — | 2 | toolchain smoke test |
85+
| `test_ff_ini_parser` | P0 | `ff_ini_parser.c` | 25 | inih-style parser |
86+
| `test_ff_log` | P0 | `ff_log.c` | 13 | 4 `rte_log` API wraps |
87+
| `test_ff_host_interface` | P1 | `ff_host_interface.c` | 22 | links libcrypto (RAND_bytes) |
88+
| `test_ff_epoll` | P1 | `ff_epoll.c` | 21 | kqueue/kevent synth stubs |
89+
| `test_ff_config` | P1 | `ff_config.c` | 50 | end-to-end via `ff_load_config`; `--wrap=calloc` for OOM |
90+
| `test_ff_thread` | P2 | `ff_thread.c` | 4 | links real pthread |
91+
| `test_ff_init` | P2 | `ff_init.c` | 6 | `--wrap=ff_malloc` |
92+
| `test_ff_dpdk_pcap` | P2 | `ff_dpdk_pcap.c` | 9 | stack mbuf + pcap rotation |
93+
| `test_ff_dpdk_if` | P2 | `ff_dpdk_if.c` | 19 | unit-mockable subset only |
94+
| `test_ff_dpdk_kni` | P3 | `ff_dpdk_kni.c` | 18 | boots EAL `--no-huge`; built `-DFF_KNI` |
2995

96+
---
97+
98+
## 4. Coverage scripts
99+
100+
### 4.1 `unit/run_coverage.sh` — unit coverage + G8 gate
101+
102+
```bash
103+
./run_coverage.sh # full: clean + gcov build + run + report + G8 summary
104+
./run_coverage.sh --quick # reuse existing .gcda (skip rebuild)
105+
./run_coverage.sh --file ff_config.c # per-line gcov detail for one file
106+
./run_coverage.sh --serve # also serve HTML report on :8080
107+
./run_coverage.sh --clean # remove all coverage artifacts and exit
108+
./run_coverage.sh -h # help
30109
```
31-
tests/
32-
└── unit/
33-
├── Makefile GNU make, no cmake/meson, no lib/Makefile pollution
34-
├── common/
35-
│ ├── ff_log_stub.{c,h} defines `struct ff_config ff_global_cfg`
36-
│ └── rte_stub.{c,h} __wrap_rte_exit / __wrap_rte_panic via mock_assert
37-
├── fixtures/ .ini files for P1 ff_config end-to-end tests
38-
├── lib_objs/ (build cache) lib/*.c built with our CFLAGS
39-
├── test_hello.c sanity (CMocka + pkg-config)
40-
├── test_ff_ini_parser.c P0 #1 — 18 TC (1 SKIP: FU-S2-NULLFILE)
41-
├── test_ff_log.c P0 #2 — 13 TC, 4 rte_log API wraps
42-
├── test_ff_host_interface.c P1 #1 — 8 TC, links libcrypto for RAND_bytes
43-
├── test_ff_epoll.c P1 #2 — 7 TC, ff_kqueue/ff_kevent stubs in test
44-
└── test_ff_config.c P1 #3 — 11 TC, end-to-end via ff_load_config
110+
111+
Exit codes: `0` success & all G8 thresholds met · `1` build/test failure or G8
112+
violation · `2` bad CLI args.
113+
114+
HTML report: `unit/coverage_report/index.html`.
115+
116+
### 4.2 `unit/coverage_threshold.sh` — the "G8" per-file gate
117+
118+
Invoked automatically at the end of `run_coverage.sh`. It parses `coverage.info`
119+
and enforces a per-file minimum **line** and **branch** percentage. Thresholds are
120+
maintained as `tline["<file>"]` / `tbr["<file>"]` entries and are ratcheted upward
121+
after each coverage-boost stage (kept at `actual − ~5pp` as a regression guard).
122+
123+
### 4.3 `run_full_coverage.sh` — merged unit + integration
124+
125+
```bash
126+
cd tests
127+
./run_full_coverage.sh # build + run both suites, merge, report
128+
./run_full_coverage.sh --quick # reuse existing .info traces
129+
./run_full_coverage.sh --clean # remove all coverage artifacts
45130
```
46131

47-
## Adding a new test for an existing lib file
132+
Merged HTML report: `tests/full_coverage_report/index.html`.
48133

49-
1. Add a target rule in `Makefile` listing the lib_objs/*.o + stubs to link
50-
2. Add `__wrap_<sym>` linker flags if the file needs to mock additional rte_*
51-
APIs (see `WRAP_FF_LOG` for an example)
52-
3. Create `test_<file>.c` following the cmocka template
53-
4. `make <target>` to build, `./<target>` to run
134+
---
54135

55-
## Test design references
136+
## 5. Integration suite — `tests/integration/`
56137

57-
- Spec docs: `docs/unit_test_spec/zh_cn/{04-cmocka-framework-and-impl.md, 06-test-cases-and-acceptance.md}`
58-
- Methodology: `.codebuddy/rules/c-unittest-expert.mdc` (Unity-based; mapped to CMocka API)
59-
- Stage-1 review: `docs/unit_test_spec/zh_cn/99-review-report.md`
60-
- Stage-2 implementation review: `docs/unit_test_spec/zh_cn/99-stage2-review.md`
138+
Unlike the unit suite, this harness boots a **real DPDK EAL** (`--no-huge`,
139+
`--no-pci`) so it can exercise `ff_dpdk_if.c` paths that depend on live mempools,
140+
rings and ethdev. If EAL init fails (e.g. insufficient permissions), the affected
141+
TCs `skip()` rather than fail.
142+
143+
```bash
144+
cd tests/integration
145+
make help
146+
make test # run integration tests (8 TC)
147+
make check # under valgrind
148+
make coverage # gcov build + report -> coverage_report/index.html
149+
make clean
150+
```
61151

62-
## Workspace mandates honored
152+
---
153+
154+
## 6. Current coverage snapshot (merged unit + integration)
155+
156+
| File | line | branch | Note |
157+
|------|------|--------|------|
158+
| `ff_log.c` | 100% | 100% | capped |
159+
| `ff_thread.c` | 100% | 100% | capped |
160+
| `ff_init.c` | 100% | 100% | capped |
161+
| `ff_dpdk_pcap.c` | 100% | 100% | L118 dead leg `LCOV_EXCL_BR_LINE` |
162+
| `ff_epoll.c` | 100% | 100% | |
163+
| `ff_host_interface.c` | 100% | 98.1% | 2 clock-assert legs need `--wrap=__assert_fail` |
164+
| `ff_ini_parser.c` | 98.7% | 91.2% | 6 `&&!error` legs dead via `INI_STOP_ON_FIRST_ERROR` |
165+
| `ff_config.c` | 89.9% | 85.4% | remaining = OOM/dataflow single legs |
166+
| `ff_dpdk_kni.c` | 59.9% | 51.6% | tx/rx/alloc need integration (`rte_eth_*_burst` are `static inline`, not wrappable) |
167+
| `ff_dpdk_if.c` | 30.8% | 22.6% | mostly integration-only |
168+
| **Project (merged)** | 62.9% | 63.7% | |
169+
170+
> Numbers are produced by `run_coverage.sh` / `run_full_coverage.sh`; re-run them
171+
> to refresh. The authoritative source is always the freshly generated
172+
> `coverage.info`, not this table.
173+
174+
---
175+
176+
## 7. Adding a test for an existing lib file
177+
178+
1. Add a per-target rule in `unit/Makefile` listing the `lib_objs/*.o` + stubs to
179+
link, and any `-Wl,--wrap=<sym>` flags the file needs (see `WRAP_FF_LOG`,
180+
`WRAP_FF_DPDKIF`, the `test_ff_config` `--wrap=calloc`, and the
181+
`ff_dpdk_kni.o: KNI_EXTRA_CFLAGS := -DFF_KNI` per-object override for examples).
182+
2. Add the binary name to the right `P{0,1,2,3}_TESTS` group.
183+
3. Create `test_<file>.c` following the cmocka template (`group_setup` /
184+
`test_setup` / `cmocka_unit_test_setup_teardown`).
185+
4. Naming convention: `test_<function>_<scenario>_<expected>`.
186+
5. `make test_<file>` to build, run, then `./run_coverage.sh --file <file>.c` to
187+
confirm the targeted branches are now covered.
188+
189+
### Mock / wrap infrastructure
190+
191+
- `common/rte_stub.c` — `__wrap_rte_exit` / `__wrap_rte_panic` redirect to
192+
cmocka `mock_assert`, so a regression can never `SIGABRT` the harness.
193+
- `common/ff_log_stub.{c,h}` — provides `struct ff_config ff_global_cfg` and a
194+
no-op `ff_log` for binaries that do not link `lib/ff_log.c`.
195+
- OOM testing: `--wrap=calloc` with an armed counter (`g_calloc_fail_after`)
196+
passes through to `__real_calloc` unless armed, letting a TC force the Nth
197+
allocation to fail without disturbing cmocka's own allocations.
198+
- Inline DPDK helpers (`rte_eth_tx_burst`, `rte_eth_rx_burst`,
199+
`rte_ring_dequeue_burst`) are `static inline` and **cannot** be `--wrap`-ed;
200+
testing those paths requires the integration suite with a real PMD.
201+
202+
---
203+
204+
## 8. Workspace mandates honored
63205

64206
- All transient file deletions go through `/data/workspace/rm_tmp_file.sh`
65-
- No direct `rm`, `kill`, `pkill`, `killall`, `chmod` invocations anywhere in
66-
the tree (zero-tolerance per workspace memory rules)
67-
- Test process never calls real `rte_exit` / `rte_panic` (intercepted via
68-
`__wrap_*` to `mock_assert`, so a regression cannot SIGABRT the harness)
207+
(the Makefile `clean` target uses it; no direct `rm`).
208+
- No direct `rm` / `kill` / `pkill` / `killall` / `chmod` anywhere in the tree.
209+
- Test processes never call real `rte_exit` / `rte_panic` (intercepted via
210+
`__wrap_*` → `mock_assert`).
211+
212+
---
213+
214+
## 9. Design references
215+
216+
- Specs: `docs/unit_test_spec/zh_cn/` (`04-cmocka-framework-and-impl.md`,
217+
`06-test-cases-and-acceptance.md`, Stage-7 `7x-*.md`, Stage-8 `8x-*.md`)
218+
- Methodology: `c-unittest-expert` skill (Unity-based, mapped to CMocka API)
219+
- Stage reviews: `docs/unit_test_spec/zh_cn/{99-*,79-stage7-review,89-stage8-review}.md`

0 commit comments

Comments
 (0)