Repository navigation
Commit b7d5532
authored
perf(pii): batch spaCy NER passes in redact_batch (#8772)
Presidio's BatchAnalyzerEngine defaults nlp.pipe to batch_size=1, so each
2000-text request ran 2000 separate forward passes. Batching at 256 roughly
halves analysis time on prod-shaped chunks with identical detections.1 parent 886cfa2 commit b7d5532
1 file changed
Lines changed: 6 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
267 | 267 | | |
268 | 268 | | |
269 | 269 | | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
270 | 275 | | |
271 | 276 | | |
272 | 277 | | |
| |||
292 | 297 | | |
293 | 298 | | |
294 | 299 | | |
| 300 | + | |
295 | 301 | | |
296 | 302 | | |
297 | 303 | | |
| |||
0 commit comments