Repository navigation
Commit df35aa1
committed
Add joint (lemma, POS) frequency ranking option
Measured on the SBLGNT: joint frequency improves POS agreement (76.2% -> 77.8%)
but lowers top-1 lemma accuracy (92.9% -> 91.9%), so it is off by default.
The lemma-frequency table now also carries per-POS counts.1 parent 691fc1b commit df35aa1
5 files changed
Lines changed: 16472 additions & 8 deletions
File tree
- python
- morpheus_toolkit
- data
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
94 | | - | |
95 | | - | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
96 | 101 | | |
97 | 102 | | |
98 | 103 | | |
| |||
0 commit comments