Add sector mapping CSV, sector_label/use rollout, and validation#3459
Add sector mapping CSV, sector_label/use rollout, and validation#3459louispt1 wants to merge 8 commits into
sector_label/use rollout, and validation#3459Conversation
|
Nice work! The mapping seems to generally work well. I updated the sector_mapping.csv to represent the correct one. An important change here is the removal of the "klimaattafel" mapping, as we want to first implement the ipcc crt code mapping; the klimaattafel mapping will be added in a later increment. I think now some parts of the ETSource spec don't work as they should anymore. Could you have a look and update this? I think it's best to use the ipcc_crt_code_agg instead. (Sorry for forgetting to tell you that Klimaattafel will be added in a later increment). Perhaps etengine and atlas need to be checked as well? |
|
I found some issues when a value of the columns contains white spaces:
So it's something about handling the blank spaces that needs some fixing. @louispt1 could you look into this? |


Context
config/sector_mapping.csv: the canonical mapping from ETM sector labels to external classification schemes (IPCC CRT codes, klimaattafels, national sectors).sector_labelattribute added across energy and molecule node files, joined by(sector_label, use);usealso added/aligned on molecule nodes. Stale per-nodeemissions_*groups removed now that mapping supersedes them.spec/sector_mapping_spec.rb: validates every labelled node's(sector_label, use)pair exists in the mapping, the join key (sector_label == slug(sector)_slug(subsector)) hasn't drifted, emissions-store coverage per mapping pair × GHG, scheme names don't collide with emissions-store sector keys, and a static scan of gqueries/inputs for literalSECTOR/MSECTOR/scheme-formEMISSIONScalls against unknown schemes or values.Atlas::SectorMapping.Note: The spec is failing currently, presumably due to a mismatch with the mock sector_mapping file.
Related
atlas
etengine
etsource
Checklist