Skip to content

SVG generation updates#597

Open
n-h-diaz wants to merge 5 commits into
datacommonsorg:masterfrom
n-h-diaz:svg4
Open

SVG generation updates#597
n-h-diaz wants to merge 5 commits into
datacommonsorg:masterfrom
n-h-diaz:svg4

Conversation

@n-h-diaz

Copy link
Copy Markdown
Contributor

Based on reviewing in dev, made a few tweaks to hierarchy generation

  • Attach constrained SVs to a fully qualified SVG (ie with both property and value). This is to avoid intermediate SVGs from containing SVs that should have been placed in a child SVG
  • Match basic 0-constraint SVs to verticals by mprop as well. (This is still deterministic per import, but produces a slightly cleaner hierarchy, especially at upper levels) Note that non-basic SVs will still generate an intermediate population group before attempting to match to verticals

Latest iteration is currently on dev

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the statistical variable group generation logic in stat_var_group_generator.py to support observationProperties (specifically measuredProperty) when matching and generating SVGs, and updates the end-to-end tests to verify these changes. The review feedback highlights two critical issues in the BigQuery SQL queries: a potential runtime error when unnesting a NULL observationProperties array, and a logic bug where equality comparisons with NULL prevent zero-constraint SVs from matching vertical specs that do not define observation properties.

Comment thread pipeline/workflow/ingestion-helper/aggregation/stat_var_group_generator.py Outdated
Comment thread pipeline/workflow/ingestion-helper/aggregation/stat_var_group_generator.py Outdated
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 0 complexity

Metric Results
Complexity 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant