Skip to content

Fix Citation behavior required by Catalog - #74

Merged
asuworks merged 4 commits into
comses:mainfrom
asuworks:release/catalog-integration
Sep 21, 2026
Merged

asuworks merged 4 commits into
comses:mainfrom
asuworks:release/catalog-integration

Conversation

@asuworks

Copy link
Copy Markdown
Contributor

Summary

  • fix primary-publication CSV export, including multi-author cell alignment;
  • correct SuggestedMerge content-type handling and add migration 0036;
  • harden graph, cache, orphan-cleanup, serialization, and URL-validation behavior;
  • add behavior-focused regression tests for exports, merges, commands, and URL validation.

This should merge before the Catalog PR so Catalog can pin the resulting upstream Citation commit.

Verification

  • Citation system and migration checks pass;
  • Citation test suite passes through Catalog CI;
  • Catalog integration, export, merge, cache, and URL-validation tests pass.

Copilot AI lite review requested due to automatic review settings September 20, 2026 15:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

At least one introduced bug in graph filtering date parsing (ISO ...Z handling) will raise at runtime and likely fail the new graphviz tests.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 Medium severity

Open (2)
What changed in this PR

This PR updates Citation’s export, merge, graph, URL-validation, and notification behavior to match expectations needed by the downstream Catalog integration, and adds targeted regression coverage to lock those behaviors in.

Changes:

  • Introduces a transaction-safe publications_changed signal with a notify_publications_changed() helper, and wires it into publication-affecting write paths (serializer saves, merges, URL status updates, admin action, orphan cleanup).
  • Reworks primary-publication CSV exporting to keep each publication on a single CSV row (including correct multi-author formatting) and adds coverage for write vs stream parity.
  • Hardens graph filtering/aggregation and SuggestedMerge content-type handling (plus migration 0036), with regression tests for the new constraints and behaviors.
File Description
tests/​test_validate_urls.py Verifies URL validation continues after request failures and logs failure details.
tests/​test_serializers.py Adds post-commit notification test for PublicationSerializer.save() and validates merge serializer constraints.
tests/​test_merge.py Adds end-to-end tests for SuggestedMerge tag merges, signals, and unsupported model errors.
tests/​test_management_commands.py Ensures orphan cleanup emits publication-change notifications with related IDs.
tests/​test_graphviz_data.py Adds regression tests for graph filtering/aggregation behavior and immutability of filter criteria.
tests/​test_export_data.py Confirms CSV export keeps a publication’s data in one row and streaming matches writing.
tests/​test_auditlog.py Makes audit log contribution tests robust to non-pk=1 publications.
tests/​test_admin.py Tests curator assignment auditing and post-commit notification behavior.
citation/​signals.py Adds publications_changed signal and notify_publications_changed() (on-commit, normalized IDs, robust receiver error logging).
citation/​serializers.py Emits post-commit publication-change notifications on PublicationSerializer.save(); restricts SuggestMerge model_name values.
citation/​ping_urls.py Fixes pattern count logging (len() vs .count()).
citation/​models.py Improves URL status logging on request failures, adds post-commit notifications, tightens SuggestedMerge supported models, fixes tag merge relationship update, and adds supported-model error.
citation/​migrations/​0036_alter_suggestedmerge_content_type.py Migration aligning SuggestedMerge content-type constraints with supported models.
citation/​management/​commands/​validate_urls.py Fixes misplaced debug log statement indentation.
citation/​management/​commands/​remove_orphans.py Wraps in a transaction and emits post-commit notifications for removed orphan IDs.
citation/​management/​commands/​cache_data.py Normalizes log message casing and minor import ordering.
citation/​graphviz/​data.py Replaces Haystack-based filtering with ORM-based filtering and updates aggregation logic.
citation/​export_data.py Refactors CSV exporter to support nested FK attributes, stable author aggregation, and consistent streaming output.
citation/​apps.py Sets AppConfig.path for the app module.
citation/​admin.py Makes curator assignment atomic/bulk-audited and emits post-commit notifications.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread citation/export_data.py
Comment thread citation/graphviz/data.py
@asuworks
asuworks requested a review from alee September 20, 2026 16:07
simplifies downstream filtering so we can use native ORM queries instead
of row by row processing

added some initial tests with cases pulled from a representative sample
of prod data but may still be incomplete

@alee alee left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Anton! Minor comments on tests but no blockers

Comment thread citation/graphviz/data.py Outdated
Comment thread tests/test_export_data.py
Comment thread tests/test_graphviz_data.py
@asuworks
asuworks merged commit fbce715 into comses:main Sep 21, 2026
2 checks passed
asuworks added a commit to asuworks/catalog that referenced this pull request Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants