Remove dead exception handler in index_collections subprocess call#28
Draft
Copilot wants to merge 14 commits intoindex_creatorsfrom
Draft
Remove dead exception handler in index_collections subprocess call#28Copilot wants to merge 14 commits intoindex_creatorsfrom
Copilot wants to merge 14 commits intoindex_creatorsfrom
Conversation
Implement complete ETL pipeline for ArchivesSpace agents: - Extract all agent records via ArchivesSpace API - Generate EAC-CPF XML documents for each agent - Auto-discover and configure traject indexing - Batch index to Solr (100 files per call for performance) - Support multiple processing modes (agents-only, collections-only, both) - Add 11 new Solr fields for agent metadata - Include 271-line traject config for EAC-CPF → Solr mapping Key features: - Parallel to existing collection record indexing - Dynamic Solr field mapping for ArcLight compatibility - Robust error handling and logging - Configurable traject config discovery paths This allows ArcLight to provide dedicated agent/creator pages with full biographical information, related collections, and authority control.
Replace per-agent API calls with single Solr query for better performance: - Query ArchivesSpace Solr to filter agents in bulk - Exclude system users (publish=false) - Exclude donors (linked_agent_role includes "dnr") - Exclude software agents (agent_type="agent_software") - Use consistent EAC namespace prefixes in XPath queries - Refactor dates extraction for improved readability Performance improvement: O(n) API calls → O(1) Solr query Reduces processing time from minutes to seconds for large repositories.
…C-CPF indexing (#13) * Skip indexing records without valid IDs instead of generating non-deterministic fallbacks Co-authored-by: alexdryden <47127862+alexdryden@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: alexdryden <47127862+alexdryden@users.noreply.github.com> Co-authored-by: Alex Dryden <adryden3@illinois.edu>
to reflect the required command line arguments Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Use filename for id
Co-authored-by: alexdryden <47127862+alexdryden@users.noreply.github.com>
Co-authored-by: alexdryden <47127862+alexdryden@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Update creator record generation and indexing based on feedback
Remove dead exception handler in index_collections subprocess call
Feb 26, 2026
f23fe83 to
89057a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses feedback on #8 regarding unreachable exception handler in
index_collectionsmethod.Changes
Removed dead
except subprocess.CalledProcessErrorhandler: The handler was unreachable becausesubprocess.run()was called withoutcheck=True. Error handling already exists viaresult.returncodechecking.Fixed subprocess invocation: Changed from passing joined string to passing command as list directly. This was broken after
shell=Truewas removed in commit 6c0942d, and the list form is more secure.Before:
After:
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.