Add pytest infrastructure with 113 tests covering core functionality

[Copilot]

Establishes comprehensive testing infrastructure to accelerate AI agent development workflows. Currently covers 8 key modules with targeted tests for file operations, EAD processing, batch logic, and configuration discovery.

Infrastructure

• pytest.ini: Coverage reporting, custom markers (unit, integration, slow, skip_complex)
• CI/CD workflow: Python 3.9/3.10/3.11, coverage upload to Codecov
• Shared fixtures: Mock ArchivesSpace client, sample data structures, temp directories
• Documentation: tests/README.md with usage patterns and testing philosophy

Test Coverage (113 passing, 10 skipped)

• File operations (15 tests): save_file, create_symlink, get_ead_from_symlink
• Subprocess/glob (18 tests): Wildcard expansion, iglob iteration, CSV discovery
• EAD operations (14 tests): ID extraction, dots→dashes sanitization, XML parsing
• Batch processing (21 tests): Size calculations, iteration patterns, edge cases
• Config discovery (15 tests): find_traject_config multi-path search (arcuit_dir → bundle → fallback)
• XML manipulation (20 tests): Escaping for plain text vs. structured XML preservation
• Utilities (11 tests): get_repo_id, path construction helpers

Agent Filtering (Intentionally Stubbed)

test_agent_filtering.py documents the 5-tier filtering system but marks all implementation tests as skipped. The current logic is too complex to test without refactoring:

@pytest.mark.skip(reason="Agent filtering too complex - needs refactoring before testing")
def test_system_user_exclusion(self):
    """
    SKIPPED: Should test that agents with is_user=True are excluded.
    
    Current issues:
    - Requires full ArcFlow instance
    - Needs mock client with complex response structure
    - Tight coupling to logging
    """
    pass

3 documentation tests pass to verify filtering tier definitions remain accurate.

Usage

pytest                              # All tests
pytest -m unit                      # Unit tests only
pytest --cov=arcflow --cov-report=term-missing

Metrics

• Test count: 123 total (113 passing, 10 skipped)
• Coverage: 16% baseline (targeted functions)
• Runtime: ~5 seconds
• Dependencies: pytest>=7.0.0, pytest-cov>=4.0.0, pytest-mock>=3.10.0

Original prompt

Add comprehensive testing infrastructure to speed up AI agent development workflow:

Infrastructure Files

1. pytest.ini - Configure pytest with coverage and markers
2. .github/workflows/test.yml - CI/CD for Python 3.9, 3.10, 3.11
3. tests/conftest.py - Shared fixtures (mock_asnake_client, temp_dir, sample data)
4. tests/README.md - Documentation on running and writing tests

Test Files

Create 8 test files in tests/ directory:

1. test_file_operations.py - save_file, create_symlink, get_ead_from_symlink
2. test_subprocess_fixes.py - glob.glob wildcard expansion for batch files
3. test_ead_operations.py - EAD ID extraction, dots→dashes sanitization
4. test_batching.py - batch calculation logic, edge cases
5. test_config_discovery.py - find_traject_config method (new in index_creators)
6. test_xml_manipulation.py - XML escaping, bioghist content handling
7. test_utilities.py - Simple helpers like get_repo_id, path construction
8. test_agent_filtering.py - STUB ONLY - Document that filtering logic is too complex and needs refactoring before testing. Mark all tests as skipped with note.

Key Points

• Use pytest fixtures from conftest.py
• Mock ArchivesSpace API calls
• Test happy paths, edge cases, and error handling
• Agent filtering gets stub warning file only (too complex to test as-is)
• Include pytest-cov for coverage reporting
• Tests should pass except agent filtering (intentionally skipped)

Dependencies

Add to setup.py or pyproject.toml:

pytest>=7.0.0
pytest-cov>=4.0.0  
pytest-mock>=3.10.0

This pull request was created from Copilot chat.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.