Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: E2E Tests

on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]

jobs:
e2e-tests:
name: E2E Tests (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,test]"
- name: Run unit tests
run: |
pytest tests/unit/ -v --cov=capiscio --cov-report=xml --cov-report=term
- name: Run E2E tests
run: |
pytest tests/e2e/ -v --tb=short
- name: Upload coverage reports
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
Comment on lines +32 to +47
Copy link

Copilot AI Dec 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The E2E workflow is running unit tests (lines 32-34) in addition to E2E tests. This duplicates the work done in the separate unit test workflow (test.yml). The E2E workflow should only run E2E tests to avoid redundancy and reduce CI time.

Suggested change
- name: Run unit tests
run: |
pytest tests/unit/ -v --cov=capiscio --cov-report=xml --cov-report=term
- name: Run E2E tests
run: |
pytest tests/e2e/ -v --tb=short
- name: Upload coverage reports
if: matrix.python-version == '3.11'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false
- name: Run E2E tests
run: |
pytest tests/e2e/ -v --tb=short

Copilot uses AI. Check for mistakes.
43 changes: 43 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Unit Tests

on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]

jobs:
test:
name: Unit Tests (Python ${{ matrix.python-version }})
runs-on: ${{ matrix.os }}

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,test]"

- name: Run unit tests
run: |
pytest tests/unit/ -v --cov=capiscio --cov-report=xml --cov-report=term

- name: Upload coverage
if: matrix.python-version == '3.11' && matrix.os == 'ubuntu-latest'
uses: codecov/codecov-action@v4
with:
file: ./coverage.xml
flags: unittests
name: codecov-umbrella
5 changes: 5 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ dev = [
"build",
"twine",
]
test = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"requests>=2.31.0",
]

[project.scripts]
capiscio = "capiscio.cli:main"
Expand Down
12 changes: 12 additions & 0 deletions pytest.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts = -v --strict-markers
markers =
e2e: marks tests as end-to-end tests requiring a live server
unit: marks tests as unit tests (no external dependencies)

filterwarnings =
ignore::DeprecationWarning
98 changes: 98 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Tests for capiscio-python CLI

This directory contains unit and E2E tests for the `capiscio` CLI wrapper.

## Directory Structure

```
tests/
├── unit/ # Unit tests with mocks (no server required)
│ ├── test_cli.py
│ └── test_manager.py
└── e2e/ # E2E tests (offline mode, no server required)
├── conftest.py # Pytest fixtures and configuration
├── fixtures/ # Test data files
│ ├── valid-agent-card.json
│ ├── invalid-agent-card.json
│ └── malformed.txt
├── test_validate_e2e.py # Validation command tests
└── test_badge_e2e.py # Badge issuance/verification tests
```

## Running Tests

### Run All Tests

```bash
pytest # All tests
pytest tests/unit/ # Unit tests only
pytest tests/e2e/ # E2E tests only
```

### Run Specific Test File

```bash
pytest tests/e2scio --cov-report=html
Copy link

Copilot AI Dec 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an error in the example command. Line 35 shows pytest tests/e2scio --cov-report=html where "e2scio" should be "e2e". This is a typo in the documentation that would cause the command to fail.

Suggested change
pytest tests/e2scio --cov-report=html
pytest tests/e2e/ --cov-report=html

Copilot uses AI. Check for mistakes.
```

## E2E Test Design

The E2E tests are designed to run **offline** without requiring a server:

- **Validate tests**: Use `--schema-only` flag for local schema validation
- **Badge tests**: Use `--self-sign` for issuance and `--accept-self-signed --offline` for verification

This approach allows E2E tests to run in CI without complex server infrastructure.

## Test Coverage

### Validate Command (`test_validate_e2e.py`)

- ✅ Valid local agent card file (schema-only mode)
- ✅ Invalid local agent card file
- ✅ Malformed JSON file
- ✅ Nonexistent file
- ✅ JSON output format
- ✅ Help command

### ├── test_validate_e2e.sue self-signed badge
Copy link

Copilot AI Dec 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 58 contains a broken line that appears to be incorrectly formatted: "### ├── test_validate_e2e.sue self-signed badge". This appears to be a copy-paste error or incomplete edit. The section header is malformed and the content doesn't make sense ("sue self-signed badge"). This should likely be removed or replaced with proper documentation about the Badge Command test coverage.

Suggested change
### ├── test_validate_e2e.sue self-signed badge
### Badge Command (`test_badge_e2e.py`)

Copilot uses AI. Check for mistakes.
- ✅ Issue badge with custom expiration
- ✅ Issue badge with audience restriction
- ✅ Verify self-signed badge (offline)
- ✅ Verify invalid token (error handling)
- ✅ Help commands (badge, issue, verify)

## CI/CD Integration

The E2E tests run in GitHub Actions without server dependencies:

```yaml
# See .github/workflows/e2e.yml
- name: Run E2E tests
run: pytest tests/e2e/
```
## Notes
- **Offline Mode**: All E2E tests run offline without server dependencies
- **Download Messages**: On first run, the CLI may download the capiscio-core binary; tests handle this gracefully
## Troubleshooting
### Build/Install Issues
Ensure the project is installed:
```bash
pip install -e .
pytest tests/e2e/
```

### Path Issues

Ensure you're running pytest from the project root:

```bash
cd /path/to/capiscio-python
pytest tests/e2e/
```
41 changes: 41 additions & 0 deletions tests/e2e/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
"""
pytest configuration for capiscio-python CLI E2E tests.

Provides fixtures for testing the CLI offline using:
- validate --schema-only
- badge issue --self-sign
- badge verify --accept-self-signed --offline
"""

import pytest
from pathlib import Path


@pytest.fixture(scope="session")
def fixtures_dir() -> Path:
"""Get path to fixtures directory."""
return Path(__file__).parent / "fixtures"


@pytest.fixture
def valid_agent_card_path(fixtures_dir: Path) -> Path:
"""Path to valid agent card fixture."""
return fixtures_dir / "valid-agent-card.json"


@pytest.fixture
def invalid_agent_card_path(fixtures_dir: Path) -> Path:
"""Path to invalid agent card fixture."""
return fixtures_dir / "invalid-agent-card.json"


@pytest.fixture
def malformed_json_path(fixtures_dir: Path) -> Path:
"""Path to malformed JSON fixture."""
return fixtures_dir / "malformed.txt"


@pytest.fixture
def nonexistent_path(fixtures_dir: Path) -> Path:
"""Path to a file that doesn't exist."""
return fixtures_dir / "does-not-exist.json"
9 changes: 9 additions & 0 deletions tests/e2e/fixtures/invalid-agent-card.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"version": "1.0",
"did": "invalid-did-format",
"name": "Invalid Agent",
"publicKey": {
"kty": "WRONG",
"x": "invalid-key-data"
}
}
4 changes: 4 additions & 0 deletions tests/e2e/fixtures/malformed.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"version": "1.0",
"did": "did:web:example.com"
"missing": "comma above"
26 changes: 26 additions & 0 deletions tests/e2e/fixtures/valid-agent-card.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"protocolVersion": "1.3.0",
"version": "1.0.0",
"name": "Test Agent",
"description": "A test agent for E2E validation tests",
"url": "https://example.com/.well-known/agent.json",
"capabilities": {
"streaming": false,
"pushNotifications": false
},
"skills": [
{
"id": "test-skill",
"name": "Test Skill",
"description": "A skill for testing",
"tags": ["test", "validation"]
}
],
"provider": {
"organization": "Test Organization",
"url": "https://example.com"
},
"authentication": {
"schemes": ["none"]
}
}
Loading