██████ ██▓ ███▄ ▄███▓ ██▓███ ██▓ ▓█████
▒██ ▒ ▓██▒▓██▒▀█▀ ██▒▓██░ ██▒▓██▒ ▓█ ▀
░ ▓██▄ ▒██▒▓██ ▓██░▓██░ ██▓▒▒██░ ▒███
▒ ██▒░██░▒██ ▒██ ▒██▄█▓▒ ▒▒██░ ▒▓█ ▄
▒██████▒▒░██░▒██▒ ░██▒▒██▒ ░ ░░██████▒░▒████▒
▒ ▒▓▒ ▒ ░░▓ ░ ▒░ ░ ░▒▓▒░ ░ ░░ ▒░▓ ░░░ ▒░ ░
░ ░▒ ░ ░ ▒ ░░ ░ ░░▒ ░ ░ ░ ▒ ░ ░ ░ ░
░ ░ ░ ▒ ░░ ░ ░░ ░ ░ ░
░ ░ ░ ░ ░ ░ ░
██████ █ ██ ██▓███ ▓█████ ██▀███ ██░ ██ ▓█████ ██▀███ ▒█████
▒██ ▒ ██ ▓██▒▓██░ ██▒▓█ ▀ ▓██ ▒ ██▒▓██░ ██▒▓█ ▀ ▓██ ▒ ██▒▒██▒ ██▒
░ ▓██▄ ▓██ ▒██░▓██░ ██▓▒▒███ ▓██ ░▄█ ▒▒██▀▀██░▒███ ▓██ ░▄█ ▒▒██░ ██▒
▒ ██▒▓▓█ ░██░▒██▄█▓▒ ▒▒▓█ ▄ ▒██▀▀█▄ ░▓█ ░██ ▒▓█ ▄ ▒██▀▀█▄ ▒██ ██░
▒██████▒▒▒▒█████▓ ▒██▒ ░ ░░▒████▒░██▓ ▒██▒░▓█▒░██▓░▒████▒░██▓ ▒██▒░ ████▓▒░
▒ ▒▓▒ ▒ ░░▒▓▒ ▒ ▒ ▒▓▒░ ░ ░░░ ▒░ ░░ ▒▓ ░▒▓░ ▒ ░░▒░▒░░ ▒░ ░░ ▒▓ ░▒▓░░ ▒░▒░▒░
░ ░▒ ░ ░░░▒░ ░ ░ ░▒ ░ ░ ░ ░ ░▒ ░ ▒░ ▒ ░▒░ ░ ░ ░ ░ ░▒ ░ ▒░ ░ ▒ ▒░
░ ░ ░ ░░░ ░ ░ ░░ ░ ░░ ░ ░ ░░ ░ ░ ░░ ░ ░ ░ ░ ▒
░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
██████ ▓█████ ██▀███ ██▒ █▓ ██▓ ▄████▄ ▓█████
▒██ ▒ ▓█ ▀ ▓██ ▒ ██▒▓██░ █▒▓██▒▒██▀ ▀█ ▓█ ▀
░ ▓██▄ ▒███ ▓██ ░▄█ ▒ ▓██ █▒░▒██▒▒▓█ ▄ ▒███
▒ ██▒▒▓█ ▄ ▒██▀▀█▄ ▒██ █░░░██░▒▓▓▄ ▄██▒▒▓█ ▄
▒██████▒▒░▒████▒░██▓ ▒██▒ ▒▀█░ ░██░▒ ▓███▀ ░░▒████▒
▒ ▒▓▒ ▒ ░░░ ▒░ ░░ ▒▓ ░▒▓░ ░ ▐░ ░▓ ░ ░▒ ▒ ░░░ ▒░ ░
░ ░▒ ░ ░ ░ ░ ░ ░▒ ░ ▒░ ░ ░░ ▒ ░ ░ ▒ ░ ░ ░
░ ░ ░ ░ ░░ ░ ░░ ▒ ░░ ░
░ ░ ░ ░ ░ ░ ░ ░ ░ ░
░ ░
I needed a self-contained, data service (no Database) for testing a number of different scenarios with a diverse and robust dataset that also contains some sparseness.
Service runs on Python and FastAPI
The service itself and the data contained within service may be useful for testing:
- CORS configuration
- Server configuration
- Bandwidth
- Form population
- Data visualization
- Stubbing out UI components
- ...
Python 3.12+ uv 0.4+
- Clone the repo
git clone https://github.com/morgangraphics/simple-superhero-service-python.git - cd into the directory and create a virtual environment and install dependencies:
uv venv .venv --python 3.12 uv sync --extra dev
- Generate a self signed cert
openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out sssp-cert.pem -keyout sssp-key.pem -days 365- Or use
make certs(see below)
- Or use
- Copy
config/default.example.cfgtoconfig/.envand update the<PLACEHOLDERS>according to your setup (see below) - Run the service:
make start - The self-signed certs will make the browser throw a
Potential Security Riskerror. Select the Advanced button/link andAccept the risk and continuebutton/link
Marvel URL: https://localhost:8000/marvel
DC URL: https://localhost:8000/dc
Swagger Interface: https://localhost:8000/swagger
⚠️ SSL cert and key paths are configured via theSSL_CERTandSSL_KEYvariables inconfig/.env
Run make help to see all available commands. Full list (alphabetical):
| Command | Description |
|---|---|
make certs |
Generate self-signed SSL certs for local development |
make coverage |
Run tests with coverage report (terminal) |
make coverage-html |
Run tests with coverage report (HTML) |
make dev |
Start the service with auto-reload |
make docker-build |
Build the Docker image |
make docker-shell |
Open a shell in a new container |
make docker-start |
Run the container (mounts local certs, exposes port 8000) |
make docker-stop |
Stop the running container |
make format |
Run black formatter |
make format-check |
Check formatting without making changes |
make install |
Install all dependencies (including dev) |
make lint |
Run ruff linter |
make lint-fix |
Run ruff linter and auto-fix |
make start |
Start the service (foreground) |
make stop |
Stop any background uvicorn process |
make test |
Run all unit tests |
make test-file |
Run file utility tests only |
make test-unit |
Run unit tests (alias) |
make test-verbose |
Run unit tests with verbose output |
Data is the comic book character dataset from @fivethrityeight
This folder contains data behind the story Comic Books Are Still Made By Men, For Men And About Men.
The data comes from Marvel Wikia and DC Wikia. Characters were scraped on August 24. Appearance counts were scraped on September 2. The month and year of the first issue each character appeared in was pulled on October 6.
The data is split into two files, for DC and Marvel, respectively: dc-wikia-data.csv and marvel-wikia-data.csv.
Read more about it here: https://datahub.io/five-thirty-eight/comic-characters#readme
| Marvel | DC |
|---|---|
| 16,376 | 6,896 |
| Variable | Type | Definition |
|---|---|---|
| page_id | integer | The unique identifier for that characters page within the wikia |
| name | string | The name of the character |
| urlslug | string | The unique url within the wikia that takes you to the character |
| id | string | The identity status of the character (Secret Identity, Public identity, [on marvel only: No Dual Identity]) |
| align | string | If the character is Good, Bad or Neutral |
| eye | string | Eye color of the character |
| hair | string | Hair color of the character |
| sex | string | Sex of the character (e.g. Male, Female, etc.) |
| gsm | string | If the character is a gender or sexual minority (e.g. Homosexual characters, bisexual characters) |
| alive | string | If the character is alive or deceased |
| appearances | int | The number of appearances of the character in comic books * |
| first appearance ** | string | The month and year of the character’s first appearance in a comic book, if available |
| year | int | The year of the character’s first appearance in a comic book, if available |
* as of Sep. 2, 2014. Number will become increasingly out of date as time goes on
** first appearance date formatting can be different
[
{
"page_id": 1422,
"name": "batman (bruce wayne)",
"urlslug": "/wiki/batman_(bruce_wayne)",
"id": "secret identity",
"align": "good characters",
"eye": "blue eyes",
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 3093,
"first appearance": "1939, may",
"year": 1939
}
][
{
"page_id": 1678,
"name": "spider-man (peter parker)",
"urlslug": "/spider-man_(peter_parker)",
"id": "secret identity",
"align": "good characters",
"eye": "hazel eyes",
"hair": "brown hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 4043,
"first appearance": "aug-62",
"year": 1962
}
]The base endpoints allow for retrieving data and applying a series of filters to that data to accomplish whatever you need with an array or JSON objects. They can be used in combination with one another
| Variable Name | Variable | Default | Description |
|---|---|---|---|
| format | format | json | Output format (currently only JSON) |
| headers | h | all | Available Columns (page_id, name, urlslug, id, align, eye, hair, sex, gsm, alive, appearances, first appearance, year) |
| help * | help | false | Display Help |
| limit | limit | 100 | Limit results (0 = unlimited) |
| nulls | nulls | first | When sorting, null values show up first or last depending e.g. [null, 1, 2, 3]\ or [1, 2, 3, null] † |
| pretty * | pretty | false | Pretty print JSON results |
| prune * | prune | false | Prune null values from result sets |
| random * ** | random | false | Retrieve a random number of characters based on limit |
| seed * ** | seed | false | Keep the random number of characters consistent between requests |
| sort | s | unsorted | Sort response asc or desc e.g.s=name,appearances:desc |
* Shorthand query parameter, meaning its presence automatically equates to true
** Only available on /{character} endpoints
† Does not apply when sorting on column/header which contains a null value, records with null values are removed
curl -X GET --header 'Accept: application/json' 'https://localhost:8000/dc?pretty&limit=3&s=name:asc'
[
{
"page_id": 1422,
"name": "batman (bruce wayne)",
"urlslug": "/wiki/batman_(bruce_wayne)",
"id": "secret identity",
"align": "good characters",
"eye": "blue eyes",
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 3093,
"first appearance": "1939, may",
"year": 1939
},
{
"page_id": 1458,
"name": "green lantern (hal jordan)",
"urlslug": "/wiki/green_lantern_(hal_jordan)",
"id": "secret identity",
"align": "good characters",
"eye": "brown eyes",
"hair": "brown hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 1565,
"first appearance": "1959, october",
"year": 1959
},
{
"page_id": 23387,
"name": "superman (clark kent)",
"urlslug": "/wiki/superman_(clark_kent)",
"id": "secret identity",
"align": "good characters",
"eye": "blue eyes",
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 2496,
"first appearance": "1986, october",
"year": 1986
}
]curl -X GET --header 'Accept: application/json' 'https://localhost:8000/dc?h=name,appearances&pretty&limit=3&s=name:asc'
[
{
"name": "batman (bruce wayne)",
"appearances": 3093
},
{
"name": "green lantern (hal jordan)",
"appearances": 1565
},
{
"name": "superman (clark kent)",
"appearances": 2496
}
]curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/spider+man,-woman/?pretty&s=name:asc'
[
{
"page_id": 1678,
"name": "spider-man (peter parker)",
"urlslug": "/spider-man_(peter_parker)",
"id": "secret identity",
"align": "good characters",
"eye": "hazel eyes",
"hair": "brown hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 4043,
"first appearance": "aug-62",
"year": 1962
},
{
"page_id": 336259,
"name": "spiderman (1940s) (earth-616)",
"urlslug": "/spiderman_(1940s)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": "white hair",
"sex": "male characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 1,
"first appearance": "dec-46",
"year": 1946
},
{
"page_id": 15388,
"name": "strongman (spider-squad) (earth-616)",
"urlslug": "/strongman_(spider-squad)_(earth-616)",
"id": null,
"align": null,
"eye": null,
"hair": null,
"sex": null,
"gsm": null,
"alive": "living characters",
"appearances": 1,
"first appearance": "aug-77",
"year": 1977
},
{
"page_id": 117593,
"name": "vern (spider-man) (earth-616)",
"urlslug": "/vern_(spider-man)_(earth-616)",
"id": null,
"align": null,
"eye": null,
"hair": null,
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 2,
"first appearance": "jun-08",
"year": 2008
}
]curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/spider+man,-woman/?pretty&s=name:asc&prune'
[
{
"page_id": 1678,
"name": "spider-man (peter parker)",
"urlslug": "/spider-man_(peter_parker)",
"id": "secret identity",
"align": "good characters",
"eye": "hazel eyes",
"hair": "brown hair",
"sex": "male characters",
"alive": "living characters",
"appearances": 4043,
"first appearance": "aug-62",
"year": 1962
},
{
"page_id": 336259,
"name": "spiderman (1940s) (earth-616)",
"urlslug": "/spiderman_(1940s)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"hair": "white hair",
"sex": "male characters",
"alive": "deceased characters",
"appearances": 1,
"first appearance": "dec-46",
"year": 1946
},
{
"page_id": 15388,
"name": "strongman (spider-squad) (earth-616)",
"urlslug": "/strongman_(spider-squad)_(earth-616)",
"alive": "living characters",
"appearances": 1,
"first appearance": "aug-77",
"year": 1977
},
{
"page_id": 117593,
"name": "vern (spider-man) (earth-616)",
"urlslug": "/vern_(spider-man)_(earth-616)",
"sex": "male characters",
"alive": "living characters",
"appearances": 2,
"first appearance": "jun-08",
"year": 2008
}
]curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel?help'
format | format | json | Output format (currently only JSON)
headers | h | all | Available Columns (page_id, name, urlslug, id, align, eye, hair, sex, gsm, alive, appearances, first appearance, year)
| | |
| | | Variable | Definition
| | | ------------------|----------------
| | | page_id | The unique identifier for that characters page within the wikia
| | | name | The name of the character
| | | urlslug | The unique url within the wikia that takes you to the character
| | | id | The identity status of the character (Secret Identity, Public identity, [on marvel only: No Dual Identity])
| | | align | If the character is Good, Bad or Neutral
| | | eye | Eye color of the character
| | | hair | Hair color of the character
| | | sex | Sex of the character (e.g. Male, Female, etc.)
| | | gsm | If the character is a gender or sexual minority (e.g. Homosexual characters, bisexual characters)
| | | alive | If the character is alive or deceased
| | | appearances | The number of appearances of the character in comic books *
| | | first appearance | The month and year of the character’s first appearance in a comic book, if available
| | | year | The year of the character’s first appearance in a comic book, if available
| | |
help | help | false | Display Help
limit | limit | 100 | Limit results ( 0 = unlimited)
nulls | nulls | first | null values sorted first or last e.g. [null, 1, 2, 3] or [1, 2, 3, null] †
pretty | pretty | false | Pretty print JSON results
prune | prune | false | Remove null values from output
random | random | false | Array of random characters based on limit
seed | seed | false | Keep the same random characters on multiple requests
sort | s | unsorted | Sort response asc|desc e.g. s=name,appearances:desc
* (as of Sep. 2, 2014. Number will become increasingly out of date as time goes on.)
† Does not apply when sorting on column/header which contains a null value, records with null values are removed
curl -X GET --header 'Accept: application/json' 'https://localhost:8000/dc?limit=2&random&seed'
[{"page_id":127398,"name":"cassandra cartland (new earth)","urlslug":"/wiki/cassandra_cartland_(new_earth)","id":"","align":"bad characters","eye":"green eyes","hair":"brown hair","sex":"female characters","gsm":"","alive":"living characters","appearances":6,"first appearance":"1997, february","year":1997},{"page_id":192282,"name":"poltergeist (new earth)","urlslug":"/wiki/poltergeist_(new_earth)","id":"","align":"bad characters","eye":"blue eyes","hair":"black hair","sex":"male characters","gsm":"","alive":"living characters","appearances":1,"first appearance":"1996, december","year":1996}]In many cases where there are multiple characters with the same name (in alternative story lines) the first result is usually the "Original" character. That is how the data is structured. However, you might want to specifically target a character or characters by using the search filters.
curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/spider+man,-woman,-616?pretty'
This request is looking for a character name that contains both "spider" AND 'man' BUT NOT "woman" or '616'
[
{
"page_id": 1678,
"name": "spider-man (peter parker)",
"urlslug": "/spider-man_(peter_parker)",
"id": "secret identity",
"align": "good characters",
"eye": "hazel eyes",
"hair": "brown hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 4043,
"first appearance": "aug-62",
"year": 1962
}
]character: character can be a string, or an array of strings (preferred) e.g.
{
"character": "spider-man,iron man"
}OR
{
"character": ["spider-man", "iron man"]
}h: h can be a string, or an array of strings (preferred) e.g.
{
"h": "name,appearances,year"
}OR
{
"h": ["name", "appearances", "year"]
}s: can be a string, an object, array of strings, or an array of objects (preferred) e.g.
{
"s": "name:asc,appearances:desc"
}OR
{
"s": {
"column": "name",
"sort": "asc"
}
}OR
{
"s": ["name:asc", "appearances:desc"]
}OR
{
"s": [
{ "column": "name", "sort": "asc" },
{ "column": "appearances", "sort": "desc" }
]
}curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ "format": "json", "character": [ "spider man" ], "h": [ "name", "appearances", "year" ], "limit": 100, "pretty": true, "s": [ { "column": "name", "sort": "asc" } ] }' 'https://localhost:8000/marvel'
[
{
"name": "spider-man (peter parker)",
"appearances": 4043,
"year": 1962
},
{
"name": "spiderman (1940s) (earth-616)",
"appearances": 1,
"year": 1946
},
{
"name": "vern (spider-man) (earth-616)",
"appearances": 2,
"year": 2008
}
]Character names can have a bit of variation which can be a bit inconsistent between story lines. The official name for Spider-man contains a dash, however, other variants can be Spiderman, Spider man, in addition to Spider-man. The service attempts to use permutation to generate different spellings depending on how the name is entered. If you are unsure, leave a space e.g. spider man and the service will search for spider-man, spiderman, and spider man
curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/spider%20man/?pretty'
[
{
"page_id": 1678,
"name": "spider-man (peter parker)",
"urlslug": "/spider-man_(peter_parker)",
"id": "secret identity",
"align": "good characters",
"eye": "hazel eyes",
"hair": "brown hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 4043,
"first appearance": "aug-62",
"year": 1962
},
{
"page_id": 117593,
"name": "vern (spider-man) (earth-616)",
"urlslug": "/vern_(spider-man)_(earth-616)",
"id": null,
"align": null,
"eye": null,
"hair": null,
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 2,
"first appearance": "jun-08",
"year": 2008
},
{
"page_id": 336259,
"name": "spiderman (1940s) (earth-616)",
"urlslug": "/spiderman_(1940s)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": "white hair",
"sex": "male characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 1,
"first appearance": "dec-46",
"year": 1946
}
]Filters work the same as the base endpoint. (Excluding random and seed)
curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/iron%20man/?pretty&s=year:desc'
[
{
"page_id": 753403,
"name": "zola iron man (earth-616)",
"urlslug": "/zola_iron_man_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": "black eyes",
"hair": "no hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 1,
"first appearance": null,
"year": null
},
{
"page_id": 639805,
"name": "iron man (sentient armor) (earth-616)",
"urlslug": "/iron_man_(sentient_armor)_(earth-616)",
"id": "public identity",
"align": "neutral characters",
"eye": "white eyes",
"hair": "no hair",
"sex": "agender characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 4,
"first appearance": "mar-00",
"year": 2000
},
{
"page_id": 755268,
"name": "iron man (taskmaster robot) (earth-616)",
"urlslug": "/iron_man_(taskmaster_robot)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": null,
"sex": "agender characters",
"gsm": null,
"alive": "living characters",
"appearances": 1,
"first appearance": "oct-98",
"year": 1998
},
{
"page_id": 235195,
"name": "white dragon (iron man) (earth-616)",
"urlslug": "/white_dragon_(iron_man)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 2,
"first appearance": "jul-71",
"year": 1971
},
{
"page_id": 1868,
"name": "iron man (anthony \"tony\" stark)",
"urlslug": "/iron_man_(anthony_%22tony%22_stark)",
"id": "public identity",
"align": "good characters",
"eye": "blue eyes",
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 2961,
"first appearance": "mar-63",
"year": 1963
}
]curl -X GET --header 'Accept: application/json' 'https://localhost:8000/marvel/iron%20man/?pretty&s=year:desc&nulls=last'
[
{
"page_id": 639805,
"name": "iron man (sentient armor) (earth-616)",
"urlslug": "/iron_man_(sentient_armor)_(earth-616)",
"id": "public identity",
"align": "neutral characters",
"eye": "white eyes",
"hair": "no hair",
"sex": "agender characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 4,
"first appearance": "mar-00",
"year": 2000
},
{
"page_id": 755268,
"name": "iron man (taskmaster robot) (earth-616)",
"urlslug": "/iron_man_(taskmaster_robot)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": null,
"sex": "agender characters",
"gsm": null,
"alive": "living characters",
"appearances": 1,
"first appearance": "oct-98",
"year": 1998
},
{
"page_id": 235195,
"name": "white dragon (iron man) (earth-616)",
"urlslug": "/white_dragon_(iron_man)_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": null,
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "deceased characters",
"appearances": 2,
"first appearance": "jul-71",
"year": 1971
},
{
"page_id": 1868,
"name": "iron man (anthony \"tony\" stark)",
"urlslug": "/iron_man_(anthony_%22tony%22_stark)",
"id": "public identity",
"align": "good characters",
"eye": "blue eyes",
"hair": "black hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 2961,
"first appearance": "mar-63",
"year": 1963
},
{
"page_id": 753403,
"name": "zola iron man (earth-616)",
"urlslug": "/zola_iron_man_(earth-616)",
"id": "secret identity",
"align": "bad characters",
"eye": "black eyes",
"hair": "no hair",
"sex": "male characters",
"gsm": null,
"alive": "living characters",
"appearances": 1,
"first appearance": null,
"year": null
}
]MIT / BSD
dm00000 via MORGANGRAPHICS, INC
