diff --git a/README.md b/README.md index 67b77dd7..55d16ba5 100644 --- a/README.md +++ b/README.md @@ -51,11 +51,16 @@ The store provides a mechanism to upload and register new Flow Segments, and an The media data contained within Flow Segments may be stored separately from the metadata linking them to a position on the timeline, separating the media data and metadata planes. For example our implementation uses a database (_e.g._ Amazon DynamoDB) to store Flow Segment metadata and an object store (_e.g._ AWS S3) to store the media data for Flow Segments. -We refer to media data stored in the object store as 'Media Objects'. +We refer to data stored in the object store as 'Objects'. +Where these Objects contain media, they are referred to as 'Media Objects'. +Where they contain initialisation segments required to decode associated media (e.g. for CMAF-compatible media), the are referred to as 'init Objects'. The Flow Segment has a list of S3 download urls which are the location of the Media Object(s) that contains the stored media data for the Flow Segment. If multiple URLs are returned in the list, they are assumed to return identical media data. When writing to the store, the S3 URLs can be passed to a client permitting them to upload media data directly. +> [!NOTE] +> For more information on working with init segments, see [AppNote 0024 - Using Media with Initialisation Segments in TAMS](./docs/appnotes/0024-using-init-segments.md). + Another advantage of separating the media data and metadata planes in this way is that a particular Media Object can be referenced by multiple Flows. On the metadata side, the Flow Segment is just a mapping of a Media Object's ID to a Flow's Timeline, so any number of Flows can record that same ID against other `` tuples. This allows for copy-on-write semantics: immutability means a new Flow must be created to make changes to existing parts of the timeline, but for unmodified portions of the timeline the new `` tuple points to the existing Media Object or a part of it. diff --git a/api/TimeAddressableMediaStore.yaml b/api/TimeAddressableMediaStore.yaml index 1bde34d2..a4659df3 100644 --- a/api/TimeAddressableMediaStore.yaml +++ b/api/TimeAddressableMediaStore.yaml @@ -23,7 +23,7 @@ info: See the [bbc/tams](https://github.com/bbc/tams) repository for more background on TAMS. - **Note**: the examples provided in this specification are for a system which uses a Storage Backend that provides HTTP URLs for uploading and downloading Media Objects in buckets. + **Note**: the examples provided in this specification are for a system which uses a Storage Backend that provides HTTP URLs for uploading and downloading Objects in buckets. This could for example be implemented using an AWS S3 compatible store using presigned URLs or by a simple file system storage with an HTTP frontend. Clients should parse the [/service](#/operations/GET_service) endpoint and handle the Storage Backend type in use appropriately. version: "8.1" @@ -865,6 +865,11 @@ paths: description: Filter on video Flows that have the given frame height. schema: type: integer + - name: init_segments + in: query + description: Filter Flows on the value of `init_segments`. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -952,6 +957,11 @@ paths: description: Filter on video Flows that have the given frame height. schema: type: integer + - name: init_segments + in: query + description: Filter Flows on the value of `init_segments`. + schema: + type: boolean - $ref: '#/components/parameters/trait_resource_paged_key' - $ref: '#/components/parameters/trait_paged_limit' responses: @@ -1053,6 +1063,9 @@ paths: mxf_h264: summary: Video Flow - H.264 (MXF container) externalValue: examples/flow-get-200-video-h264-mxf.json + fmp4_h264: + summary: Video Flow - H.264 (fmp4 container) + externalValue: examples/flow-get-200-video-h264-fmp4.json jp2_jpeg2k_singleframe: summary: Video Flow - JPEG-2000 (JP2 container, single frame segments) externalValue: examples/flow-get-200-video-jpeg-jp2.json @@ -1143,7 +1156,9 @@ paths: summary: Delete Flow description: | Deletes the Flow and associated Segments. - If Flow Segment deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress + If Flow Segment deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress. + + Services SHALL only delete Objects when they are no longer referenced by any Flow Segments or other Objects (i.e. as init Objects). operationId: DELETE_flows-flowId tags: - Flows @@ -1162,7 +1177,7 @@ paths: example: $ref: examples/deletion-request-get-200.json "204": - description: No content. The Flow has been deleted and the Flow Segments have been or will be deleted. Media Objects referenced in other Flows will not be deleted. Media Objects that are no longer referenced by any Segments will be deleted. + description: No content. The Flow has been deleted and the Flow Segments have been or will be deleted. Objects referenced in other Flows will not be deleted. Objects that are no longer referenced by any Segments will be deleted. "403": description: Forbidden. You do not have permission to modify this Flow. It may be marked read-only. "404": @@ -1754,39 +1769,39 @@ paths: in: query description: | Include storage metadata in `get_urls` in the response. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Flow Segment `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Flow Segment `get_urls` with no label will only be returned if `accept_get_urls` is omitted. + A comma separated list of labels of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Flow Segment `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. Without `get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Flow Segment `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Flow Segment `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Flow Segment `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/uuid-list.json' - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` in the response. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls` in the response. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: include_object_timerange @@ -1856,6 +1871,9 @@ paths: Note that for codecs with temporal re-ordering, the timerange representes the _presentation_ timeline, and clients may need to check the `key_frame_count` property and/or read backwards from the start of the requested timerange to retrieve enough reference material to start decoding. + Where Flow Segments reference initialisation segment Objects with the same ID, the initialisation segment is the same. + Consuming clients may choose to ignore initialisation segment Objects that haven't changed in subsequent Flow Segments. + When making requests to the provided `get_urls`, clients should include credentials if the provided URL is on the same origin as the API itself, akin to the `same-origin` mode in the [WhatWG Fetch Standard](https://fetch.spec.whatwg.org/#concept-request-credentials-mode). operationId: GET_flows-flowId-segments tags: @@ -1881,39 +1899,39 @@ paths: in: query description: | Include storage metadata in `get_urls` in the response. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Flow Segment `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Flow Segment `get_urls` with no label will only be returned if `accept_get_urls` is omitted. + A comma separated list of labels of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Flow Segment `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. Without `get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Flow Segment `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Flow Segment `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Flow Segment `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Flow Segment `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/uuid-list.json' - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: include_object_timerange @@ -1974,6 +1992,12 @@ paths: This example shows how the Media Object for a Flow Segment could be accessed using multiple URLs. Clients are free to choose any of the URLs given (which return identical Media Objects), however the service has expressed a preference for the "pipeline-a" URLs by putting them first in the list. externalValue: examples/flow-segments-get-200-multiple-urls.json + init_segments: + summary: Objects with initialisation segments + description: | + This example shows how Media Objects signal the use of initialisation segments. + Note the re-use of the same initialisation segment in some consecutive Flow Segments. + externalValue: examples/flow-segments-get-200-init.json "400": description: Bad request. Invalid query options. "404": @@ -2014,7 +2038,10 @@ paths: - The timerange of the Segment MUST NOT overlap any other Segment in the same Flow. - The Flow Segment's `timerange` start and end, once offset by `ts_offset`, MUST be contained entirely within the Media Object's `timerange` - When re-using Media Objects, requests which change object properties (e.g. `key_frame_count` or `object_timerange`) SHOULD be rejected. + When re-using Media Objects, requests which change object properties (e.g. `key_frame_count`, `object_timerange`, or `init_object_id`) SHOULD be rejected. + + If an Object has previously been registered as an initialisation segment (i.e. via `init_object_id`), Service implementations SHOULD reject its use as a media segment (i.e. via `object_id`). + If an Object has previously been registered as a media segment (i.e. via `object_id`), Service implementations SHOULD reject its use as an initialisation segment (i.e. via `init_object_id`). operationId: POST_flows-flowId-segments tags: - FlowSegments @@ -2050,7 +2077,9 @@ paths: delete: summary: Delete Flow Segment description: | - Deletes the Flow Segments. If the deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress + Deletes the Flow Segments. If the deletion takes too long then this request will return 202 Accepted and the `Location` header will point to a Flow Delete Request to monitor deletion progress. + + Services SHALL only delete Objects when they are no longer referenced by any Flow Segments or other Objects (i.e. as init Objects). operationId: DELETE_flows-flowId-segments tags: - FlowSegments @@ -2081,7 +2110,7 @@ paths: example: $ref: examples/deletion-request-get-200.json "204": - description: No content. The Flow Segments have been or will be deleted. Media Objects referenced in other Segments will not be deleted. Media Objects that are no longer referenced by any Segments will be deleted. + description: No content. The Flow Segments have been or will be deleted. Objects referenced in other Segments will not be deleted. Objects that are no longer referenced by any Segments will be deleted. "400": description: Bad request. Invalid query options. "403": @@ -2099,13 +2128,13 @@ paths: post: summary: Allocate Initial Flow Storage description: | - Allocate initial storage locations for writing Media Objects. + Allocate initial storage locations for writing Objects. The Storage Backend type, which is indicated in the [/service](#/operations/GET_service) resource, determines the information provided in the response. The examples and description below are for the "http_object_store" Storage Backend type. - This Storage Backend type provides HTTP URLs for uploading and downloading Media Objects in buckets. + This Storage Backend type provides HTTP URLs for uploading and downloading Objects in buckets. - The response will include a PUT URL that a client uses to upload the Media Object. + The response will include a PUT URL that a client uses to upload the Object. The client is expected to register the Flow Segment using the [/flows/{flowId}/segments](#/operations/POST_flows-flowId-segments) endpoint once the upload is complete. Clients MAY request Objects in batches to reduce the number of HTTP requests made to the Service. @@ -2113,7 +2142,13 @@ paths: Objects will likely go unused in cases such as shutdown of ingesting clients, the end of ingested live streams, and unexpected network congestion. Clients SHOULD, however, adapt the number of Objects they request such that they may reasonably expect to use them before the timeout advertised in [`min_object_timeout` at the `/service`](#/operations/GET_service) endpoint, which is subject to a specified minimum (see service endpoint schema). - Service implementations need to handle situations where Objects are not used, and where content is were uploaded but no Flow Segment was registered successfully. + Where media format make use of initialisation segments, the mime-type of those initialisation segments may differ from that of the media segments. + In such cases, the `container` on the Flow will be set to the mime-type of the media segments and is the assumed default. + A separate request to this endpoint should be made to allocate storage for the initialisation segment(s) with `content-type` set to the mime type of the initialisation segment(s). + `content-type` MUST NOT be set in the request body of this end point at any other time. + Initialisation segment Objects SHOULD be re-used wherever possible. + + Service implementations need to handle situations where Objects are not used, and where content is were uploaded but the Object is not successfully registered against a Flow Segment or as an init Object against Media Objects. In these circumstances, Services should garbage collect Objects after the timeout advertised in [`min_object_timeout` at the `/service`](#/operations/GET_service) endpoint. When making requests to the provided `put_url`, clients should include credentials if the provided URL is on the same origin as the API itself, akin to the `same-origin` mode in the [WhatWG Fetch Standard](https://fetch.spec.whatwg.org/#concept-request-credentials-mode). @@ -2134,7 +2169,7 @@ paths: $ref: schemas/flow-storage-post.json responses: "201": - description: Storage locations for writing Media Objects. + description: Storage locations for writing Objects. content: application/json: schema: @@ -2156,15 +2191,15 @@ paths: description: The requested Flow does not exist. /objects/{objectId}: head: - summary: Media Object Information - description: Return Flow references and other information about Media Objects. + summary: Object Information + description: Return Flow references and other information about init or Media Objects. operationId: HEAD_objects tags: - Objects parameters: - name: objectId in: path - description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. + description: The Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. required: true schema: type: string @@ -2172,42 +2207,42 @@ paths: in: query description: | Include storage metadata in `get_urls`. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Media Object `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Media Object `get_urls` with no label will only be returned if `accept_get_urls` is omitted. - Without `get_urls`, the response from the service could be substantially faster if it is not required to + A comma separated list of labels of Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Object `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. + Without `get_urls` or `init_object.get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Media Object `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Media Object `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Object `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: string pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$ - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: flow_tag.{name} @@ -2247,21 +2282,21 @@ paths: "400": $ref: '#/components/responses/trait_resource_info_head_400' "404": - description: The requested Media Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. + description: The requested Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. get: - summary: Media Object Information + summary: Object Information description: | - Contains Flows that references the Media Object and other information. + Contains Flows that references the init/Media Object and other information. - The paging query parameters and headers are required for the list of Flow references in the Media Object. - Service implementations should return a complete list of Flow references within reason and API clients should expect paging to happen in some rare cases where a Media Object is used in many Flows. + The paging query parameters and headers are required for the list of Flow references in the Object. + Service implementations should return a complete list of Flow references within reason and API clients should expect paging to happen in some rare cases where a Object is used in many Flows. operationId: GET_objects tags: - Objects parameters: - name: objectId in: path - description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. + description: The Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. required: true schema: type: string @@ -2269,42 +2304,42 @@ paths: in: query description: | Include storage metadata in `get_urls`. - When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls`. + When `verbose_storage` is `false` only `url`, `presigned`, and `label` will be included in `get_urls` and `init_object.get_urls`. schema: default: false type: boolean - name: accept_get_urls in: query description: | - A comma separated list of labels of Media Object `get_urls` to include in the response. - Omitting `accept_get_urls` will result in no filtering of `get_urls`. - An empty `accept_get_urls` results in an empty or no `get_urls` in the response. - Media Object `get_urls` with no label will only be returned if `accept_get_urls` is omitted. - Without `get_urls`, the response from the service could be substantially faster if it is not required to + A comma separated list of labels of Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_get_urls` will result in no filtering of `get_urls` or `init_object.get_urls`. + An empty `accept_get_urls` results in `get_urls` and `init_object.get_urls` being empty or omitted in the response. + Object `get_urls` and `init_object.get_urls` with no label will only be returned if `accept_get_urls` is omitted. + Without `get_urls` and `init_object.get_urls`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs for example. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: $ref: 'schemas/url-label-list.json' - name: accept_storage_ids in: query description: | - A comma separated list of `storage_id`s of Media Object `get_urls` to include in the response. - Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls`. - Media Object `get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. + A comma separated list of `storage_id`s of Object `get_urls` and `init_object.get_urls` to include in the response. + Omitting `accept_storage_ids`, or providing an empty `accept_storage_ids` will result in no filtering of `get_urls` or `init_object.get_urls`. + Object `get_urls` and `init_object.get_urls` with no storage ID will only be returned if `accept_storage_ids` is omitted or empty. A full list of available `storage_id`s may be found at the `service/storage-backends` endpoint. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: string pattern: ^([0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})(,[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})*$ - name: presigned in: query description: | - If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls`. - If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls`. + If set to `true`, only presigned URLs (i.e. those whos `presigned` property is `true`) will be returned in `get_urls` and `init_object.get_urls`. + If set to `false`, only non-presigned URLs (i.e. those whos `presigned` property is `false`) will be returned in `get_urls` and `init_object.get_urls`. If omitted, both presigned and non-presigned URLs will be returned. If `presigned` is set to `false`, the response from the service could be substantially faster if it is not required to generate a large number of pre-signed URLs. - Where multiple filter query parameters are provided, the returned `get_urls` will match all filters. + Where multiple filter query parameters are provided, the returned `get_urls` and `init_object.get_urls` will match all filters. schema: type: boolean - name: flow_tag.{name} @@ -2339,28 +2374,36 @@ paths: type: string content: application/json: - example: - $ref: examples/objects-get-200.json + examples: + basic: + summary: Basic example + externalValue: examples/objects-get-200.json + init_segment: + summary: Object that makes use of an initialisation segment + externalValue: examples/objects-get-200-init.json schema: $ref: "schemas/object.json" "400": description: Bad request. Invalid query options. "404": - description: The requested Media Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. + description: The requested Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. /objects/{objectId}/instances: post: - summary: Register a Media Object instance + summary: Register an Object instance description: | Request the service to create an Object instance on a new Storage Backend. Or add a new uncontrolled URL to `get_urls`. To request the duplication of the Object to a new Storage Backend, clients POST a `storage_id` to this endpoint that does not currently have an instance of the Object. The API will then: - - Allocate storage for Media Object `objectId` on Storage Backend `storage_id` - - Copy the Media Object from an existing location to the newly allocated storage + - Allocate storage for Object `objectId` on Storage Backend `storage_id` + - Copy the Object from an existing location to the newly allocated storage - Start advertising the new copy in `get_urls` once ready The API instances SHOULD be capable of handling the case where the only existant instances are uncontrolled. + API instances SHALL NOT cascade the creation of new Object instances to initialisation Objects. + Clients MUST initiate the creation of new Object instances of initialisation Objects directly. + Where a client has written a new uncontrolled Object instance, the client is responsible for ensuring that the Object written is complete and correct before registering it with this method. All instances of an Object MUST be identical. @@ -2370,7 +2413,7 @@ paths: parameters: - name: objectId in: path - description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. + description: The Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. required: true schema: type: string @@ -2393,46 +2436,49 @@ paths: "400": description: Bad request. Invalid request JSON. "403": - description: Forbidden. You do not have permission to modify this Media Object. + description: Forbidden. You do not have permission to modify this Object. "404": - description: The Media Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. + description: The Object does not exist. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. delete: - summary: Delete a Media Object instance + summary: Delete an Object instance description: | - Delete an instance of a Media Object. + Delete an instance of a Object. One of `storage_id` or `label` MUST be specified in the query parameters. `storage_id` SHOULD be used where `controlled` is `True` for the instance. - API instances should remove the Media Object instance from the `get_urls` list and then, if the instance is controlled, delete the Object instance from storage. + API instances should remove the Object instance from the `get_urls` list and then, if the instance is controlled, delete the Object instance from storage. API instances SHOULD prevent clients from deleting all Object instances. Additionally, API instances MAY prevent clients from deleting all controlled Object instances. Where clients wish to remove all copies of an Object from the store, they should do so by deleting all Flows or Flow Segments which reference the Object. + + API instances SHALL NOT cascade the deletion of Object instances to initialisation Objects. + Clients MUST initiate the deletion of Object instances of initialisation Objects directly. operationId: DELETE_objects-instances tags: - Objects parameters: - name: objectId in: path - description: The Media Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. + description: The Object identifier. The Object ID may include special characters such as `/` which should be URL encoded. required: true schema: type: string - name: storage_id in: query - description: The storage_id identifying the Media Object instance to be deleted. + description: The storage_id identifying the Object instance to be deleted. schema: type: string - name: label in: query - description: The label identifying the Media Object instance to be deleted. + description: The label identifying the Object instance to be deleted. schema: type: string responses: "204": - description: No content. The Media Object instance has been deleted. + description: No content. The Object instance has been deleted. "400": description: Bad request. Invalid query options. "403": - description: Forbidden. You do not have permission to modify this Media Object. + description: Forbidden. You do not have permission to modify this Object. "404": description: The requested Object ID in the path is invalid. 404 MUST be returned if the ID has been assigned via the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage), but not yet registered against a Flow Segment. /flow-delete-requests: @@ -2852,7 +2898,7 @@ tags: - name: Objects description: The Object in the Storage Backend(s) that contains the media essence. - name: MediaStorage - description: The system that stores the Media Objects referenced by Flow Segments. + description: The system that stores the Objects referenced by Flow Segments. - name: FlowDeleteRequests description: Resource for monitoring long running deletion of Flows and Flow Segments. - name: Webhooks diff --git a/api/examples/flow-get-200-video-h264-fmp4.json b/api/examples/flow-get-200-video-h264-fmp4.json new file mode 100644 index 00000000..1de5e6d1 --- /dev/null +++ b/api/examples/flow-get-200-video-h264-fmp4.json @@ -0,0 +1,54 @@ +{ + "id": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "source_id": "2aa143ac-0ab7-4d75-bc32-5c00c13d186f", + "generation": 0, + "created": "2008-05-27T18:51:00Z", + "metadata_updated": "2023-09-14T09:45:26Z", + "segments_updated": "2023-09-14T09:45:26Z", + "description": "Big Buck Bunny", + "label": "bbb", + "format": "urn:x-nmos:format:video", + "created_by": "tams-dev", + "updated_by": "tams-dev", + "tags": { + "input_quality": "contribution" + }, + "codec": "video/h264", + "container": "video/iso.segment", + "avg_bit_rate": 2479, + "segment_duration": { + "numerator": 10 + }, + "essence_parameters": { + "init_segments": true, + "frame_rate": { + "numerator": 24, + "denominator": 1 + }, + "frame_width": 1280, + "frame_height": 720, + "bit_depth": 8, + "interlace_mode": "progressive", + "colorspace": "BT709", + "transfer_characteristic": "SDR", + "aspect_ratio": { + "numerator": 16, + "denominator": 9 + }, + "pixel_aspect_ratio": { + "numerator": 1, + "denominator": 1 + }, + "component_type": "YCbCr", + "vert_chroma_subs": 2, + "horiz_chroma_subs": 2, + "avc_parameters": { + "profile": 100, + "level": 31, + "flags": 0 + } + }, + "collected_by": [ + "e85efab4-993b-4ad6-9af3-4cd8d0d38860" + ] +} diff --git a/api/examples/flow-segments-get-200-init.json b/api/examples/flow-segments-get-200-init.json new file mode 100644 index 00000000..eb95f4ef --- /dev/null +++ b/api/examples/flow-segments-get-200-init.json @@ -0,0 +1,53 @@ +[ + { + "object_id": "846023d3-612d-5014-bc47-88f6eb2d04bb", + "timerange": "[0:0_10:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb" + } + ], + "init_object": { + "object_id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } + }, + { + "object_id": "25be83fc-11d1-5743-9d47-6865cef5ea35", + "timerange": "[10:0_20:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/25be83fc-11d1-5743-9d47-6865cef5ea35" + } + ], + "init_object": { + "object_id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } + }, + { + "object_id": "8b785422-6a82-5d60-b25a-f77e0a748321", + "timerange": "[20:0_30:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/8b785422-6a82-5d60-b25a-f77e0a748321" + } + ], + "init_object": { + "object_id": "192472f1-55fd-45dc-b355-e6833a3a140c", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/192472f1-55fd-45dc-b355-e6833a3a140c" + } + ] + } + } +] diff --git a/api/examples/objects-get-200-init.json b/api/examples/objects-get-200-init.json new file mode 100644 index 00000000..3c7c4a0b --- /dev/null +++ b/api/examples/objects-get-200-init.json @@ -0,0 +1,22 @@ +{ + "id": "846023d3-612d-5014-bc47-88f6eb2d04bb", + "referenced_by_flows": [ + "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "0fde9c11-da9d-434a-a113-d3b20a2cf251" + ], + "first_referenced_by_flow": "4f79cfd1-c057-47f4-8e4d-1b126ca7bf34", + "timerange": "[150:0_200:0)", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/846023d3-612d-5014-bc47-88f6eb2d04bb" + } + ], + "init_object": { + "id": "9ef06e23-b882-4a93-b005-480de42ef4e9", + "get_urls": [ + { + "url": "https://store.example.com/tams-e2b89b02-21e7-5f9d-aa2d-db38b01453c9/9ef06e23-b882-4a93-b005-480de42ef4e9" + } + ] + } +} diff --git a/api/schemas/flow-audio.json b/api/schemas/flow-audio.json index b0632d3c..4f6a2a76 100644 --- a/api/schemas/flow-audio.json +++ b/api/schemas/flow-audio.json @@ -78,6 +78,10 @@ ] } } + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } } } diff --git a/api/schemas/flow-core.json b/api/schemas/flow-core.json index 9f6acae4..55a907d8 100644 --- a/api/schemas/flow-core.json +++ b/api/schemas/flow-core.json @@ -68,7 +68,7 @@ "$ref": "mime-type.json" }, "container": { - "description": "The container MIME type for Flow Segments. Note that the `type` component of the container MIME type (i.e. the component before the `/`) may be different to the `type` component of the codec MIME type. e.g. An audio Flow may have `audio/aac` coded content may be wrapped in a `video/mp2t` container. Where multiple types exist for a subtype (e.g. `video/mp4`, `audio/mp4`, `application/mp4`), the closest MIME type to the Flow `format` should be used (e.g. `audio/mp4` for a Flow `format` of `urn:x-nmos:format:audio`). Mime types from the [IANA registry](https://www.iana.org/assignments/media-types/media-types.xhtml) should be preferred. Where multiple MIME types are possible, the most common should be preferred. Where this is insufficient, the maintainers of the TAMS repository may create an application note advising which MIME type to use. Where the Flow does not reference any Media Object(s) directly (e.g. an empty Multi Flow that serves only to collect related mono-essence Flows that do reference Media Objects), this property MUST NOT be set.", + "description": "The container MIME type for Flow Segments. Where the media format employs initialisation segments, this is the mime type of the media segments and NOT the initialisation segment(s). Note that the `type` component of the container MIME type (i.e. the component before the `/`) may be different to the `type` component of the codec MIME type. e.g. An audio Flow may have `audio/aac` coded content may be wrapped in a `video/mp2t` container. Where multiple types exist for a subtype (e.g. `video/mp4`, `audio/mp4`, `application/mp4`), the closest MIME type to the Flow `format` should be used (e.g. `audio/mp4` for a Flow `format` of `urn:x-nmos:format:audio`). Mime types from the [IANA registry](https://www.iana.org/assignments/media-types/media-types.xhtml) should be preferred. Where multiple MIME types are possible, the most common should be preferred. Where this is insufficient, the maintainers of the TAMS repository may create an application note advising which MIME type to use. Where the Flow does not reference any Media Object(s) directly (e.g. an empty Multi Flow that serves only to collect related mono-essence Flows that do reference Media Objects), this property MUST NOT be set.", "$ref": "mime-type.json" }, "avg_bit_rate": { diff --git a/api/schemas/flow-data.json b/api/schemas/flow-data.json index d8d4ab7c..8d2deb4f 100644 --- a/api/schemas/flow-data.json +++ b/api/schemas/flow-data.json @@ -30,6 +30,10 @@ "data_type": { "description": "The type of information encoded in the Flow, identified using a URN. e.g. The data_type may be urn:x-tams:data:bounding-box, and the codec `application/json`.", "type": "string" + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } } } diff --git a/api/schemas/flow-multi.json b/api/schemas/flow-multi.json index 88e283a7..c82891a8 100644 --- a/api/schemas/flow-multi.json +++ b/api/schemas/flow-multi.json @@ -18,6 +18,18 @@ "enum": [ "urn:x-nmos:format:multi" ] + }, + "essence_parameters": { + "title": "Multi Flow Essence Parameters", + "description": "Describes the parameters of the essence inside this multi Flow", + "type": "object", + "additionalProperties": false, + "properties": { + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" + } + } } } } diff --git a/api/schemas/flow-segment-post.json b/api/schemas/flow-segment-post.json index bb7d26dc..5f5e60fc 100644 --- a/api/schemas/flow-segment-post.json +++ b/api/schemas/flow-segment-post.json @@ -8,7 +8,11 @@ ], "properties": { "object_id": { - "description": "The Object identifier for the Media Object.", + "description": "The Object identifier for the Media Object. The `content-type` of the Media Object MUST match the `container` mime-type of the Flow. Service implementations SHOULD reject Objects IDs which have previously been registered as an `init_object_id` on other Flow Segments. i.e. init segments may not be used as media segments.", + "type": "string" + }, + "init_object_id": { + "description": "The Object identifier for the initialisation segment Object required to decode the Media Object. The `content-type` of the initialisation segment Object MAY differ from the `container` mime-type of the Flow. This parameter MUST only be set where the media format makes use of initialisation segments. Initialisation Objects SHOULD be re-used where possible. This parameter SHOULD be omitted where the Object `object_id` already exists and is being re-used. Service implementations SHOULD reject Objects IDs which have previously been registered as an `id` on other Flow Segments. i.e. media segments may not be used as init segments.", "type": "string" }, "ts_offset": { diff --git a/api/schemas/flow-segment.json b/api/schemas/flow-segment.json index df0244d6..7226f4e9 100644 --- a/api/schemas/flow-segment.json +++ b/api/schemas/flow-segment.json @@ -39,11 +39,33 @@ "description": "The count of samples in the Segment (which may be fewer than in the Media Object). The count could be less than expected given the Segment duration and rate if there are gaps. If not set, every sample from sample_offset onwards is used. Note that a sample is a video frame or audio sample. A (coded) audio frame has multiple audio samples. DEPRECATED: Use object_timerange instead - see AppNote 0036. Service implementations SHOULD continue to store and return it if set.", "type": "integer", "deprecated": true + }, + "init_object": { + "title": "Initialisation Object", + "description": "The Object containing the initialisation segment required to decode the parent Media Object.", + "unevaluatedProperties": false, + "allOf": [ + { + "type": "object", + "required": [ + "object_id" + ], + "properties": { + "object_id": { + "description": "The identifier of the initialisation Object.", + "type": "string" + } + } + }, + { + "$ref": "object-core.json" + } + ] } } }, { - "$ref": "object-core.json" + "$ref": "object-media-core.json" } ] } diff --git a/api/schemas/flow-storage-post.json b/api/schemas/flow-storage-post.json index df85e17a..fdb785ef 100644 --- a/api/schemas/flow-storage-post.json +++ b/api/schemas/flow-storage-post.json @@ -4,7 +4,7 @@ "type": "object", "properties": { "limit": { - "description": "Limit the number of Media Objects in each response page. Service implementations may specify their own default and maximum for the limit", + "description": "Limit the number of Objects in each response page. Service implementations may specify their own default and maximum for the limit", "type": "integer" }, "object_ids": { @@ -17,6 +17,10 @@ "storage_id": { "description": "The Storage Backend to allocate storage in. A Storage Backend identifier as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint. If not set the default, as advertised at the [/service/storage-backends](#/operations/GET_storage-backends) endpoint, will be used if available. An invalid Storage Backend identifier will result in a 400 error.", "$ref": "uuid.json" + }, + "content_type": { + "description": "The `content_type` to use for the Objects. This parameter MUST only be set where requesting storage for initialisation segments in media formats which require them, and where the mime-type of those initialisation segments differs to that of the media segments. Assumed to be the `container` type of the Flow if not set.", + "$ref": "mime-type.json" } }, "not": { diff --git a/api/schemas/flow-storage.json b/api/schemas/flow-storage.json index 2614b27a..9abd76e3 100644 --- a/api/schemas/flow-storage.json +++ b/api/schemas/flow-storage.json @@ -1,21 +1,21 @@ { "type": "object", "title": "Media Bucket Object Store", - "description": "Gives information on storage for Media Objects. This schema is for the `http_object_store` Storage Backend type which provides URLs for storing Media Objects in object store buckets, and is the only Storage Backend type currently implemented. URLs SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details.", + "description": "Gives information on storage for Objects. This schema is for the `http_object_store` Storage Backend type which provides URLs for storing Objects in object store buckets, and is the only Storage Backend type currently implemented. URLs SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details. Where included in the response, `content-type` MUST match the corresponding value in the request.", "properties": { "media_objects": { "type": "array", - "description": "List of information for identifying and uploading Media Objects", + "description": "List of information for identifying and uploading Objects", "items": { "type": "object", - "description": "Information for a Media Object", + "description": "Information for a Object", "required": [ "object_id", "put_url" ], "properties": { "object_id": { - "description": "The object store identifier for the Media Object.", + "description": "The object store identifier for the Object.", "type": "string" }, "put_url": { diff --git a/api/schemas/flow-video.json b/api/schemas/flow-video.json index 4a7aa0ac..a971b789 100644 --- a/api/schemas/flow-video.json +++ b/api/schemas/flow-video.json @@ -212,6 +212,10 @@ "description": "If `true`, the frame rate of the Flow is variable and `frame_rate` MUST NOT be set. If `false` or omitted, the frame rate of the Flow is fixed and `frame_rate` MUST be set.", "type": "boolean", "default": false + }, + "init_segments": { + "description": "Whether the Flow makes use of initialisation segments. This parameter MUST be set to `true` if Media Objects have `init_object` populated. If set to `true`, all Media Objects MUST have `init_object` populated. Assume `false` if omitted.", + "type": "boolean" } }, "if": { diff --git a/api/schemas/object-core.json b/api/schemas/object-core.json index c22779e2..574724b7 100644 --- a/api/schemas/object-core.json +++ b/api/schemas/object-core.json @@ -1,10 +1,10 @@ { "type": "object", - "description": "Provides the location and metadata of the media files corresponding to a Media Object.", + "description": "Provides the location and metadata of the files corresponding to a Object.", "title": "Object", "properties": { "get_urls": { - "description": "A list of URLs to which a GET request can be made to directly retrieve the contents of the Media Object. This is required by the `http_object_store` Storage Backend type, which is the only one currently described. Clients may choose any URL in the list and treat the content returned as identical, however servers may sort the list such that the preferred URL is first. Storage Backend metadata for controlled URLs should be populated by the TAMS instance based on the Storage Backend the Meda Object instance resides in.", + "description": "A list of URLs to which a GET request can be made to directly retrieve the contents of the Object. This is required by the `http_object_store` Storage Backend type, which is the only one currently described. Clients may choose any URL in the list and treat the content returned as identical, however servers may sort the list such that the preferred URL is first. Storage Backend metadata for controlled URLs should be populated by the TAMS instance based on the Storage Backend the Object instance resides in.", "type": "array", "items": { "type": "object", @@ -24,7 +24,7 @@ "$ref": "uuid.json" }, "url": { - "description": "A URL to which a GET request can be made to directly retrieve the contents of the media object. Clients should include credentials if the provide URL is on the same origin as the API endpoint. This URL SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details.", + "description": "A URL to which a GET request can be made to directly retrieve the contents of the Object. Clients should include credentials if the provide URL is on the same origin as the API endpoint. This URL SHOULD support the inclusion of checksums in headers as supported by advertised Storage Backend product. See AppNote 0048 for more details.", "type": "string" }, "presigned": { @@ -43,10 +43,6 @@ } ] } - }, - "key_frame_count": { - "description": "The number of key frames in the Media Object. This should be set greater than zero when the Media Object contains key frames that serve as a stream access point", - "type": "integer" } } } diff --git a/api/schemas/object-media-core.json b/api/schemas/object-media-core.json new file mode 100644 index 00000000..ab0e7d74 --- /dev/null +++ b/api/schemas/object-media-core.json @@ -0,0 +1,18 @@ +{ + "type": "object", + "description": "Provides the location and metadata of the media files corresponding to a Media Object.", + "title": "Object", + "allOf": [ + { + "properties": { + "key_frame_count": { + "description": "The number of key frames in the Media Object. This should be set greater than zero when the Media Object contains key frames that serve as a stream access point", + "type": "integer" + } + } + }, + { + "$ref": "object-core.json" + } + ] +} diff --git a/api/schemas/object.json b/api/schemas/object.json index 5eba2810..f9dab617 100644 --- a/api/schemas/object.json +++ b/api/schemas/object.json @@ -6,33 +6,54 @@ "type": "object", "required": [ "id", - "referenced_by_flows", - "timerange" + "referenced_by_flows" ], "properties": { "id": { - "description": "The Media Object identifier.", + "description": "The Object identifier.", "type": "string" }, "referenced_by_flows": { "type": "array", - "description": "List of Flows that reference this Media Object via Flow Segments in this store instance.", + "description": "List of Flows that reference this Object via Flow Segments in this store instance. For init Objects, this reference is indirect via Media Objects.", "items": { "$ref": "uuid.json" } }, "first_referenced_by_flow": { - "description": "The first Flow that had a Flow Segment reference the Media Object in this store instance. This Flow is also present in 'referenced_by_flows' if it is still referenced by the Flow. This property is optional and may in some implementations become unset if the Flow no longer references the media object, e.g. because it was deleted.", + "description": "The first Flow that had a Flow Segment reference the Object in this store instance. For init Objects, this reference is indirect via Media Objects. This Flow is also present in 'referenced_by_flows' if it is still referenced by the Flow. This property is optional and may in some implementations become unset if the Flow no longer references the Object, e.g. because it was deleted.", "$ref": "uuid.json" }, "timerange": { - "description": "The timerange covering the sample timestamps embedded in or derived from the Media Object itself, on the Media Object's timeline.", + "description": "The timerange covering the sample timestamps embedded in or derived from the Object itself, on the Media Object's timeline. This parameter MUST be set where the Object contains media. It MUST NOT be set where the Object contains an init segment.", "$ref": "timerange.json" + }, + "init_object": { + "title": "Initialisation Object", + "description": "The Object containing the initialisation segment required to decode the parent Media Object.", + "unevaluatedProperties": false, + "allOf": [ + { + "type": "object", + "required": [ + "id" + ], + "properties": { + "id": { + "description": "The identifier of the initialisation Object.", + "type": "string" + } + } + }, + { + "$ref": "object-core.json" + } + ] } } }, { - "$ref": "object-core.json" + "$ref": "object-media-core.json" } ] } diff --git a/api/schemas/objects-instances-post.json b/api/schemas/objects-instances-post.json index 85aa6646..41714ed5 100644 --- a/api/schemas/objects-instances-post.json +++ b/api/schemas/objects-instances-post.json @@ -1,10 +1,10 @@ { "type": "object", - "description": "Register a Media Object instance in the store.", - "title": "Media object registration", + "description": "Register a Object instance in the store.", + "title": "Object registration", "oneOf": [ { - "description": "Request the duplication of a Media Object instance to a new Storage Backend, via it's `storage_id`.", + "description": "Request the duplication of a Object instance to a new Storage Backend, via it's `storage_id`.", "title": "Controlled instance", "type": "object", "required": [ @@ -18,7 +18,7 @@ } }, { - "description": "Register an uncontrolled Media Object instance via its `url`.", + "description": "Register an uncontrolled Object instance via its `url`.", "title": "Uncontrolled instance", "type": "object", "required": [ @@ -27,11 +27,11 @@ ], "properties": { "url": { - "description": "A URL to which a GET request can be made to directly retrieve the contents of the media object. Clients should include credentials if the provide URL is on the same origin as the API endpoint", + "description": "A URL to which a GET request can be made to directly retrieve the contents of the Object. Clients should include credentials if the provide URL is on the same origin as the API endpoint", "type": "string" }, "label": { - "description": "Label identifying this Media Object instance. Service implementations should reject any requests using labels that are already associated with Storage Backends.", + "description": "Label identifying this Object instance. Service implementations should reject any requests using labels that are already associated with Storage Backends.", "type": "string" } } diff --git a/api/schemas/service.json b/api/schemas/service.json index 1934ae71..3da94441 100644 --- a/api/schemas/service.json +++ b/api/schemas/service.json @@ -37,7 +37,7 @@ } }, "min_object_timeout": { - "description": "The minimum timeframe within which a Media Object created by this service must be registered against a Flow segment before it is garbage collected. Services SHOULD allow a small grace period beyond the advertised value to account for latency in assigning the Objects and returning them to the Client. This timeout MUST be `300:0` (i.e. 5 minutes) or greater. Clients MUST be capable of reaching this minimum performance level. Clients SHOULD adapt to this value by balancing how many object URLs to request per page against how fast they will be used. Services MAY allow this value to be configured at deploy-time. Format as described by the [Timestamp](#/schemas/timestamp) type. For more infomation, see the documentation of the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage) endpoint.", + "description": "The minimum timeframe within which a Object created by this service must be registered against a Flow segment before it is garbage collected. Services SHOULD allow a small grace period beyond the advertised value to account for latency in assigning the Objects and returning them to the Client. This timeout MUST be `300:0` (i.e. 5 minutes) or greater. Clients MUST be capable of reaching this minimum performance level. Clients SHOULD adapt to this value by balancing how many object URLs to request per page against how fast they will be used. Services MAY allow this value to be configured at deploy-time. Format as described by the [Timestamp](#/schemas/timestamp) type. For more infomation, see the documentation of the [`/flows/{flowId}/storage`](#/operations/POST_flows-flowId-storage) endpoint.", "$ref": "timestamp.json" }, "min_presigned_url_timeout": { diff --git a/api/schemas/url-label-list.json b/api/schemas/url-label-list.json index 537864a3..5af087c7 100644 --- a/api/schemas/url-label-list.json +++ b/api/schemas/url-label-list.json @@ -1,6 +1,6 @@ { "title": "Query String GET URL Label list", - "description": "A list of Media Object GET URL Labels, formatted for use in query string parameters", + "description": "A list of Object GET URL Labels, formatted for use in query string parameters", "type": "string", "pattern": "^([^,]+(,[^,]+)*)?$" } diff --git a/docs/README.md b/docs/README.md index 6a646f47..c6719ba7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -27,7 +27,8 @@ For more information on how we use application notes, see [here](./appnotes/READ | [0018](./appnotes/0018-managing-multiple-object-instances.md) | Managing Multiple Object Instances | | [0019](./appnotes/0019-implementing-retention-management.md) | Methods of implementing retention management | | [0021](./appnotes/0021-media-integrity.md) | Integrity model for media in TAMS, and when interacting with other systems | -| [0022](./appnotes/0022-transform-architectures.md) | Architectures for the transforming of media | +| [0022](./appnotes/0022-transform-architectures.md) | Architectures for the transforming of media | +| [0024](./appnotes/0024-using-init-segments.md) | Using Media with Initialisation Segments in TAMS | ## ADRs @@ -78,12 +79,13 @@ For more information on how we use ADRs, see [here](./adr/README.md). | [0037](./adr/0037-improve-webhooks.md) | Proposal for improvements to the Webhooks endpoints | | [0038](./adr/0038-improved-storage-management.md) | Improved Storage Management | | [0039](./adr/0039-remove-pre-actions.md) | Proposal to remove pre-actions from storage allocation response | -| [0040](./adr/0040-tag-usability-enhancements.md) | Tag Usability Enhancements | -| [0041](./adr/0041-require-explicit-framerate.md) | Requiring explicit frame rates | -| [0042](./adr/0042-uncontrolled-object-instance-labels.md) | Make `label` Mandatory for Uncontrolled Object Instances | -| [0043](./adr/0043-signalling-retention-time.md) | Signalling retention time | -| [0044](./adr/0044-signalling-timeouts.md) | Signalling timeout periods | -| [0046](./adr/0046-governance.md) | Governance | +| [0040](./adr/0040-tag-usability-enhancements.md) | Tag Usability Enhancements | +| [0041](./adr/0041-require-explicit-framerate.md) | Requiring explicit frame rates | +| [0042](./adr/0042-uncontrolled-object-instance-labels.md) | Make `label` Mandatory for Uncontrolled Object Instances | +| [0043](./adr/0043-signalling-retention-time.md) | Signalling retention time | +| [0044](./adr/0044-signalling-timeouts.md) | Signalling timeout periods | +| [0045](./adr/0045-flow-init-segments.md) | Support for init Segments in Flows | +| [0046](./adr/0046-governance.md) | Governance | | [0048](./adr/0048-media-integrity.md) | Integrity model for media in TAMS, and when interacting with other systems | \* Note: ADR 0004a was the unintended result of a number clash in the early development of TAMS which wasn't caught before publication diff --git a/docs/adr/0045-flow-init-segments.md b/docs/adr/0045-flow-init-segments.md new file mode 100644 index 00000000..485544cc --- /dev/null +++ b/docs/adr/0045-flow-init-segments.md @@ -0,0 +1,204 @@ +--- +status: "accepted" +--- +# Support for init Segments in Flows + +## Context and Problem Statement + +Various media formats employ initialisation segments (aka init segments) for communicating parameters required to configure decoders. +Notably, CMAF-compatible formats such as MPEG DASH and Fragmented MP4 (fMP4). +Init segments are normally communicated via a manifest, alongside media segments. +The decoder will fetch and process the init segment, before identifying the required media segments (which may be mid-stream) and fetching and processing those. + +From a TAMS perspective, init segments are not normally associated with any particular point of time. +A client must be able to write/read, or construct, an init segment independently of the timeline. + +## Decision Drivers + +We would like to: + +* support TAMS features (e.g. edit-by-reference) as fully as possible +* be compatible with as many decoders/players as possible +* place as little burden on implementations as possible with TAMS-specific functionality +* have no/minimal impact on existing (non-init segment based) implementations +* re-use existing patterns/functionality where possible/practical +* maintain, as far as possible, the "independently decodable Objects" property of TAMS +* allow the use of existing init segment extensions (e.g. C2PA) +* avoid specifying Flow parameters that are format-specific + +## Considered Options + +* Option 1: Generate init segments from essence parameters +* Option 2: Significantly expand essence parameters to fully capture everything +* Option 3: Create a Flow Segment with a never timerange for the init segment +* Option 4: Make the init segment a Flow-level property for which a URL can be generated +* Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated +* Option 6: Have a Flow that is the init segment Flow +* Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) +* Option 8: Embed the init segment in every Object + +## Decision Outcome + +Chosen option: Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated. + +Options 1, 2, 3, and 4 would not allow for Flows which have multiple init segments. +This would place additional restrictions on workflows that TAMS interfaces with, or would require transformation to/from a single-init segment rendition where multiple init segment renditions are required elsewhere. +Examples of such workflows are those which use multi-period manifests, and those which use certain profiles of C2PA. +Additionally, edit-by-reference workflows would be unduly restricted by the need for a single init segment per Flow. + +Option 1, 2, 3, and 7 were considered particularly undesirable in terms of how they interacted with the TAMS data model, clashed with existing patterns, or would have a poor developer experience. + +Option 6 would require implementations generic operations (e.g. store transfer) to have specific understanding/behaviour of media types to operate correctly. +It may also break existing implementations of such functionality silently in some cases (e.g. media segments copied without init segments), or result in unexpected behaviour. + +Option 8 would again place restrictions on workflows that TAMS interfaces with, or would require transformation to/from an embedded-segment rendition. +Such an approach was also identified as being expressly prohibited by some media formats. +Real-world issues were also identified regarding processing load, and smoothness of playback with some player/decoder implementations where superfluous init segments are provided. + +Option 5 provides little/no restrictions on broader workflows. +It provides good backwards compatibility. +It provides good feature compatibility with the likes of edit-by-reference. +It provides minimal burden on client implementations. +It re-uses existing patterns, keeping the burden on service implementations as low as possible. + +### Implementation + +Implemented by . + +## Pros and Cons of the Options + +### Option 1: Generate init segments from essence parameters + +This option would see reading clients (or the store) generate their own init segments based on the existing technical metadata in the Flow metadata. +This is likely not possible with TAMS currently. +The TAMS API likely does not capture all metadata media formats may need to include in init segments. + +* Good, because it requires no changes to the API specification +* Good, because it supports features like edit-by-reference without any modification +* Bad, because initial research indicates this will not be possible due to missing metadata +* Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise +* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing + +### Option 2: Significantly expand essence parameters to fully capture everything + +This option would see Option 1 extended to include the addition of required parameters to Flow metadata. + +* Good, because it supports features like edit-by-reference without any modification +* Bad, because it will likely require significant additions to Flow technical metadata +* Bad, because it will likely require the addition of format specific parameters to Flow technical metadata +* Bad, because it requires a potentially complex implementation that requires in-depth knowledge of the structure of init segments and significant domain expertise +* Bad, because extensions such as C2PA metadata may need to be re-created on ingest due to the sequence of init and media segments changing + +### Option 3: Create a Flow Segment with a never timerange for the init segment + +The TAMS TimeRange format has a special "never" value - `()`. +This currently has no meaning for Flow Segments, as media segments always have a duration. +Init segments can be considered to have no duration. +This option would see the never TimeRange used to represent init segments. + +* Good, because it requires little/no spec changes +* Good, because it doesn't require format-specific Flow parameters +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it features like edit-by-reference would require a different workflow to non-init-segment Flows +* Bad, because it doesn't allow for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Bad, because the approach is somewhat unintuitive +* Bad, because it may result in unexpected behaviour with existing TAMS implementations + +### Option 4: Make the init segment a Flow-level property for which a URL can be generated + +This option would see the init segment managed via the existing Object CRUD mechanisms. +But rather than being registered against a Flow Segment, it would be registered against a new Flow property. +Existing Object patterns would be used for upload, URL generation, re-use, delete, etc. +Like Flow Segments, different Flows could re-use the same init Object. +Implementations might use the init segment's Object ID to determine the compatibility of Flow Segments in edit-by-reference workflows. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Good, because it supports features like edit-by-reference without any modification + * Though additional checks for Flow compatibility may be required +* Neutral, because it requires a backwards-compatible addition to the spec +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it doesn't allow for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters + +### Option 5: Make the init segment an object-level property, which can be uploaded and have a URL generated + +As with Option 4, this option would see the init segment managed via the existing Object CRUD mechanisms. +Rather than associating the init Object with the Flow, this option would see it associated with each Media Object. +A change in the init segment's Object ID may be used to determine a need to re-provision the init segment in the consuming client. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Good, because it supports features like edit-by-reference without any modification +* Neutral, because it requires a backwards-compatible addition to the spec +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it will result in a significant increase in Object metadata which makes up a large proportion of data stored in TAMS + +### Option 6: Have a Flow that is the init segment Flow + +This option would see a new init segment Flow type. +An init Flow would be associated with the media Flow. +Init segments would be assigned the TimeRange over which they are valid in the associated media Flow. +This would allow for multiple init segments to be applied to different parts of the media Flow's timeline. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Neutral, because it requires a backwards-compatible addition to the spec +* Neutral, because it would require open-ended Flow Segments to support live Flows +* Neutral, because extensions such as C2PA metadata should be supported as the sequence of init and media segments isn't changed +* Bad, because it features like edit-by-reference would require a different workflow to non-init-segment Flows +* Bad, because it may be unintuitive +* Bad, because it requires Objects from multiple Flows (init segment Flow, and media segment Flow) to decode a give piece of media + +### Option 7: Put the init segment somewhere on the TAMS API instead of as an object (e.g. as a base64 blob) + +As with Options 4, 5, or 6. +But the init segment would be stored as a base 64 encoded blob (or similar) that may be held in the backend database, instead of as file stored in an object store. + +In addition to the base options: + +* Bad, because it would represent a new pattern where one isn't strictly required +* Bad, because it would require a different GET path to media segments +* Bad, because it would likely require additional work on the part of clients to construct a virtual file for the init segment to pass to decoders/players + +### Option 8: Embed the init segment in every Object + +As with Option 5, but with the init segment embedded in the Object file itself, rather than as a property against the Object. +This would make each Object entirely independently decodable, maintaining this important property of TAMS. +Relevant formats have a concept of "self-initialising" media segments. +But these are, in at least some formats, explicitly not allowed for segmented media such as is used in TAMS. +And are only permitted for playlist-style use cases where a media segment is used to contain a full file. + +* Good, because it doesn't require format-specific Flow parameters +* Good, because it re-uses existing patterns/mechanisms +* Good, because it requires no changes to the API specification +* Good, because it allows for different init segments for different parts of the Flow + * Relevant formats support multiple init-segments via features such as "multi-Period" manifests + * Some relevant formats may use init segments to signal minor changes to encoding parameters +* Good, because it supports features like edit-by-reference without any modification +* Bad, because media using extensions such as C2PA metadata may need to be re-encoded/re-packaged on ingest due to the sequence of init and media segments changing +* Bad, because it will result in an increase in the size of Objects, which makes up a large proportion of data stored in TAMS + * Such an increase is likely to be minor for individual Objects, but significant in larger deployments with relatively small Segment sizes + * Note: This point is in comparison to the other Options in the ADR, rather than to the current situation in the TAMS ecosystem where equivalent metadata is already stored in Objects +* Bad, because this approach may result in un-needed overheads or even interrupted playback in some player implementations + * Some relevant implementations of player technologies, such as Media Source Extensions, are known to re-initialise decoders on (superfluous) new init segments which may result in disruption to playback + +### Option 9: Do not add support for init segments + +Many/all of the above options require significant changes to how the TAMS API is used, or requires formats such as fMP4 to be used in non-standard ways. +This may lead to fragmentation of the TAMS ecosystem, or require fMP4 codecs to be modified in a manner that requires significant in-depth domain knowledge. +We have, however, seen multiple off-spec extensions to the TAMS specification to add init-segment support which itself exacerbates the issue of fragmentation of the TAMS ecosystem. + +* Good, because it encourages TAMS interoperability by maintaining a small number of workflow variations. +* Neutral, because we would explicitly not support a format commonly used in the media industry + * If the way the format would be supported would be non-standard from an fMP4 point of view, TAMS support may bring its own issues - hence "neutral" +* Bad, because this option may persist the existing situation with off-spec extensions diff --git a/docs/appnotes/0024-using-init-segments.md b/docs/appnotes/0024-using-init-segments.md new file mode 100644 index 00000000..83b2b1c7 --- /dev/null +++ b/docs/appnotes/0024-using-init-segments.md @@ -0,0 +1,76 @@ +# 0024: Using Media with Initialisation Segments in TAMS + +> [!NOTE] +> There is an unfortunate overlap in terminology between TAMS and CMAF/DASH/HLS/fMP4. +> As such, where Segments or Flow Segments are referred to capitalised this document is referring to the TAMS resource types. +> Where init segments or media segments are referred to in lower case, this document is referring to the CMAF resource types. +> Where this document refers to Media Objects or Objects, it is referring to the TAMS Media Object resource type. +> Objects (without Media prepended) has been preferred when referring Objects containing initialisation segments to avoid potential confusion "init Media Object" may cause. +> There is no implied distinction between Media Object or Objects in the TAMS API. + +>[!NOTE] +> This Application Note refers to CMAF, MPEG-DASH, HLS, and fMP4. +> It also uses terms of art from those formats. +> It should, however, be read as applying to the general case of any format that uses equivalent concepts of initialisation segments. + +> [!IMPORTANT] +> This Application Note assumes basic knowledge of how to write/read media to/from a TAMS service. + +Some media formats, such as [CMAF](https://www.iso.org/standard/85623.html) compatible MPEG-DASH and fMP4, split content into initialisation (init) segments and media segments. +The init segments contain parameters used to configure decoders common across the stream of media +The media segments generally contain the media itself with additional configuration parameters specific to that media segment. +The split of configuration parameters between init and media segments varies according to the profile used. +When the configuration changes, a new init segment may be required. +While a change in technical parameters would generally require a new Flow in TAMS, some init segment changes may be minor codec parameter changes which would not warrant a new Flow. +A change in init-segment will normally result in a new "period" in DASH, or a "discontinuity" in HLS. +Other formats may have similar concepts. + +Additionally, technologies implemented as extensions to init segments may result in a new init segment for non-technical changes. +One example being C2PA signing of live media (specifically fragmented MP4, applicable in segmented TAMS use cases). +C2PA uses cryptography to verify the provenance of media. +One of the two C2PA live profiles places cryptographic keys in the init segment which may be used to verify media segments and their sequencing. +When that cryptographic key is rotated, a new init segment will be published. +See [C2PA Section 19](https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html#live-video) for more information on using C2PA with live media. + +TAMS support for init segments makes use of our existing Media Object resource to store and re-use init segments. +These are then associated via their ID to media segment Objects. +This allows existing Media Object re-use mechanisms (i.e. edit-by-reference) to continue to function. +Init segments follow the media. +A client may then detect if the init segment has changed by observing if the init Object ID remains the same or changes between subsequent Flow Segments. + +> [!IMPORTANT] +> Encoder implementations/configurations should minimise the creation of new init segments as much possible. +> Over-use of init segments may result in un-needed re-initialisation of decoders which may adversely affect performance and/or playback. + +## General Workflows + +The general workflows described here refer to init and media segments in isolation from manifests used by specific media formats. +The approaches have, however, been designed to map to manifests where required. + +### Write + +1. [Register the Flow](https://bbc.github.io/tams/main/index.html#/operations/PUT_flows-flowId) + * The `init_segments` essence parameter MUST be set to `true` + * The `container` parameter MUST be set to the mime type of the media segments +2. [Request storage for the init segment(s)](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-storage) + * The `content-type` MUST be set to the mime type of the init segment(s) + * If the mime type of the init segment(s) matches that of the media segments, this step may be skipped and storage for media and init segments requested together +3. Upload the init segment(s) +4. [Request storage for the media segments](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-storage) + * `content-type` MUST NOT be set +5. Upload the media segments +6. [Register the media segments as Flow Segments](https://bbc.github.io/tams/main/index.html#/operations/POST_flows-flowId-segments) + * `object_id` must be set to the Object ID of the media segment + * `init_object_id` must be set to the Object ID of the init segment + * Init segment objects should be reused wherever possible + +### Read + +1. [Get the Flow Segments listing for the Flow](https://bbc.github.io/tams/main/index.html#/operations/GET_flows-flowId-segments) +2. For each Flow Segment + 1. If init segment Object ID has changed + 1. Get init segment Object + 2. (Re)initialise decoder with init segment + 2. Get media segment Object + 3. Perform [media timeline mapping](https://github.com/bbc/tams/#flow-and-media-timelines) as usual in TAMS + 4. Play media segment