Skip to content

Commit 4382a5b

Browse files
haffi96claude
andcommitted
Add LengthPrefixedAccessUnitParser for length-prefixed H.264/H.265
EncodedVideoSource takes Annex-B access units, but hardware encoder bridges commonly deliver 4-byte length-prefixed NAL units (avc3/hev1). The parser accepts any chunking of such a stream and returns EncodedVideoSource::Frame values: Annex-B (converted in place during the single copy), keyframe flag, and optional packet-trailer metadata from an LKTS SEI (the LiveKit Go SDK convention). single_slice_flush completes an access unit as soon as its slice arrives instead of when the next one begins, which otherwise holds every picture for a frame interval. Mirrors livekit-capture's LengthPrefixedAccessUnitParser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 792687a commit 4382a5b

5 files changed

Lines changed: 670 additions & 0 deletions

File tree

‎CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -430,6 +430,7 @@ add_library(livekit SHARED
430430
src/ffi_handle.cpp
431431
src/ffi_client.cpp
432432
src/ffi_client.h
433+
src/length_prefixed_access_unit_parser.cpp
433434
src/livekit.cpp
434435
src/logging.cpp
435436
src/local_audio_track.cpp
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
/*
2+
* Copyright 2026 LiveKit
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an “AS IS” BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
#pragma once
18+
19+
#include <cstddef>
20+
#include <cstdint>
21+
#include <optional>
22+
#include <vector>
23+
24+
#include "livekit/encoded_video_source.h"
25+
#include "livekit/video_codec.h"
26+
#include "livekit/visibility.h"
27+
28+
namespace livekit {
29+
30+
/// @brief Splits a length-prefixed H.264 (AVC) or H.265 (HVCC) NAL unit
31+
/// stream into access units ready for EncodedVideoSource::captureFrame.
32+
///
33+
/// This is the framing of `stream-format=avc3` (H.264) and `hev1` (H.265),
34+
/// with parameter sets in-band. Each access unit is converted to Annex-B,
35+
/// which the pre-encoded passthrough requires.
36+
///
37+
/// Push bytes as they arrive (any chunking), then drain complete access
38+
/// units with next(). Call flush() once at the end of the stream. Not
39+
/// thread-safe.
40+
class LIVEKIT_API LengthPrefixedAccessUnitParser final {
41+
public:
42+
/// @brief Parser configuration.
43+
struct Options {
44+
/// Size of each big-endian NAL length prefix, 1 to 4 bytes
45+
/// (`lengthSizeMinusOne + 1` from `avcC`/`hvcC`; 4 for most encoders).
46+
std::uint8_t nal_length_size = 4;
47+
/// Complete each access unit as soon as its first slice arrives, instead
48+
/// of when the next access unit begins.
49+
///
50+
/// The end of an access unit is otherwise only known from the NAL that
51+
/// starts the next one, which holds every picture for a full frame
52+
/// interval unless the encoder terminates access units with an AUD.
53+
/// Enable this only for streams with one slice per picture, which is
54+
/// what hardware encoders typically emit. NAL units that follow the
55+
/// slice, such as an H.265 suffix SEI, are sent with the next picture.
56+
bool single_slice_flush = false;
57+
/// Fill Frame::metadata from an LKTS `user_data_unregistered` SEI ahead
58+
/// of the slice (the packet-trailer convention of the LiveKit Go SDK).
59+
bool frame_metadata_from_sei = false;
60+
};
61+
62+
/// @brief Creates a parser with default options.
63+
/// @param codec VideoCodec::H264 or VideoCodec::H265.
64+
/// @throws std::invalid_argument if the codec is not H.264 or H.265.
65+
explicit LengthPrefixedAccessUnitParser(VideoCodec codec);
66+
/// @brief Creates a parser.
67+
/// @param codec VideoCodec::H264 or VideoCodec::H265.
68+
/// @param options Parser configuration.
69+
/// @throws std::invalid_argument if the codec is not H.264 or H.265, or
70+
/// nal_length_size is not 1 to 4.
71+
LengthPrefixedAccessUnitParser(VideoCodec codec, Options options);
72+
73+
/// @brief Appends stream bytes.
74+
/// @param data Bytes to append.
75+
/// @param size Number of bytes at data.
76+
void push(const std::uint8_t* data, std::size_t size);
77+
78+
/// @brief Returns the next complete access unit, if one is buffered.
79+
///
80+
/// Frame::width/height are left at zero (the source resolution) and
81+
/// Frame::timestamp_us at zero (capture time).
82+
/// @return The access unit, or nullopt if none is complete yet.
83+
/// @throws std::runtime_error on malformed input, or when an access unit
84+
/// exceeds 32 MiB.
85+
std::optional<EncodedVideoSource::Frame> next();
86+
87+
/// @brief Returns the remaining buffered NAL units as the final access unit.
88+
/// Call repeatedly until it returns nullopt. A trailing group of NAL units
89+
/// without a slice, such as a lone AUD, is dropped.
90+
/// @return The next access unit, or nullopt once the stream is drained.
91+
/// @throws std::runtime_error on a truncated final NAL unit.
92+
std::optional<EncodedVideoSource::Frame> flush();
93+
94+
private:
95+
struct NalRange {
96+
std::size_t start;
97+
std::size_t end;
98+
};
99+
100+
void scanPending();
101+
std::optional<std::size_t> splitIndex() const;
102+
EncodedVideoSource::Frame takeAccessUnit(std::size_t nal_count);
103+
104+
VideoCodec codec_;
105+
Options options_;
106+
std::vector<std::uint8_t> pending_;
107+
// Complete NAL units of the access unit being assembled, as payload
108+
// ranges (after the length prefix) into pending_.
109+
std::vector<NalRange> nals_;
110+
// Offset of the first unparsed length prefix in pending_.
111+
std::size_t scan_cursor_ = 0;
112+
};
113+
114+
} // namespace livekit

‎include/livekit/livekit.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
#include "livekit/build.h"
2424
#include "livekit/e2ee.h"
2525
#include "livekit/encoded_video_source.h"
26+
#include "livekit/length_prefixed_access_unit_parser.h"
2627
#include "livekit/local_audio_track.h"
2728
#include "livekit/local_participant.h"
2829
#include "livekit/local_track_publication.h"

0 commit comments

Comments
 (0)