Repository navigation
[FIX] Escape special characters in WebVTT and SMPTE-TT output (#2437) - #2438
Aditya-Saroha wants to merge 1 commit into
Conversation
cfsmp3
left a comment
There was a problem hiding this comment.
This is Carlos' claude doing the first pass. Before Carlos takes a look, please address these issues.
-
Braces. The new entity selection in
get_decoder_line_encoded()(ccx_encoders_helpers.c) is anif / else ifchain with one-line bodies and no braces. Everyif/elsebody needs{ }. -
The fix only covers CEA-608.
write_cc_buffer_as_webvtt()and the 608 SMPTE-TT path now escape, but the other writers into the same formats don't:write_cc_subtitle_as_webvtt()andwrite_cc_subtitle_as_smptett()(teletext, ISDB, DVB OCR text)write_stringz_as_webvtt()andwrite_stringz_as_smptett()(--startcreditstext/--endcreditstext)
A teletext or credits line containing
&or<still produces a broken WebVTT cue or non-well-formed SMPTE-TT XML. Please escape in all of them. Teletext text can already contain the<font color=…>/<i>tags that telxcc inserts, so the escaping has to apply to the character data only, not to those tags. Please show the result on a teletext sample as well. -
Please attach the sample to the PR (or link it directly), and show the before/after for each writer you change.
…tractor#2438) - Add curly braces to if/else if branches in get_decoder_line_encoded() - Implement escape_markup_text() to escape bare '&', '<', and '>' in character data while preserving markup tags (<font>, <i>, <b>, <u>, etc.) and valid HTML/XML entities - Escape subtitle lines in write_stringz_as_webvtt() and write_stringz_as_smptett(), covering teletext, ISDB, DVB text, and credits (--startcreditstext / --endcreditstext) - Escape OCR text in write_cc_bitmap_as_webvtt() and write_cc_bitmap_as_smptett()
…tractor#2438) - Add curly braces to if/else if branches in get_decoder_line_encoded() - Implement escape_markup_text() to escape bare '&', '<', and '>' in character data while preserving markup tags (<font>, <i>, <b>, <u>, etc.) and valid HTML/XML entities - Escape subtitle lines in write_stringz_as_webvtt() and write_stringz_as_smptett(), covering teletext, ISDB, DVB text, and credits (--startcreditstext / --endcreditstext) - Escape OCR text in write_cc_bitmap_as_webvtt() and write_cc_bitmap_as_smptett()
1f4af45 to
aadca93
Compare
…tractor#2438) - Add curly braces to if/else if branches in get_decoder_line_encoded() - Implement escape_markup_text() to escape bare '&', '<', and '>' in character data while preserving markup tags (<font>, <i>, <b>, <u>, etc.) and valid HTML/XML entities - Escape subtitle lines in write_stringz_as_webvtt() and write_stringz_as_smptett(), covering teletext, ISDB, DVB text, and credits (--startcreditstext / --endcreditstext) - Escape OCR text in write_cc_bitmap_as_webvtt() and write_cc_bitmap_as_smptett()
aadca93 to
94e5834
Compare
…actor#2437) WebVTT and SMPTE-TT outputs did not properly escape special XML/HTML characters ('&', '<', '>'), causing XML parsers to reject SMPTE-TT documents as malformed and browser rendering engines to drop or misinterpret tags in WebVTT cues (e.g. '<BOB>'). Key changes: - Escape at plain-text sources: ISDB text (ccx_decoders_isdb.c), MP4 tx3g subtitles (mp4.c and mp4_rust_bridge.c), start/end credits text (ccx_encoders_common.c), and bitmap OCR (ccx_encoders_webvtt.c and ccx_encoders_smptett.c). - Escape in CEA-608 writers: in get_decoder_line_encoded() (ccx_encoders_helpers.c), escape characters when targeting WebVTT or SMPTE-TT. Add explicit braces to all conditional branches. - Teletext handling: teletext already tags formatting (<font>, <i>, <br/>) and escapes character data via ENTITIES[] in telxcc.c. Pass-through writers (write_stringz_as_webvtt, write_stringz_as_smptett) remain pure pass-through to prevent double-escaping. Fixed -nofc mode in telxcc.c where character data previously bypassed escaping. - Memory safety: add OOM checks with fatal(EXIT_NOT_ENOUGH_MEMORY, ...) on all new escaping allocations. - Non-XML formats: SRT, SSA, SAMI, etc. remain strictly untouched and byte-identical to master. Fixes CCExtractor#2437
94e5834 to
50c81ca
Compare
|
Hi Carlos, Thank you for the constructive and detailed review. Below is an itemized breakdown of how each point has been addressed, along with test evidence, CI diff analysis, and architectural details. 1. Addressing Review PointsPoint 1: Braces in
|
CCExtractor CI platform finished running the test files on linux. 171/238 tests matched the approved output:
67 tests do not match the approved output. That is the pass/fail verdict. Whether this branch caused it is a separate question, answered below.
Compared with the tip of master — test 9978, commit f4519ff:
Pass there, fail here:
Compared with the commit this branch was cut from: the same run as the tip of master (test 9978), so the comparison above already covers it. This branch changes the behaviour of 1 test(s) relative to the tip of master. Those are the ones worth looking at; anything else in the list fails the same way on both sides. |
CCExtractor CI platform finished running the test files on windows. 171/238 tests matched the approved output:
67 tests do not match the approved output. That is the pass/fail verdict. Whether this branch caused it is a separate question, answered below.
Compared with the tip of master — test 9977, commit 672f012:
Pass there, fail here:
Compared with the commit this branch was cut from: the same run as the tip of master (test 9977), so the comparison above already covers it. This branch changes the behaviour of 1 test(s) relative to the tip of master. Those are the ones worth looking at; anything else in the list fails the same way on both sides. |
In raising this pull request, I confirm the following (please check boxes):
Reason for this PR:
Sanity check:
Repro instructions:
Sample:
markup.binattached to issue #2437 (a 77-byte RCWT sample containing<BOB> & HI).Commands to reproduce:
ccextractor markup.bin --out=webvtt -o markup.vtt ccextractor markup.bin --out=smptett -o markup.xml python3 -c 'import xml.etree.ElementTree as E; E.parse("markup.xml")'Before fix:
After fix:
Description of Changes
Fixes #2437.
Verification