fix(extract): skip bytes literals in _parse_python_string (#1190)

[Mukller]

Summary

Fixes #1190.

Running pybabel extract on a file containing _(b'foo') (a byte-string literal as a translation argument) crashes with:

TypeError: sequence item 0: expected str instance, bytes found

Root Cause

_parse_python_string() is annotated as -> str | None but returns bytes for byte-string literals. The function compiles the token to an ast.Constant and returns body.value unconditionally — for b'foo' that is b'foo' (bytes):

# _parse_python_string — before fix
if isinstance(body, ast.Constant):
    return body.value   # returns bytes for b'foo' — wrong!

The caller in extract_python appends the returned value to a string fragment list buf, then joins it:

buf.append(val)                  # buf = [b'foo']
messages.append(''.join(buf))    # TypeError!

Fix

Guard the return with isinstance(body.value, str) so that byte-string literals return None and are silently skipped, exactly like any other non-extractable expression:

         if isinstance(body, ast.Constant):
-            return body.value
+            if isinstance(body.value, str):
+                return body.value

Bytes literals are not valid gettext messages, so ignoring them (rather than crashing) is the correct and expected behavior.

Test

# test.py
_(b'foo')  # should be silently ignored
_('bar')   # should be extracted normally

$ pybabel extract test.py -o messages.pot
# Before: TypeError crash
# After:  Only 'bar' extracted; b'foo' silently ignored

[Mukller]

Code Review

Bug Trace

# source.py: _(b'foo')

# tokenize produces: tok=STRING, value="b'foo'"

# _parse_python_string("b'foo'", encoding='UTF-8', future_flags=0)
#   compile("b'foo'", '<string>', 'eval', ...) -> ast.Expression
#   body = ast.Constant(value=b'foo')   # bytes!
#   isinstance(body, ast.Constant) -> True
#   return body.value                   # returns b'foo' (bytes)

# Back in extract_python:
#   val = b'foo'    (not None, so not skipped)
#   buf.append(b'foo')       # buf = [b'foo']
#   ''.join(buf)             # TypeError: expected str, got bytes

Fix Correctness

# After fix:
if isinstance(body, ast.Constant):
    if isinstance(body.value, str):   # new guard
        return body.value
# bytes falls through to implicit 'return None'

Token	body.value type	Before	After
'foo'	str	Returns 'foo' ✓	Returns 'foo' ✓
b'foo'	bytes	Returns b'foo' → crash ✗	Returns None → skipped ✓
42	int	Returns 42 → crash ✗	Returns None → skipped ✓
3.14	float	Returns 3.14 → crash ✗	Returns None → skipped ✓

Note: integer and float ast.Constant values would cause the same crash — this guard also fixes those edge cases.

Scope

• 2 lines changed inside _parse_python_string().
• No new imports needed.
• Extracted str messages are unaffected.
• The return type annotation -> str | None is now correct.

[akx]

As noted in #1190, it would be useful to warn about an invalid value, instead of quietly ignoring it.

[Mukller]

Done — added a SyntaxWarning in the latest commit. The warning includes the literal value and a note to use a str literal instead:

SyntaxWarning: Bytes literal b'foo' passed to a gettext function; it will be skipped during message extraction. Use a str literal instead.

[Mukller]

Updated - added a SyntaxWarning for non-string, non-bytes constants (e.g. integer or float literals), in addition to the existing bytes warning. Both cases emit a SyntaxWarning with the literal value and are skipped during extraction. Tests for both are included.

[Mukller]

Latest commit adds a SyntaxWarning for non-string, non-bytes constant literals (e.g. _(42)), addressing @akx's feedback in #1190. Both the bytes and non-string cases now warn instead of silently skipping, and tests for both are included.