Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ jobs:
toxenv: ['django42', 'django52']

steps:
- uses: actions/checkout@v4
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- name: setup python
uses: actions/setup-python@v5
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ matrix.python-version }}

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
- name: setup python
uses: actions/setup-python@v5
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: 3.12

Expand Down
1 change: 0 additions & 1 deletion requirements/base.in
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,3 @@

django
babel>=1.3
enmerkar==0.7.1
11 changes: 3 additions & 8 deletions requirements/base.txt
Original file line number Diff line number Diff line change
@@ -1,21 +1,16 @@
#
# This file is autogenerated by pip-compile with Python 3.11
# This file is autogenerated by pip-compile with Python 3.14
# by the following command:
#
# make upgrade
# pip-compile --allow-unsafe --output-file=requirements/base.txt requirements/base.in
#
asgiref==3.10.0
# via django
babel==2.17.0
# via
# -r requirements/base.in
# enmerkar
# via -r requirements/base.in
django==4.2.25
# via
# -c https://raw.githubusercontent.com/edx/edx-lint/master/edx_lint/files/common_constraints.txt
# -r requirements/base.in
# enmerkar
enmerkar==0.7.1
# via -r requirements/base.in
sqlparse==0.5.3
# via django
7 changes: 0 additions & 7 deletions requirements/tox.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ asgiref==3.10.0
babel==2.17.0
# via
# -r requirements/base.in
# enmerkar
# sphinx
certifi==2025.10.5
# via requests
Expand All @@ -22,14 +21,8 @@ coverage[toml]==7.11.0
# -r requirements/test.in
# pytest-cov
# python-coveralls
# via
# -c https://raw.githubusercontent.com/edx/edx-lint/master/edx_lint/files/common_constraints.txt
# -r requirements/base.in
# enmerkar
docutils==0.21.2
# via sphinx
enmerkar==0.7.1
# via -r requirements/base.in
execnet==2.1.1
# via pytest-cache
flake8==7.3.0
Expand Down
2 changes: 1 addition & 1 deletion src/enmerkar_underscore/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from django.template.base import TOKEN_TEXT

from django.utils.encoding import force_str
from enmerkar.extract import extract_django
from .vendor.enmerkar.enmerkar_extract import extract_django

from .vendor.markey import underscore
from .vendor.markey.machine import parse_arguments, tokenize
Expand Down
30 changes: 30 additions & 0 deletions src/enmerkar_underscore/vendor/enmerkar/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
Copyright (C) 2013-2014 django-babel Team
Copyright (C) 2019 Extracover Holdings Limited

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
3. The name of the author may not be used to endorse or promote
products derived from this software without specific prior
written permission.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
174 changes: 174 additions & 0 deletions src/enmerkar_underscore/vendor/enmerkar/enmerkar_extract.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense, we eventually want to fully drop the django underscore tooling but this is a good intermediate step to reduce un-necessary dependencies by taking on some more probably fairly minimal maintenance burden in the near term.

When you're vendoring in code from other libraries, it's really important to read their License. In this case, we should copy the COPYING file next to this file in the vendor directory.

I think it's worth it to rename COPYING to LICENSE here for clarity and to put the vendored enmerkar code into a folder named for the library it's coming from. It's unlikely that we'll have multiple vendored items in this repo but it's good practice to have a separate vendor folder for each upstream item you're vendoring. It has the side benefit that we can keep the name of this file the same as the upstream file it's copying which also makes provenance easier.

Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
# Vendored from https://github.com/Zegocover/enmerkar (enmerkar 0.7.1)
# enmerkar/__init__.py uses pkg_resources (a setuptools artifact not guaranteed
# to be present in Python 3.12+ environments). We only need extract_django, and
# enmerkar/extract.py has no such dependency, so we vendor it here directly.
from django.template.base import Lexer
from django.template.base import TokenType

from django.utils.translation import trim_whitespace
from django.utils.encoding import smart_str

from django.utils.translation.template import (
inline_re, block_re, endblock_re, plural_re, constant_re)


TOKEN_TEXT = TokenType.TEXT
TOKEN_VAR = TokenType.VAR
TOKEN_BLOCK = TokenType.BLOCK


def join_tokens(tokens, trim=False):
message = ''.join(tokens)
if trim:
message = trim_whitespace(message)
return message


def strip_quotes(s):
if (s[0] == s[-1]) and s.startswith(("'", '"')):
return s[1:-1]
return s


def extract_django(fileobj, keywords, comment_tags, options):
"""Extract messages from Django template files.

:param fileobj: the file-like object the messages should be extracted from
:param keywords: a list of keywords (i.e. function names) that should
be recognized as translation functions
:param comment_tags: a list of translator tags to search for and
include in the results
:param options: a dictionary of additional options (optional)
:return: an iterator over ``(lineno, funcname, message, comments)``
tuples
:rtype: ``iterator``
"""
intrans = False
inplural = False
trimmed = False
message_context = None
singular = []
plural = []
lineno = 1

encoding = options.get('encoding', 'utf8')
text = fileobj.read().decode(encoding)

try:
text_lexer = Lexer(text)
except TypeError:
# Django 1.9 changed the way we invoke Lexer; older versions
# require two parameters.
text_lexer = Lexer(text, None)

for t in text_lexer.tokenize():
lineno += t.contents.count('\n')
if intrans:
if t.token_type == TOKEN_BLOCK:
endbmatch = endblock_re.match(t.contents)
pluralmatch = plural_re.match(t.contents)
if endbmatch:
if inplural:
if message_context:
yield (
lineno,
'npgettext',
[smart_str(message_context),
smart_str(join_tokens(singular, trimmed)),
smart_str(join_tokens(plural, trimmed))],
[],
)
else:
yield (
lineno,
'ngettext',
(smart_str(join_tokens(singular, trimmed)),
smart_str(join_tokens(plural, trimmed))),
[])
else:
if message_context:
yield (
lineno,
'pgettext',
[smart_str(message_context),
smart_str(join_tokens(singular, trimmed))],
[],
)
else:
yield (
lineno,
None,
smart_str(join_tokens(singular, trimmed)),
[])

intrans = False
inplural = False
message_context = None
singular = []
plural = []
elif pluralmatch:
inplural = True
else:
raise SyntaxError('Translation blocks must not include '
'other block tags: %s' % t.contents)
elif t.token_type == TOKEN_VAR:
if inplural:
plural.append('%%(%s)s' % t.contents)
else:
singular.append('%%(%s)s' % t.contents)
elif t.token_type == TOKEN_TEXT:
if inplural:
plural.append(t.contents)
else:
singular.append(t.contents)
else:
if t.token_type == TOKEN_BLOCK:
imatch = inline_re.match(t.contents)
bmatch = block_re.match(t.contents)
cmatches = constant_re.findall(t.contents)
if imatch:
g = imatch.group(1)
g = strip_quotes(g)
message_context = imatch.group(3)
if message_context:
# strip quotes
message_context = message_context[1:-1]
yield (
lineno,
'pgettext',
[smart_str(message_context), smart_str(g)],
[],
)
message_context = None
else:
yield lineno, None, smart_str(g), []
elif bmatch:
if bmatch.group(2):
message_context = bmatch.group(2)[1:-1]
for fmatch in constant_re.findall(t.contents):
stripped_fmatch = strip_quotes(fmatch)
yield lineno, None, smart_str(stripped_fmatch), []
intrans = True
inplural = False
trimmed = 'trimmed' in t.split_contents()
singular = []
plural = []
elif cmatches:
for cmatch in cmatches:
stripped_cmatch = strip_quotes(cmatch)
yield lineno, None, smart_str(stripped_cmatch), []
elif t.token_type == TOKEN_VAR:
parts = t.contents.split('|')
cmatch = constant_re.match(parts[0])
if cmatch:
stripped_cmatch = strip_quotes(cmatch.group(1))
yield lineno, None, smart_str(stripped_cmatch), []
for p in parts[1:]:
if p.find(':_(') >= 0:
p1 = p.split(':', 1)[1]
if p1[0] == '_':
p1 = p1[1:]
if p1[0] == '(':
p1 = p1.strip('()')
p1 = strip_quotes(p1)
yield lineno, None, smart_str(p1), []
Loading