Starter full-text search updates v830 - #23615
Conversation
ti-chi-bot
Bot
added
contribution
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
the
size/XXL
📝 WalkthroughWalkthroughAdded full-text search documentation for indexes, ChangesFull-Text Search Documentation
Local settings ignore
Estimated code review effort: 3 (Moderate) | ~20 minutes Merge Risk: 🟡 Moderate · up to The PR adds full-text search documentation but currently includes unsupported TiDB 8.5 syntax and conflicting usage guidance that could lead to failed setup or incorrect queries. Merge should wait until these documentation issues are corrected. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Description checkExplanation The description includes the required sections, but it provides only a vague change summary and does not select the affected TiDB version. The PR objective identifies TiDB 8.5, but the description does not. Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (6 skipped: 6 unsupported.)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@zhaoshangzi: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 71138577-d8fc-4544-bd61-c843658eaf38
📒 Files selected for processing (6)
.gitignoreTOC-ai.mdai/guides/vector-search-full-text-search-sql.mdai/reference/full-text-search-functions.mdai/reference/full-text-search-index.mdai/reference/full-text-search-limitations.md
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.
| -- ADD_COLUMNAR_REPLICA_ON_DEMAND is optional. If you omit it, | ||
| -- make sure that a TiFlash replica is already created for the table. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
printf '%s\n' '--- applicable repository guidance ---'
find /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c -type f -name '*.md' -maxdepth 3 -print
printf '%s\n' '--- target excerpt ---'
sed -n '55,100p' ai/guides/vector-search-full-text-search-sql.md
printf '%s\n' '--- related clause references ---'
rg -n -C 3 'ADD_COLUMNAR_REPLICA_ON_DEMAND|FULLTEXT INDEX|TiFlash replica' ai resources docs 2>/dev/null | head -160Repository: pingcap/docs
Length of output: 14857
🌐 Web query:
TiDB 8.5 ADD_COLUMNAR_REPLICA_ON_DEMAND FULLTEXT INDEX parse error issue 20964
💡 Result:
The issue referenced as 20964 in the context of TiDB's full-text search and the ADD_COLUMNAR_REPLICA_ON_DEMAND clause is a documentation/user experience issue identified in the pingcap/docs repository [1]. It specifically reports that executing an ALTER TABLE statement containing both a full-text index creation and the ADD_COLUMNAR_REPLICA_ON_DEMAND clause (e.g., ALTER TABLE t_text ADD FULLTEXT INDEX(ct) WITH PARSER MULTILINGUAL ADD_COLUMNAR_REPLICA_ON_DEMAND;) results in a SQL parsing error [1]. This error occurs because the TiDB SQL parser has historically not supported this specific combined syntax in certain environments, such as the TiDB Cloud SQL Editor [1]. As a workaround, users are advised to execute the operations in two separate steps [1]: 1. Set the TiFlash replica first: ALTER TABLE t_text SET TIFLASH REPLICA 2; [1] 2. Add the full-text index: ALTER TABLE t_text ADD FULLTEXT INDEX(ct) WITH PARSER MULTILINGUAL; [1] While some documentation examples may still display the combined syntax [2][3][4], it is known to fail in specific interfaces [1]. Development work has been ongoing to improve the parser to support such clauses [5]. It is important to note that the issue number 20964 in the pingcap/tidb repository refers to an unrelated historical pull request from 2020 regarding documentation links [6] and is not the same issue as the documentation issue 20964 in the pingcap/docs repository [1].
Citations:
- 1: full text index: Parsing sql failed, near "ADD_COLUMNAR_REPLICA_ON_DEMAND;" #20964
- 2: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 3: https://pingcap.github.io/ai/guides/fulltext-search/
- 4: https://github.com/pingcap/docs/blob/master/ai/guides/vector-search-full-text-search-sql.md
- 5: parser: Add ADD_COLUMNAR_REPLICA_ON_DEMAND tidb#61159
- 6: site: update the broken link about "Contribution Guide" in the README.md tidb#20964
🏁 Script executed:
printf '%s\n' '--- repository-wide review conventions ---'
cat /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c/conventions/repo-wide.md
printf '%s\n' '--- issue and parser-change metadata ---'
curl -L --fail --silent --show-error https://api.github.com/repos/pingcap/docs/issues/20964 | jq '{title,state,body,html_url,created_at,updated_at}'
curl -L --fail --silent --show-error https://api.github.com/repos/pingcap/tidb/pulls/61159 | jq '{title,state,merged_at,body,html_url,base:.base.ref,head:.head.ref}'Repository: pingcap/docs
Length of output: 6850
🌐 Web query:
site:docs.pingcap.com/tidb/stable ADD_COLUMNAR_REPLICA_ON_DEMAND FULLTEXT TiDB 8.5
💡 Result:
In TiDB, the ADD_COLUMNAR_REPLICA_ON_DEMAND clause is used in conjunction with adding a FULLTEXT index to ensure the necessary columnar (TiFlash) infrastructure is available to support full-text search capabilities [1]. Key points regarding this feature: 1. Purpose: Full-text search in TiDB relies on columnar storage (TiFlash) [1]. Including ADD_COLUMNAR_REPLICA_ON_DEMAND during the creation of a FULLTEXT index ensures that the required TiFlash replica is provisioned for the table [1]. 2. Availability: While the syntax for FULLTEXT search is parsed by most TiDB environments for compatibility, actual support for full-text indexes is currently limited to specific environments, such as TiDB Cloud Starter and Essential instances in select AWS regions [2][3][4][5]. 3. Syntax Usage: The clause is typically appended to the ALTER TABLE ADD FULLTEXT INDEX statement [1]: ALTER TABLE table_name ADD FULLTEXT INDEX (column_name) WITH PARSER parser_name ADD_COLUMNAR_REPLICA_ON_DEMAND; The supported parsers for the WITH PARSER clause include STANDARD (for English, space/punctuation-based) and MULTILINGUAL (for broader language support, including Chinese, Japanese, and Korean) [1].
Citations:
- 1: https://docs.pingcap.com/tidb/stable/vector-search-full-text-search-sql/
- 2: https://docs.pingcap.com/tidb/stable/sql-statement-add-index/
- 3: https://docs.pingcap.com/tidb/stable/sql-statement-create-index/
- 4: https://docs.pingcap.com/tidb/stable/sql-statement-create-table/
- 5: https://docs.pingcap.com/tidb/stable/mysql-compatibility/
🏁 Script executed:
set -e
printf '%s\n' '--- parser-support change ---'
curl -L --fail --silent --show-error https://api.github.com/repos/pingcap/tidb/pulls/61159/commits | jq '.[-1] | {sha,commit:{message,author}}'
curl -L --fail --silent --show-error https://api.github.com/repos/pingcap/tidb/pulls/61159 | jq '{merge_commit_sha,merged_at,base:.base.ref}'
printf '%s\n' '--- TiDB 8.5 refs ---'
git ls-remote --tags https://github.com/pingcap/tidb.git 'refs/tags/v8.5*' | tail -20
git ls-remote https://github.com/pingcap/tidb.git refs/heads/release-8.5
printf '%s\n' '--- parser change references on GitHub ---'
curl -L --fail --silent --show-error 'https://api.github.com/repos/pingcap/tidb/commits?path=parser/parser.y&sha=release-8.5&per_page=5' | jq '.[] | {sha,commit:{message,author}}'Repository: pingcap/docs
Length of output: 9054
🏁 Script executed:
set -e
for ref in v8.5.0 v8.5.1 v8.5.6 v8.5.7 release-8.5; do
printf '%s: ' "$ref"
url="https://raw.githubusercontent.com/pingcap/tidb/$ref/parser/parser.y"
if curl -L --fail --silent --show-error "$url" | rg -n -m 2 'ADD_COLUMNAR_REPLICA_ON_DEMAND'; then
:
else
printf '%s\n' 'token not found or parser/parser.y unavailable'
fi
done
printf '%s\n' '--- merge commit details ---'
curl -L --fail --silent --show-error https://api.github.com/repos/pingcap/tidb/commits/6148d608bf3b40fdfb455363aa0a32aa3e2ca962 | jq '{sha,commit:{message,author,date:.commit.author.date},parents:[.parents[].sha]}'Repository: pingcap/docs
Length of output: 2030
🏁 Script executed:
set -e
printf '%s\n' '--- files changed by parser PR ---'
curl -L --fail --silent --show-error 'https://api.github.com/repos/pingcap/tidb/pulls/61159/files?per_page=100' |
jq -r '.[].filename'
printf '%s\n' '--- matching parser files in v8.5.0 ---'
curl -L --fail --silent --show-error 'https://api.github.com/repos/pingcap/tidb/git/trees/v8.5.0?recursive=1' |
jq -r '.tree[] | select(.path|test("parser|Parser|grammar";"i")) | .path' |
head -80Repository: pingcap/docs
Length of output: 2697
🏁 Script executed:
set -e
for ref in v8.5.0 v8.5.1 v8.5.6 v8.5.7 release-8.5; do
printf '%s\n' "--- $ref ---"
for path in pkg/parser/parser.y pkg/parser/keywords.go pkg/parser/ast/ddl.go; do
printf '%s: ' "$path"
body=$(curl -L --fail --silent --show-error "https://raw.githubusercontent.com/pingcap/tidb/$ref/$path")
if printf '%s' "$body" | rg -n -m 3 'ADD_COLUMNAR_REPLICA_ON_DEMAND'; then
:
else
printf '%s\n' 'token not found'
fi
done
doneRepository: pingcap/docs
Length of output: 810
Remove the unsupported clause from the setup example.
TiDB 8.5 does not recognize ADD_COLUMNAR_REPLICA_ON_DEMAND. Create the TiFlash replica before adding the full-text index.
| -- ADD_COLUMNAR_REPLICA_ON_DEMAND is optional. If you omit it, | |
| -- make sure that a TiFlash replica is already created for the table. | |
| -- Ensure that a TiFlash replica is already created for the table. | |
| ALTER TABLE stock_items ADD FULLTEXT INDEX (title) WITH PARSER MULTILINGUAL; |
Source: MCP tools
| | Intended use case | Select among multiple indexes that use the same parser. This is a pure execution-path choice. | | ||
| | Conflicts with `WITH PARSER` in the function (the specified index does not contain that parser) | `USE_INDEX` is ignored. The parser in `WITH PARSER` takes precedence, and TiDB selects another index that contains that parser. | | ||
| | `WITH PARSER` is not specified | TiDB follows `USE_INDEX` and uses the specified index and its parser. | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Align USE_INDEX semantics with the examples.
Lines 151-156 use USE_INDEX to select NGRAM and MULTILINGUAL indexes. However, line 163 says that USE_INDEX is only for indexes with the same parser and is a pure execution-path choice. Lines 164-165 document parser selection through WITH PARSER or the selected index. Update the row to describe both behaviors consistently.
Suggested replacement
-| Intended use case | Select among multiple indexes that use the same parser. This is a pure execution-path choice. |
+| Intended use case | Select a full-text index for the query. Without `WITH PARSER`, the specified index also determines the parser; with `WITH PARSER`, the parser in the function takes precedence. |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| | Intended use case | Select among multiple indexes that use the same parser. This is a pure execution-path choice. | | |
| | Conflicts with `WITH PARSER` in the function (the specified index does not contain that parser) | `USE_INDEX` is ignored. The parser in `WITH PARSER` takes precedence, and TiDB selects another index that contains that parser. | | |
| | `WITH PARSER` is not specified | TiDB follows `USE_INDEX` and uses the specified index and its parser. | | |
| | Intended use case | Select a full-text index for the query. Without `WITH PARSER`, the specified index also determines the parser; with `WITH PARSER`, the parser in the function takes precedence. | | |
| | Conflicts with `WITH PARSER` in the function (the specified index does not contain that parser) | `USE_INDEX` is ignored. The parser in `WITH PARSER` takes precedence, and TiDB selects another index that contains that parser. | | |
| | `WITH PARSER` is not specified | TiDB follows `USE_INDEX` and uses the specified index and its parser. | |
| | `multilingual` | None | - | Language-aware tokenizer that matches complete tokens. Case-insensitive. | | ||
| | `ngram` | `min_gram`, `max_gram`, `granularity`, `lower_case` | `min_gram=3`, `max_gram=3`, `granularity='word'`, `lower_case=true` | Character-level n-gram tokenizer that supports prefix, infix, and suffix substring matching. See [The NGRAM parser](#the-ngram-parser). | | ||
| | `exact` | None | - | Exact-value matching using inverted posting lists. Supports `=` and `IN`. Used for tenant IDs, status, tags, and other low-cardinality filters. | | ||
| | `path_hierarchy` | `delimiter` | `delimiter='/'` | Hierarchical prefix matching for path-like values. Supports `col LIKE '/src/%'` and `col UNDER '/src/'`. The prefix must align with a delimiter boundary. See [path_hierarchy prefix alignment](#path_hierarchy-prefix-alignment). | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
printf '%s\n' '--- applicable repository conventions ---'
find /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c -maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- target document structure ---'
wc -l ai/reference/full-text-search-index.md
sed -n '1,125p' ai/reference/full-text-search-index.md
printf '%s\n' '--- relevant later sections ---'
sed -n '160,205p' ai/reference/full-text-search-index.mdRepository: pingcap/docs
Length of output: 10245
🌐 Web query:
TiDB 8.5 full-text search path_hierarchy UNDER predicate col UNDER '/src/'
💡 Result:
Based on the current technical documentation and available features for TiDB 8.5, there is no native "path_hierarchy" tokenizer or "UNDER" predicate support within TiDB's full-text search (FTS) engine [1][2][3]. The "path_hierarchy" tokenizer and "UNDER" predicate terminology are characteristic of other search systems, such as Elasticsearch or OpenSearch, which use these features to tokenize hierarchical paths (like file systems or URL paths) into cumulative prefixes to facilitate hierarchical filtering [4][5][6]. TiDB's full-text search is currently designed for keyword-based retrieval using the BM25 ranking algorithm [1][2][7]. Its SQL interface centers on the FTS_MATCH_WORD function (or MATCH... AGAINST syntax) [1][8][7]. When a native full-text index is not available, the optimizer may fall back to LIKE-based pattern matching, but this does not support specialized hierarchical predicates [9][10]. If you are attempting to implement hierarchical path filtering in TiDB, you would typically use standard SQL approaches, such as: 1. Using LIKE with wildcards (e.g., WHERE path LIKE '/src/%') [9]. 2. Storing path components in separate columns for structured querying. 3. Leveraging TiDB's compatibility with standard SQL joins and filtering, which can be combined with full-text search results [1][8]. For authoritative guidance on available full-text search capabilities in TiDB 8.5, refer to the official TiDB AI and Full-Text Search documentation [1][3].
Citations:
- 1: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 2: https://pingcap.github.io/ai/guides/fulltext-search/
- 3: https://docs.pingcap.com/ai/
- 4: https://www.elastic.co/docs/reference/text-analysis/analysis-pathhierarchy-tokenizer
- 5: https://docs.opensearch.org/latest/analyzers/tokenizers/path-hierarchy/
- 6: https://www.elastic.co/guide/en/elasticsearch/reference/8.19/analysis-pathhierarchy-tokenizer.html
- 7: https://www.pingcap.com/article/full-text-search-in-tidb-overview-and-key-features/
- 8: https://github.com/pingcap/docs/blob/master/ai/guides/vector-search-full-text-search-sql.md
- 9: planner: rewrite FTS predicates to LIKE for evaluation of non-TiCI query plan tidb#65626
- 10: pkg/planner, pkg/sessionctx: keep native TiCI FTS plan when LIKE fallback rejects syntax | tidb-test=feature/fts tiflash=feature/fts tikv=feature/fts tidb#68525
🌐 Web query:
site:docs.pingcap.com/ai TiDB path_hierarchy UNDER full-text index
💡 Result:
There is no evidence of a path_hierarchy option, parameter, or tokenizer available for full-text indexes in TiDB. TiDB's full-text search capability supports the following parser types in the WITH PARSER clause: STANDARD: A fast parser that splits text into words based on spaces and punctuation, suitable for English content [1]. MULTILINGUAL: A parser designed to support multiple languages, including English, Chinese, Japanese, and Korean [1]. TiDB also supports vector search with HNSW (Hierarchical Navigable Small World) indexing for vector data types, but this is distinct from the full-text index functionality [2]. The path_hierarchy concept is not a supported feature within TiDB's full-text indexing system [1][3].
Citations:
- 1: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 2: https://docs.pingcap.com/ai/vector-search-index/
- 3: https://docs.pingcap.com/ai/
Remove the unsupported path_hierarchy and UNDER claims.
TiDB 8.5 supports LIKE '/src/%' for this filtering pattern, but does not support the path_hierarchy attribute or the UNDER predicate. Remove these claims and document only supported syntax.
| If you do not specify an index name, TiDB generates one automatically: | ||
|
|
||
| 1. By default, TiDB uses the name of the first indexed column as the index name. | ||
| 2. If that name already exists, TiDB tries the `_2`, `_3`, and subsequent suffixes until the name is unique. | ||
| 3. If the first indexed column name is the reserved word `PRIMARY`, TiDB starts from `primary_2`. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Keep index-name rules consistent across both documents.
The automatic naming guidance conflicts with the explicit-name requirement for NGRAM and column-property mode.
ai/reference/full-text-search-index.md#L186-L190: scope automatic naming to syntax sugar with theMULTILINGUALparser, and state that NGRAM and column-property mode require explicit names.ai/guides/vector-search-full-text-search-sql.md#L89-L89: apply the same scoped wording.
📍 Affects 2 files
ai/reference/full-text-search-index.md#L186-L190(this comment)ai/guides/vector-search-full-text-search-sql.md#L89-L89
|
|
||
| - `FTS_MATCH_WORD()` cannot appear in `GROUP BY` or `HAVING` clauses. | ||
| - Exact phrase matching, where all query tokens must appear consecutively and in the specified order, is not supported yet. | ||
| - Only `INNER JOIN` is supported with full-text search. Outer joins (`LEFT`, `RIGHT`, and `FULL`) are not supported yet. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- scoped conventions ---'
find /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c -maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- target document ---'
sed -n '1,90p' ai/reference/full-text-search-limitations.md
printf '%s\n' '--- cited function guidance ---'
sed -n '175,220p' ai/reference/full-text-search-functions.md
printf '%s\n' '--- cited Python guide ---'
sed -n '1,45p' ai/guides/vector-search-full-text-search-python.mdRepository: pingcap/docs
Length of output: 9330
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- repository review conventions and learnings ---'
for f in /tmp/coderabbit-repo-knowledge/pingcap-docs-9c73f70c/*/*.md; do
printf '\n### %s\n' "$f"
cat "$f"
done
printf '%s\n' '--- source references to outer joins and full-text joins ---'
rg -n -i -C 3 'outer join|left join|right join|full join|inner join|full-text.*join|join.*full-text' ai/reference ai/guides | head -240Repository: pingcap/docs
Length of output: 9841
🌐 Web query:
TiDB 8.5 full-text search INNER JOIN LEFT JOIN outer joins support
💡 Result:
TiDB's full-text search (FTS) feature, available in TiDB 8.5, is designed to be fully compatible with standard SQL [1][2][3]. This means that full-text search operations—typically performed using the FTS_MATCH_WORD function—can be used in conjunction with all standard SQL features, including INNER JOIN, LEFT JOIN, and other outer join types [1][2][3]. Because TiDB's full-text search is integrated directly into the database engine, you can perform joins, filtering (pre-filtering or post-filtering), and grouping on tables that utilize full-text indexes without needing to move data or maintain separate search infrastructure [1][2][4]. Key points regarding this support include: - SQL Compatibility: Full-text search queries behave like standard SQL expressions, allowing them to be combined seamlessly with JOIN clauses to relate searched data with other tables [1][2]. - No Functional Restrictions: There are no specific limitations listed that would prevent INNER or OUTER joins from working with full-text search results [1][2][3]. - Integration: This approach supports hybrid workloads, allowing you to combine structured data queries (via standard joins) with unstructured text retrieval (via FTS) in a single statement [5][4]. As of the current documentation for TiDB 8.5, this feature is primarily available in specific TiDB Cloud environments (such as TiDB Cloud Starter/Serverless in supported regions) [1][2][3]. If you are using a self-managed TiDB deployment, you should verify availability in your specific release version, as rollout and regional support for FTS may vary [1][2][4].
Citations:
- 1: https://docs.pingcap.com/tidb/stable/vector-search-full-text-search-python/
- 2: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 3: https://github.com/pingcap/docs/blob/master/ai/guides/vector-search-full-text-search-python.md
- 4: https://www.pingcap.com/blog/tidb-spring-product-launch-2025-breaking-barriers-ai-saas-scale/
- 5: https://www.pingcap.com/blog/introducing-full-text-search-for-tidb/
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- SQL join example ---'
sed -n '325,385p' ai/guides/vector-search-full-text-search-sql.md
printf '%s\n' '--- all full-text limitation references in tracked docs ---'
rg -n -i -C 2 'full-text search.*(join|sql)|((join|joining).*(full-text|FTS))|outer joins?' --glob '*.md' --glob '*.mdx' . | head -300Repository: pingcap/docs
Length of output: 49817
🌐 Web query:
site:docs.pingcap.com/ai "Only INNER JOIN is supported" full-text search
💡 Result:
There is no general technical rule or common database limitation stating that "only INNER JOIN is supported" for full-text search [1][2]. In fact, systems that support full-text search integrated with SQL typically allow for standard join operations, including LEFT JOIN, RIGHT JOIN, and INNER JOIN, often by using subqueries or standard relational join syntax [1][3]. If you are encountering an error message explicitly stating "Only INNER JOIN is supported" in the context of full-text search, it is likely a specific limitation of a particular framework, abstraction layer, or early-stage feature rather than a inherent constraint of full-text searching itself. For example, when working with systems like TiDB's AI-integrated SQL features, full-text search is designed to be fully compatible with standard SQL features, including complex joins and subqueries [1][2]. If a specific tool (such as an ORM or a simplified API) restricts join types, it is usually a limitation imposed by the developer's implementation of that tool to simplify query planning or execution for specific search operations [1]. To resolve such an issue, verify the following: 1. Check the documentation for the specific library, driver, or ORM you are using, as the limitation is likely defined there rather than in the underlying database engine. 2. If you are using a high-level API (like a Python table.search() method), check if it supports "raw" or "custom" SQL queries, which often bypass such abstractions and allow for more complex joins [1][4]. 3. If necessary, refactor your query to use a subquery or a common table expression (CTE) to achieve the desired join logic if the direct join syntax is restricted by your current API layer [1].
Citations:
- 1: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 2: https://docs.pingcap.com/ai/vector-search-full-text-search-python/
- 3: https://docs.pingcap.com/ai/join-queries/
- 4: https://docs.pingcap.com/ai/raw-queries/
🌐 Web query:
TiDB FTS_MATCH_WORD "INNER JOIN" full-text search
💡 Result:
In TiDB, FTS_MATCH_WORD is a function used to perform keyword-based full-text searches [1][2]. Because it is fully compatible with standard SQL, it can be used seamlessly in conjunction with INNER JOIN operations, just like any other boolean predicate [2][3]. Usage and Integration When you use FTS_MATCH_WORD within an INNER JOIN, it functions as a filtering criterion [2]. TiDB's optimizer handles these queries by integrating the full-text search index access with standard join algorithms (such as Hash Join or Index Join) [4]. Example usage: SELECT t.id, t.title, u.name FROM table_a t INNER JOIN users u ON t.user_id = u.id WHERE FTS_MATCH_WORD('search_query', t.title); Key Technical Considerations 1. SQL Compatibility: FTS_MATCH_WORD behaves like a standard SQL function within the WHERE clause [2][3]. You can join tables and apply full-text filters either directly in the WHERE clause or within subqueries, depending on your query requirements [1]. 2. Optimizer Integration: The TiDB optimizer is designed to recognize FTS_MATCH_WORD predicates [4]. In execution plans, this is typically represented by an IndexRangeScan or a specific FTS-related scan operator, which the optimizer then joins with other tables using standard join operators (e.g., HashJoin, IndexLookUpJoin) [4]. 3. Performance: For optimal performance, ensure that a full-text index has been created on the column being searched [2][3][5]. Without an index, the search may default to less efficient scan methods [5]. You can use the EXPLAIN statement to verify that the query is correctly utilizing the FTS index in the join execution path [5][4]. By leveraging TiDB’s native full-text search, you maintain the ability to use complex SQL operations, including multi-table joins, without needing external search middleware or specialized query languages [3].
Citations:
- 1: https://docs.pingcap.com/ai/vector-search-full-text-search-sql/
- 2: https://docs.pingcap.com/ai/vector-search-full-text-search-python/
- 3: https://www.pingcap.com/article/full-text-search-in-tidb-overview-and-key-features/
- 4: https://github.com/pingcap/tidb/commit/bd92bb9c1dd08d9d702afa024d3462b226904a7a/
- 5: https://docs.pingcap.com/tidbcloudlake/full-text-index/
🌐 Web query:
site:github.com/pingcap/tidb FTS_MATCH_WORD join full-text
💡 Result:
FTS_MATCH_WORD is an internal or low-level function used by TiDB for full-text search operations [1][2][3]. It is specifically associated with the TiDB database's implementation of FULLTEXT index support [4]. When performing full-text searches in TiDB, the query planner may use fts_match_word as a search function within an IndexRangeScan to process queries that leverage full-text indexes [3]. This function is not typically intended for direct use by end-users in standard SQL queries; it is a mechanism by which the engine executes matches against a full-text indexed column [4]. Regarding joins, fts_match_word can be used in conjunction with standard SQL joins (such as hash joins or index-nested-loop joins) [3]. For example, when querying across multiple tables where one table contains a full-text indexed column, the optimizer may incorporate fts_match_word into the execution plan alongside join operators to filter rows efficiently [3]. Key characteristics include: - It acts as an internal search function for full-text indexed columns [3][4]. - It is visible in EXPLAIN plans when a query utilizes a full-text index [2][3]. - It can be combined with standard relational operators and joins [3].
Citations:
- 1: planner: FTS LIKE fallback can return false positives for boolean MATCH AGAINST tidb#68488
- 2: EXPLAIN ANALYZE FORMAT=brief fails with duplicate Projection executor id for FTS TiFlash MPP join tidb#68568
- 3: https://github.com/pingcap/tidb/commit/bd92bb9c1dd08d9d702afa024d3462b226904a7a/
- 4: *: Support building FULLTEXT index tidb#60720
Clarify the join scope in the feature descriptions.
Keep this limitation: direct full-text-search queries support only INNER JOIN. Update the Python and SQL guides to distinguish this from the supported pattern that filters IDs with FTS_MATCH_WORD() in a subquery and then uses an outer join.
1 participant
First-time contributors' checklist
What is changed, added or deleted? (Required)
Provides new functionalities for full-text search.
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
AI agent involvement
Do your changes match any of the following descriptions?
Summary by CodeRabbit