Repository navigation
Refactor GreptimeDB Python SDK to separate gRPC and PostgreSQL backends - #2
Merged
Merged
Conversation
paomian
commented
Oct 29, 2026
Collaborator
- Updated documentation to clarify backend usage and import paths.
- Removed automatic backend selection; users must explicitly import the desired backend.
- Simplified error handling by consolidating GreptimeError into a single implementation.
- Enhanced type handling in the gRPC and PostgreSQL modules to ensure compatibility with shared schema.
- Removed legacy native module and associated imports.
- Added tests to verify that both backends correctly utilize the shared schema and handle errors appropriately.
- Ensured that the public schema is consistent across both backends.
- Updated documentation to clarify backend usage and import paths. - Removed automatic backend selection; users must explicitly import the desired backend. - Simplified error handling by consolidating GreptimeError into a single implementation. - Enhanced type handling in the gRPC and PostgreSQL modules to ensure compatibility with shared schema. - Removed legacy native module and associated imports. - Added tests to verify that both backends correctly utilize the shared schema and handle errors appropriately. - Ensured that the public schema is consistent across both backends.
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the refactor separating explicit backend selection from shared schema and error types.
No open issues 🎉
What changed in this PR
This PR separates backend selection from the SDK’s shared schema. Callers explicitly import the gRPC or PostgreSQL client, while both backends use the same public column types and GreptimeError.
Changes:
- Remove automatic backend selection and the legacy native import shim.
- Adapt PostgreSQL and Rust bindings to consume shared Python schema and error types.
- Update tests, examples, and documentation for explicit backend imports.
| File | Description |
|---|---|
tests/test_types.py |
Require public Column objects in PostgreSQL validation. |
tests/test_shared_schema.py |
Add shared schema and error checks for both backends. |
tests/test_postgres.py |
Update validation calls and strengthen import isolation checks. |
tests/test_postgres_schema_compat.py |
Remove obsolete native-schema compatibility tests. |
tests/test_integration.py |
Update imports and check bulk writer column types. |
tests/test_convert.py |
Use explicit gRPC and native plugin imports. |
tests/test_backends.py |
Verify root exports and backend import isolation. |
README.zh-CN.md |
Document explicit imports and refresh reference links. |
README.md |
Mirror backend import guidance and reference updates. |
python/greptimedb_ingester/postgres.py |
Use shared columns with explicit SQL type mappings. |
python/greptimedb_ingester/grpc.py |
Limit exports to backend-specific APIs. |
python/greptimedb_ingester/_types.py |
Clarify shared schema ownership and simplify error creation. |
python/greptimedb_ingester/_native.py |
Remove the legacy native import shim. |
python/greptimedb_ingester/_errors.py |
Define one backend-independent GreptimeError. |
python/greptimedb_ingester/__init__.py |
Remove automatic client selection and backend exports. |
packages/grpc/src/types.rs |
Convert between shared Python columns and internal Rust types. |
packages/grpc/src/lib.rs |
Stop registering native schema and error classes. |
packages/grpc/src/error.rs |
Construct the shared Python error from Rust. |
packages/grpc/src/bulk.rs |
Return public Python columns from the writer getter. |
examples/insert_example.py |
Import Client explicitly from gRPC. |
examples/bulk_example.py |
Import Client and WriteOptions from gRPC. |
examples/bench_write.py |
Separate shared schema and gRPC imports. |
docs/usage.md |
Update examples and backend selection guidance. |
AGENTS.md |
Document explicit imports and prohibit legacy shims. |
Review effort: Balanced
- Update `.gitignore` to ignore all files in the `docs` directory. - Modify README files to reference the new PG getting started guide. - Refactor decimal handling in gRPC to improve precision and scale validation. - Change `exponent` type in `DecimalParts` from `i32` to `i64` for better compatibility. - Enhance error handling in `raise` function to ensure retriable status is correctly set. - Introduce `checked_duration` function to validate duration inputs. - Add comprehensive tests for input validation and error handling in gRPC and PostgreSQL clients. - Ensure schema validation is robust against forged inputs and maintains integrity across operations. - Update PostgreSQL client to handle connection errors gracefully and invalidate cached schemas when necessary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.