Skip to content

feat(gcp_cloud_storage sink): add Parquet encoder#25216

Open
sonnens wants to merge 8 commits intovectordotdev:masterfrom
sonnens:gcp_parquet_codec
Open

feat(gcp_cloud_storage sink): add Parquet encoder#25216
sonnens wants to merge 8 commits intovectordotdev:masterfrom
sonnens:gcp_parquet_codec

Conversation

@sonnens
Copy link
Copy Markdown
Contributor

@sonnens sonnens commented Apr 16, 2026

Summary

Adds parquet batch encoder support to the google_cloud_storage sink similar to the aws_s3 sink

Vector configuration

sinks:
  gcs_parquet:
    type: gcp_cloud_storage
    inputs:
      - enforce_schema
    bucket: vector-parquet-test-bucket
    key_prefix: "date=%F/"
    encoding:
      codec: text
    batch_encoding:
      codec: parquet
      schema_file: test.schema
      compression: snappy
      schema_mode: strict

How did you test this PR?

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

#25156

@sonnens sonnens requested review from a team as code owners April 16, 2026 19:44
@github-actions github-actions Bot added work in progress domain: sinks Anything related to the Vector's sinks domain: external docs Anything related to Vector's external, public documentation labels Apr 16, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 58d86c8881

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread website/cue/reference/components/sinks/gcp_cloud_storage.cue Outdated
@github-actions github-actions Bot removed the domain: external docs Anything related to Vector's external, public documentation label Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks work in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant