Skip to content

feat(splunk_hec source): Support custom extractors#25180

Open
20agbekodo wants to merge 9 commits intovectordotdev:masterfrom
20agbekodo:josue.agbekodo/OPA-5005-opw-vector-poc-custom-extractors-for-splunk-hec-source
Open

feat(splunk_hec source): Support custom extractors#25180
20agbekodo wants to merge 9 commits intovectordotdev:masterfrom
20agbekodo:josue.agbekodo/OPA-5005-opw-vector-poc-custom-extractors-for-splunk-hec-source

Conversation

@20agbekodo
Copy link
Copy Markdown
Contributor

@20agbekodo 20agbekodo commented Apr 13, 2026

This PR is the complete version of this one which implements custom extractors for the SplunkHEC source.

Explanation

From the first PR

  • a trait Extractor was created and added as a generic parameter to SplunkSource
  • the array of DefaultExtractors used in the EventIterator were grouped under a single DefaultExtractor type implementing Extractor and provided as the default type for the splunk source generic (the original DefaultExtractor type was renamed MetaExtractor)
  • this Extractor trait provides a constructor which is called when instantiating the EventIterator, passing a RequestMeta object containing information about the request being processed (splunk token, remote address)
  • the splunk source initialization logic was put out of the SplunkConfig into a listen method so that the SplunkSource can be easily initialized in a different context
  • some methods declared in the SplunkSource impl which did not depend on the SplunkSource struct (options, required_channel, lenient_json_content_type_check) were moved to plain functions to avoid the need for calling them with SplunkSource::<DefaultExtractor>::options()

Added on top of that

Add type Config: Send + Sync + Clone to the Extractor trait so that
wrapper implementations (e.g. OPW's splunk_hec_opw) can receive
per-source config data (such as a token-to-attribute mapping) at
extractor construction time without relying on global state.

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

@github-actions github-actions bot added the domain: sources Anything related to the Vector's sources label Apr 13, 2026
@20agbekodo 20agbekodo changed the title Josue.agbekodo/opa 5005 opw vector poc custom extractors for splunk hec source feat(splunk_hec source): Support custom extractors Apr 16, 2026
Add `type Config: Send + Sync + Clone` to the `Extractor` trait so that
wrapper implementations (e.g. OPW's splunk_hec_opw) can receive
per-source config data (such as a token-to-attribute mapping) at
extractor construction time without relying on global state.

- `DefaultExtractor` uses `type Config = ()` — no behaviour change
- `SplunkSource<E>` stores `extractor_config: E::Config` and clones it
  into the warp closure for each request
- Add `SplunkSource::new_from_parts` public constructor for use by
  wrapper source implementations that don't have a `SplunkConfig`
- Re-export `HecAcknowledgementsConfig` from the `splunk_hec` module
@20agbekodo 20agbekodo force-pushed the josue.agbekodo/OPA-5005-opw-vector-poc-custom-extractors-for-splunk-hec-source branch from 62c1a5c to af94a00 Compare April 16, 2026 11:49
@20agbekodo 20agbekodo marked this pull request as ready for review April 16, 2026 11:53
@20agbekodo 20agbekodo requested a review from a team as a code owner April 16, 2026 11:53
@pront
Copy link
Copy Markdown
Member

pront commented Apr 17, 2026

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants