Rebuild EventViewerX as a native event engine with portable providers#250
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1ea67ef297
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b8019a1e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- preserve checkpoint compatibility and raw candidate scan limits - avoid Cartesian XPath query amplification and forward EVTX record IDs - honor read-mode attachment and sub-second session timeout contracts
- stream unlimited chunk batches without unbounded candidate retention - bound timeline scans and local session construction - honor named-event expansion and Pester 5/6 invocation contracts
Use a bounded synchronous task wait so native-operation deadlines remain reliable when CI or production thread pools are saturated.
- avoid preallocating caller-sized top-N buffers - probe beyond named-event scan caps before reporting truncation - expose candidate scan and output truncation metadata to timeline consumers
- make native reads, session opens, and RPC probes deadline-safe - stream parallel results through an async bounded channel - preserve typed catalog and timeline truncation diagnostics - enforce supported Pester and cap caller-sized report allocations
- retain legacy positional named-query cancellation - cancel stalled EVTX reads and avoid worker-start starvation - report catalog and per-name truncation only when proven - centralize null-safe EVTX projection overrides
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a58870b6ea
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3d8894be08
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 23f5423948
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- persist generation-aware checkpoints under a cross-process lock - keep checkpointed reads oldest-first without degrading ordinary oldest queries - bound sparse named-event scans and isolate watcher ownership - refuse reconstructed-script overwrite and link targets
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Intersect caller selectors with native TimeCreated predicates so scan caps count only in-range candidates.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 074d6b5b39
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c2303a92b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93d67f81d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7747c499c7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- enforce atomic no-overwrite provider definition promotion - preserve EVTX access-denied versus missing-file failures - use invariant named-data text for queries and checkpoint identities
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa42ffd356
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1876d13ce9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54bd06d31b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fb2d4c692e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fba6615636
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 similar comment
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 participant
Summary This rebuilds EventViewerX and PSEventViewer as one Windows Event Log product: - EventViewerX owns the dependency-free C# engine over the native Windows Event Log API. - PSEventViewer is a thin compiled PowerShell surface over that engine. - Live, remote, and offline EVTX workflows share the same query, projection, checkpoint, subscription, export, administration, and provider services. - The old monolithic
SearchEventspath and overlapping legacy cmdlets are removed instead of retained as a second implementation. - No EvtxECmd code, Rust parser, or third-party EVTX runtime is embedded. ## Querying, performance, and parityGet-EVXEventand the typed C# API support live channels, remote computers, offline EVTX files, FilterHashtable, XPath, QueryList XML, credentials, cultures, bookmarks, durable checkpoints, subscriptions, catalog and health queries, channel administration, WEC, classic writes, and direct atomic CSV, JSONL, XML, and EVTX exports. Callers chooseMetadata,Message,StructuredData,RawXml, orFull. Large reads stream through bounded buffers, and multi-source work has explicit concurrency and failure policies. Deferred queries snapshot mutable inputs before returning. Large FilterHashtable queries preserve Get-WinEvent semantics: - alternatives for one named field are OR conditions; - distinct named fields remain AND conditions; - exclusions become native QueryListSuppressclauses; - filters are partitioned within Windows' 22-expression XPath limit; - Cartesian expansion is capped before allocation and fails with a useful error when the request is impractically large; - partitioned watcher filters use one QueryList union subscription, avoiding duplicate callbacks and incorrectStopAftercounts; - the reusable C# multi-subscription watcher also deduplicates stable event identities defensively. Remote query setup, subscriptions, catalog reads, catalog name enumeration, provider metadata, channel policy, message rendering, analytic/debug classification, and remote clears are timeout-bounded and cancellation-aware. Native session leases remain alive until detached work finishes, preventing timeout and disposal races. Explicit remote authentication requires a credential instead of silently falling back. Catalog session creation receives caller cancellation, and reusable named-event diagnostics reset at execution start even when filters eliminate every source. Batch consolidation preservesMaxEventsper independent source while exact duplicates and partitions of one logical source share a native cap. Resetting a checkpoint by RecordIdKey starts new generations for both the base key and every existing per-source key derived from it. EVTX report callbacks cannot change a captured event cap after execution starts. Checkpoint updates materialize caller enumerables before taking the cross-process lock. Local-machine classification and checkpoint aliases avoid DNS lookups. Security account-lockout, failed-logon, and user-logon reports keep missing timestamps as null and exclude them from time bounds without losing counts or samples. EVTX event report rows expose missing TimeCreatedUtc and RecordId values as null instead of serializing year-1 or numeric-zero sentinels. Live event rows use nullable timestamp and record-identifier contracts instead of coercing missing values to DateTime.MinValue or record ID 0. Event report filters consistently accept the full Windows event-ID range 0 through 65535. Security and statistics builders return detached snapshots, so later builder reuse cannot mutate an earlier report. The checked-in PowerForge benchmark harness publishes exact-output, common-projection, and native-tool comparisons with event counts, schemas, bytes, hashes, elapsed time, and memory. EvtxECmd is a benchmark target only.powershell Get-EVXEvent -LogName Security -EventId 4624, 4625 ` -TimePeriod Last24Hours -ReadMode Message -MessageCulture en-US Export-EVXEvent -Path C:\Logs\Security.evtx ` -OutputPath C:\Exports\Security.xml -Format Xml -Oldest -Force## Portable custom manifest providers PowerShell users can describe named, typed fields with a hashtable or JSON. C# callers can use typed definitions,FromType<TPayload>(), payload attributes, andResolvedManifestEventWriter. A developer or CI host compiles and optionally signs one.evxprovider. Target machines install it without the Windows SDK, Visual Studio, a compiler, generated source, or a package repository.powershell $provider = @{ ProviderName = 'Contoso.Scanner' ProviderGuid = '7a87f315-4b5e-40a2-b748-b0cdd8adab41' Version = '1.0.0' Events = @{ Name = 'ScanCompleted' Id = 1000 Message = 'Scan of {ComputerName} found {FindingCount} issues.' Fields = [ordered]@{ ComputerName = 'String' FindingCount = 'UInt32' } } } New-EVXProviderPackage -Definition $provider ` -OutputPath .\Contoso.Scanner-1.0.0.evxprovider Install-EVXProviderPackage .\Contoso.Scanner-1.0.0.evxprovider -Confirm:$false Write-EVXEvent -ProviderName Contoso.Scanner -EventName ScanCompleted ` -Data @{ ComputerName = $env:COMPUTERNAME; FindingCount = 7 } -Confirm:$falseThe lifecycle covers install, upgrade, repair, rollback, inventory, and uninstall with serialized provider ownership, signature and trust policy, schema compatibility, bounded extraction, immutable released versions, retained historical resources, and least-privilege managed roots. Active archives must match installation state by exact provider name, GUID, version, and SHA-256. Installation binds privileged activation to the exact byte snapshot that passed preflight validation. Extraction validates package consistency and signatures before writing files, removes only outputs owned by the failed call, and preserves caller-owned destination directories. Custom output types must be Windows SDK-definedwinorxsQNames compatible with the selected input type, and payload lengths and counts are rejected before native allocation when they exceed the event budget. ## Documentation and examples The README is a concise entry point. Task-based guides cover: - all 30 PowerShell cmdlets and practical local, remote, and offline workflows; - synchronous and asynchronous C# reads, batching, cancellation, failures, exports, subscriptions, administration, and writes; - the custom-provider lifecycle for PowerShell hashtables, JSON, and typed C#; - signing, trust, deployment, compatibility, upgrade, repair, rollback, troubleshooting, and performance; - reproducible benchmark contracts and generated result tables. The typed custom-provider example compiles with the solution. ## Intentional v4 cleanup This is an intentional breaking 4.0.0 cleanup. The PowerShell surface is 30 focused cmdlets plus theFind-WinEventalias. C# consumers should use the focused query, batch, catalog, export, subscription, administration, report, and provider services rather thanSearchEvents. Known in-repository consumers have local source migrations prepared and validated. They remain local until EventViewerX and PSEventViewer 4.0.0 are public; no package-publication compatibility shim is included. ## Validation The current candidate passes 653 EventViewerX tests on both .NET 8 and .NET 10, warning-free Debug and Release builds across net472, net8, and net10, PowerShell 7 with 186 passes and 2 environment skips, Windows PowerShell 5.1 with 183 passes and 2 environment skips, the coordinated unsigned NuGet/module build, and packed-module queries in both editions. ## Release sequence 1. Publish a public PSPublishModule release containing the merged resolver fix from EvotecIT/PSPublishModule#623. 2. Publish EventViewerX 4.0.0. 3. Publish PSEventViewer 4.0.0. 4. Repin and publish the prepared TestimoX and IntelligenceX migrations.EvotecIT/EventViewerX.Platformremains archived; this repository is the reusable owner. Do not merge this PR until the release sequence is intentionally started. ## Asynchronous PowerShell lifecycle PSEventViewer's repo-local AsyncPSCmdlet is synchronized with the canonical PSPublishModule lifecycle while preserving EventViewerX logger attachment, resource-owner tracking, and the newer stopping token. Post-await pipeline writes, prompts, and terminating errors are marshalled to the PowerShell pipeline thread through a lossless non-blocking lifecycle transport, retaining streaming backpressure without introducing a package dependency. Cancellation is normalized to PowerShell pipeline-stop semantics, and subscription startup rechecks cancellation after registration so it cannot return an already-stopped subscription. ## Review follow-through The synchronized lifecycle retains the lossless non-blocking lifecycle transport, logger and resource ownership hooks, stopping token, typed nullable replies, complete interaction overloads, late-callback rejection, cancellation-on-dispose, and non-blocking pump failures. Synchronous hook output now writes directly after draining earlier queued worker records. The lossless non-blocking lifecycle transport remains for post-await and background output, preserving backpressure without self-deadlocking before the first incomplete await. Provider package preflight rejects null file maps before member access. Optional probe record-count session creation and metadata reads honor caller cancellation while retaining late native ownership. Named-event candidate caps are iterator-local, so reuse of the reporting object cannot reset or consume another stream's limit. Union-query named-data exclusions are scoped to their originating select predicates, including same-field intersection and native-safe partitioning, so one logical select cannot suppress another. Watcher startup cancellation is rechecked at the commit boundary, disappearing provider directories are isolated during inventory, and native cursors honor cancellation before query setup and throughout diagnostics/bookmark seeking. Initial subscription diagnostics now use bounded, cancellable native ownership. Caller-owned catalog sessions cannot leave detached native work after the API returns, while internally owned sessions retain their bounded leases. Explicitly allowed provider downgrades are treated as rollbacks instead of forward schema migrations. The exact candidate passes 653/653 EventViewerX tests on both .NET 8 and .NET 10, warning-free Debug and Release builds across net472/net8/net10, PowerShell 7 with 186 passes and 2 environment skips, and Windows PowerShell 5.1 with 183 passes and 2 environment skips. Provider package extraction now creates payload files exclusively and cleans up only files owned by that invocation, so a concurrent writer cannot be overwritten or deleted. Temporary build cleanup is best-effort and cannot replace a promoted package result or the primary tool failure. Caller-budget RPC/session timeouts no longer poison the shared remote-host negative cache; only definitive reachability failures do. RPC endpoint probes now preserve Connected, Failed, and TimedOut as distinct outcomes. Managed and native remote readers cache only definitive reachability failures, so a small caller timeout cannot poison later normal-budget requests. Provider activation cleanup is best-effort and cannot replace the primary activation result or failure. Live and EVTX projections preserve absent Level metadata as null, statistics report those records separately instead of classifying them as level zero, and optional Task, Opcode, and Keywords values retain null-versus-zero semantics. Trailing-dot local machine aliases are normalized without changing the single-dot local sentinel. ## Final async lifecycle semantics Synchronous PowerShell calls restore the host synchronization context before queued work is drained or the base API is invoked. Queue admission and hook completion are serialized, preserving records admitted before completion while rejecting stale producers afterward. Reentrant drains preserve FIFO order, including context-free captured callbacks made while a lazy queued item is actively being pumped. Hook completion wakes the pump immediately. Direct downstream stops cancel work already started with the cmdlet token, cancellation-source disposal waits until cancellation callbacks return, and derived synchronousEndProcessingoverrides can continue interacting with the pipeline after calling the base implementation. Pipeline-thread ownership uses volatile publication, captured callbacks drop cleanly after stop, and progress snapshots retain optional runtimeTotalvalues. Queued host interactions now atomically arbitrate cancellation against the pipeline claim: canceled unclaimed requests never enter the host, while a claimed prompt keeps its reply observed. Completed and failing synchronous hooks drain all causally reentrant records, and a synchronous pipeline stop cancels the shared token before rethrow. PSPublishModule remains the canonical source owner. Consumer repositories retain namespace-adjusted local source copies and their repository-specific interfaces, helpers, formatting, and split-file conventions without adding a runtime package dependency. ## Final validation PSEventViewer Debug and Release builds passed at the exact head with its watcher-owner and logger extensions retained; the exact suite passes 653/653 on net8-windows and net10-windows. Streaming export cleanup and reconstructed-script partial-file cleanup are best-effort while preserving their primary failures. Native Win32 query errors retain typed invalid-query, missing-log, access, timeout, and host-unavailable categories in live event and statistics reports. Fatal synchronous and asynchronous batch primer failures cancel sibling sources immediately while retaining the first failure and disposing completed cursors. Provider-definition writes enforce no-overwrite at atomic promotion, EVTX source probing preserves access-denied versus missing-file failures, and culture-sensitive named-data values use the same invariant representation in native queries and checkpoint identities. Final review hardening preserves existence-only NamedData suppressions, pipeline-safe explicit checkpoints, provider uninstall failures, nonblocking subscription cancellation, and native-compatible offline file identities. Archive access failures, provider-state promotion failures, completed channel clears, and callback exceptions now retain their authoritative outcomes instead of being misclassified or replaced.FilterHashtable data literals now preserve ordinal case, whitespace, and empty strings. Checkpoint and package cleanup preserve primary failures, package opens retain native access errors, exact-package repair transactionally replaces corrupt activation directories, async stream cancellation no longer waits on stalled projection work, first-time provider-root claiming is serialized across processes, and unsupported watcher timeouts are rejected before startup.
Latest review hardening
Initial asynchronous batch priming now detaches only for caller cancellation and retains late cursor ownership; fatal source failures still await sibling cleanup and preserve the first failure. Provider-package promotion honors overwrite when a competing destination appears between the existence check and atomic move.
Get-EVXEvent -MaxEventsis invocation-wide for pipeline-bound sources and avoids starting later source queries after the cap. Watcher token callbacks detach lifecycle state immediately and perform guarded subscription cleanup asynchronously, so cancellation does not wait for active event or stopped callbacks.The exact candidate
0f10b99passes 655/655 EventViewerX tests under simultaneous net8-windows and net10-windows coverage, including 10/10 repeated cancellation regressions on each target, plus warning-free Debug and Release builds across net472/net8/net10. The production lifecycle contract is unchanged; the test now waits for its deliberately stalled iterator to finish after release so compiler-generatedDisposeAsynccannot race cleanup under simultaneous coverage. PowerShell and coordinated package evidence remains the exact parent proof: PowerShell 7 with 185 passes and 3 environment skips, Windows PowerShell 5.1 with 182 passes and 3 environment skips, payload-verified EventViewerX/PSEventViewer 4.0.0 artifacts, local build 4.0.0.69, and packed Core/Desktop invocation-wideMaxEventsproofs.