Skip to content

Query builder template mode#620

Open
arturminchukov wants to merge 21 commits intomainfrom
query-builder-template-mode
Open

Query builder template mode#620
arturminchukov wants to merge 21 commits intomainfrom
query-builder-template-mode

Conversation

@arturminchukov
Copy link
Copy Markdown
Member

@arturminchukov arturminchukov commented Apr 15, 2026

Related issue: #578

Describe Your Changes

Introduces a segment-based template builder as an alternative to the code editor.
Users construct queries by adding typed pipes (filter, stream, aggregate, modify, sort, limit)
through an interactive UI with field name/value dropdowns, keyboard navigation,
and real-time query serialization to LogsQL.

Key components:

  • Segment model: Pipe[] → serialization → LogsQL string
  • Template registry with configurable pipe types and optional extensions (by, if, as)
  • Unified popup manager ensuring mutual exclusivity of dropdowns
  • Insertable separators for adding pipes at any position
  • Field name quoting for names with special characters
  • Each stream pipe serialized as individual {…} block

Demo link

image

Checklist

The following checks are mandatory:


Summary by cubic

Adds a new template-based query builder that lets users build LogsQL with guided pipes and placeholders, as an alternative to the code editor. This improves discoverability, adds keyboard-driven editing, and keeps the query in sync with the editor state. Addresses #578.

  • New Features

    • Segment model for pipes with real-time serialization to LogsQL; stream pipes render as {...} blocks and field names auto-quote when needed.
    • Template registry for stream, filter, modify, aggregate, sort, limit, with optional extensions like by, if, as, partition by, limit, offset; searchable add-pipe menu with keyboard navigation.
    • Insertable separators to add pipes anywhere; unified popup manager and floating, portal-based dropdowns for conflict-free menus.
    • Placeholder chips with dropdowns, multi-select, grouped option sections with improved styling, and tab/arrow-key navigation across placeholders.
    • Per-pipe field/stream loaders with LRU caching, template variable support, time range–aware keys, and server-side filter for field endpoints.
    • Code and builder modes stay in sync via a header toggle; builder state is stored on the query as templateBuilder.
    • Minor DX: stable lint:fix script, @grafana/ui bumped to 12.4.2, added @types/react-dom.
  • Bug Fixes

    • Prevent duplicate sort/order by pipes from being appended when any sort exists.
    • Avoid losing queries when switching between Code and Builder; skip confirmation when expression matches the “all” filter; re-sync expr from templateBuilder when needed.
    • Stats warning now reflects the builder model, not stale expr.
    • Stream/field option loading respects template variables and server-side filtering; fixed stale caches, debounce cross-cancellation, and cleared options when dependencies are missing.

Written for commit f7831a8. Summary will update on new commits.

  Introduces a segment-based template builder as an alternative to the code editor.
  Users construct queries by adding typed pipes (filter, stream, aggregate, modify, sort, limit)
  through an interactive UI with field name/value dropdowns, keyboard navigation,
  and real-time query serialization to LogsQL.

  Key components:
  - Segment model: Pipe[] → serialization → LogsQL string
  - Template registry with configurable pipe types and optional extensions (by, if, as)
  - Unified popup manager ensuring mutual exclusivity of dropdowns
  - Insertable separators for adding pipes at any position
  - Field name quoting for names with special characters
  - Each stream pipe serialized as individual {…} block
…rder by

  Previously, `addSortPipeToQuery` only checked for `sort by (_time)` and would
  append a second sort pipe if the query already sorted by a different field.
  Now it detects any existing `sort by (...)` or `order by (...)` pipe and
  skips adding a duplicate.
  - Skip confirmation modal when switching Code -> Builder if expr matches
    the serialized templateBuilder state (user didn't manually edit code)
  - Re-sync expr from templateBuilder when switching to Code mode to
    restore query that was cleared by the confirmation modal
  - Also clear templateBuilder on modal confirm to keep state consistent
  - Remove unused `builder` field from Query type and query serialization
  In builder mode, derive the expression from templateBuilder directly instead of relying on query.expr, which can be stale or empty after page reload with a saved panel. This ensures the warning appears and disappears correctly as the user adds or removes stats pipes.
…cks, extract STREAM_TEMPLATE_TYPE constant

  - Wrap handleNavigationKeyDown in useCallback to stabilise its reference in deps arrays
  - Replace mutable flatCounter in PipeTypeSearchMenu render with useMemo + flatIndexMap (O(1) lookup)
  - Replace IIFE with mutable flatIndex in PlaceholderChip grouped rendering with filteredOptions.indexOf
  - Extract STREAM_TEMPLATE_TYPE constant in segmentHelpers.ts and replace all 'stream' magic strings
  - Replace '__open__' sentinel in PipeTypeSearchMenu with explicit onOpenMenu prop
  - Split single useEffect handling two pending refs into two independent effects with correct deps
  - Destructure openAddMenu from popup to satisfy react-hooks/exhaustive-deps
  - Fix pre-existing set-state-in-effect lint error in PipeTypeSearchMenu by simplifying useEffect structure
  - Delete AddPipeButton.tsx (unused dead code)
…erer and consolidate multi-value filtering in useOptionLoading

  - Extract new PipeFieldLoadersProvider component that owns per-pipe useFieldFetch/useFetchStreamFilters hooks and exposes loaders via FieldLoadersContext
  - Remove datasource, timeRange, queryContext, extraStreamFilters props from PipeRenderer — it now focuses on segment rendering
  - Wrap each pipe in PipeFieldLoadersProvider inside TemplateQueryEditor.renderPipe
  - Move multi-value filtering (previously duplicated as filteredGroups/filteredOptions in PlaceholderChip) into useOptionLoading via a new multiValues option
  - Return already-filtered options/optionGroups from the hook so useDropdownNavigation gets an exact itemCount
…onent and extract usePipeTypeSearch hook

  - Replace buildPipeElements function with PipeList React.FC; move useStyles2 inside and drop useMemo wrapper in TemplateQueryEditor
  - Extract search/filter logic (keyword groups, label/description match, navItems, flatIndexMap) from PipeTypeSearchMenu into new usePipeTypeSearch hook
  - Memoize displayGroups properly (was recomputed every render via IIFE)
  - PipeTypeSearchMenu is now a pure UI component: state, focus, keyboard wiring, and render
@arturminchukov arturminchukov self-assigned this Apr 15, 2026
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9 issues found across 81 files

Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed. cubic prioritises the most important files to review.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/utils/LRUCache.ts">

<violation number="1" location="src/utils/LRUCache.ts:16">
P2: `get` uses value comparison instead of key existence, which breaks LRU promotion for cached `undefined` values.</violation>
</file>

<file name="src/components/QueryEditor/TemplateBuilder/hooks/useTabNavigation.ts">

<violation number="1" location="src/components/QueryEditor/TemplateBuilder/hooks/useTabNavigation.ts:21">
P2: Handle `activeId` values that are no longer present in `allIds` before advancing; otherwise Tab can jump to the first placeholder unexpectedly.</violation>
</file>

<file name="src/components/QueryEditor/TemplateBuilder/PlaceholderChip/useOptionLoading.ts">

<violation number="1" location="src/components/QueryEditor/TemplateBuilder/PlaceholderChip/useOptionLoading.ts:109">
P2: Clear options when `dependencyValue` is missing to avoid showing stale field values after the dependent field is removed.</violation>

<violation number="2" location="src/components/QueryEditor/TemplateBuilder/PlaceholderChip/useOptionLoading.ts:121">
P2: Reset stream field value options when `dependencyValue` is empty; otherwise stale options remain visible.</violation>
</file>

<file name="src/components/QueryEditor/shared/useFetchStreamFilters.ts">

<violation number="1" location="src/components/QueryEditor/shared/useFetchStreamFilters.ts:43">
P1: Interpolated query/filter values can become stale because `useMemo` dependencies do not track template variable value changes.</violation>

<violation number="2" location="src/components/QueryEditor/shared/useFetchStreamFilters.ts:52">
P2: Field-name cache key does not include time range, so options can become stale after time-range changes.</violation>
</file>

<file name="src/components/QueryEditor/shared/useFieldFetch.ts">

<violation number="1" location="src/components/QueryEditor/shared/useFieldFetch.ts:54">
P2: Field-name cache key omits `timeRange`, so changing time range can serve stale field suggestions.</violation>

<violation number="2" location="src/components/QueryEditor/shared/useFieldFetch.ts:138">
P1: A single shared debounced function is reused across different loaders, so concurrent typing in different fields can cancel each other’s requests and leave pending Promises unresolved.</violation>
</file>

<file name="src/components/QueryEditor/TemplateBuilder/TemplateQueryEditor.tsx">

<violation number="1" location="src/components/QueryEditor/TemplateBuilder/TemplateQueryEditor.tsx:140">
P2: Scope the global Shift+Enter shortcut to key events originating from this editor, otherwise it can trigger unintended query runs from unrelated UI elements.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.

Comment thread src/components/QueryEditor/shared/useFetchStreamFilters.ts Outdated
Comment thread src/components/QueryEditor/shared/useFieldFetch.ts Outdated
Comment thread src/utils/LRUCache.ts Outdated
Comment thread src/components/QueryEditor/shared/useFetchStreamFilters.ts Outdated
Comment thread src/components/QueryEditor/shared/useFieldFetch.ts Outdated
Comment thread src/components/QueryEditor/TemplateBuilder/TemplateQueryEditor.tsx
…nents

  - Create per-module styles.ts in PipeRenderer/, PipeList/, PlaceholderChip/, PipeTypeSearchMenu/ with only the styles each module uses
  - Root styles.ts reduced to editor and clearButton (TemplateQueryEditor only)
  - Remove dead styles: multiValue, multiChip, addButton
…evel listeners

  - Extract popup into PipeTypeSearchPopup with its own mount/unmount lifecycle
  - Move keyboard navigation from input onKeyDown to document.addEventListener
  - Move click-outside detection into the popup component
  - Switch useDropdownNavigation.handleNavigationKeyDown to native KeyboardEvent
  - PipeTypeSearchMenu becomes a thin orchestrator: button + conditional popup
…RU promotion

  - LRUCache.get: use cache.has() instead of value !== undefined so undefined-valued entries are correctly promoted on access
  - useTabNavigation: deactivate when activeId is no longer in allIds instead of jumping to the first placeholder unexpectedly
  - useOptionLoading: clear stale options when dependencyValue is missing for fieldValues and streamFieldValues sources
  - useFetchStreamFilters: drop useMemo for interpolateString calls so template variable value changes are picked up on re-render; include timeRange in stream field-names cache key
  - useFieldFetch: include timeRange in field-names cache key; split shared debouncedFilter into separate debouncedFilterNames / debouncedFilterValues instances with pendingResolve tracking to prevent cross-cancellation and dangling Promises
…stream_field_names`, `stream_field_values`
@arturminchukov
Copy link
Copy Markdown
Member Author

Check failed cause there are vulnerabilities that will be fixed in the other pr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant