Skip to content

feat(pr-review): add summary-prompt input for context-aware review summaries#119

Open
daniel-richter wants to merge 2 commits into
mainfrom
feature/add-summary-field
Open

feat(pr-review): add summary-prompt input for context-aware review summaries#119
daniel-richter wants to merge 2 commits into
mainfrom
feature/add-summary-field

Conversation

@daniel-richter
Copy link
Copy Markdown
Contributor

@daniel-richter daniel-richter commented May 25, 2026

Context: The action previously generated the review body text using disclaimer-prompt in isolation, with no access to the actual review findings. This meant the summary could not include context-aware content such as finding counts or severity breakdowns, since the findings only existed after the review was generated.

Changes:

  • Added a new summary-prompt input parameter (default: "Append a brief summary of the key review findings. No lists, no headings.") to allow customizing the summary text
  • Replaced the standalone disclaimer-prompt AI call with a combined system prompt that merges disclaimer-prompt and summary-prompt, passing the actual review findings as user content so the model can produce a context-aware summary
  • Added missing comments, review, and summary output parameters to action.yml and updated their descriptions
  • Updated README documentation to document the new summary-prompt parameter, the new outputs, and added the new "Customize the review summary" section

@github-actions
Copy link
Copy Markdown

This update adds a new summary-prompt input to generate and include a concise AI-driven summary in the PR review body, replacing the previous disclaimer-only step. It updates the action’s outputs (adding summary), refactors the workflow to compose reviews with this summary, and enhances the documentation and examples to reflect these changes.

Walkthrough

  • New Feature: Introduce summary-prompt to configure and generate a brief AI summary in PR reviews
  • Chore: Add summary to action outputs and parse it in configuration
  • Documentation: Update README with new table entries, output descriptions, and a usage example
  • Refactor: Replace standalone disclaimer call with combined summary generation and adjust review assembly
  • Style: Clarify output descriptions for comments, review, and reviewId

Model: o4-mini-2025-04-16 | Prompt Tokens: 1707 | Completion Tokens: 829

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for sharing your code. I’ve offered ideas supported by AI to improve readability, control flow and maintainability while respecting your judgment. Feel free to apply what fits your style as you guide the decisions. The feedback covers extracting summary generation into its own helper, adding a guard to skip unnecessary API calls, improving display text assembly through a utility function, updating action.yml inputs with consistent structure and examples, and clarifying summary-prompt defaults and formatting in the README.

Model: o4-mini-2025-04-16 | Prompt Tokens: 3468 | Completion Tokens: 4933

Comment thread pr-review/src/main.ts
Comment on lines +168 to +177
core.startGroup(`Ask LLM for a summary to add to the review description`)
const summary = await aiCoreClient.chatCompletion([
{
role: "system",
content: [config.disclaimerPrompt, config.summaryPrompt].join("\n"),
},
{ role: "user", content: `Review findings:\n${JSON.stringify(comments.map(({ path, body }) => ({ path, body })))}` },
])
core.info(inspect(summary, { depth: undefined, colors: true }))
core.setOutput("summary", summary)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider extracting the summary generation logic into its own helper function to improve readability and separation of concerns. For example:

async function generateSummary(
  comments: { path: string; body: string }[],
  prompts: { disclaimer: string; summary: string }
): Promise<string> {
  const messages = [
    { role: "system", content: [prompts.disclaimer, prompts.summary].join("\n") },
    {
      role: "user",
      content: `Review findings:\n${JSON.stringify(
        comments.map(({ path, body }) => ({ path, body }))
      )}`,
    },
  ];
  return aiCoreClient.chatCompletion(messages);
}
// Usage in main:
core.startGroup("Ask LLM for a summary to add to the review description");
const summary = await generateSummary(comments, {
  disclaimer: config.disclaimerPrompt,
  summary: config.summaryPrompt,
});
core.info(inspect(summary, { depth: undefined, colors: true }));
core.setOutput("summary", summary);

Comment thread pr-review/src/main.ts
Comment on lines +168 to +177
core.startGroup(`Ask LLM for a summary to add to the review description`)
const summary = await aiCoreClient.chatCompletion([
{
role: "system",
content: [config.disclaimerPrompt, config.summaryPrompt].join("\n"),
},
{ role: "user", content: `Review findings:\n${JSON.stringify(comments.map(({ path, body }) => ({ path, body })))}` },
])
core.info(inspect(summary, { depth: undefined, colors: true }))
core.setOutput("summary", summary)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a guard to skip the summary API call when there are no review comments. This prevents unnecessary LLM invocations and makes the control flow clearer:

let summary = "";
if (comments.length > 0) {
  core.startGroup("Ask LLM for a summary to add to the review description");
  summary = await generateSummary(comments, {
    disclaimer: config.disclaimerPrompt,
    summary: config.summaryPrompt,
  });
  core.info(inspect(summary, { depth: undefined, colors: true }));
  core.setOutput("summary", summary);
} else {
  core.info("No review comments found – skipping summary generation.");
}

Comment thread pr-review/src/main.ts
Comment on lines -183 to 191
const modelMetadataFooter = config.showModelMetadataFooter ? `<sub>${metadata.filter(Boolean).join(" | ")}</sub>` : ""
const displayText = [markerStart, baseheadMaker, header, disclaimer, footer, modelMetadataFooter, markerEnd].filter(Boolean).join("\n")
const modelMetadataFooter = config.showModelMetadataFooter ? `\n<sub>${metadata.filter(Boolean).join(" | ")}</sub>` : ""
const displayText = [markerStart, baseheadMaker, header, summary, footer, modelMetadataFooter, markerEnd].filter(Boolean).join("\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The construction of displayText can be made more readable by using a template literal or a small utility function instead of filtering an array. For example:

function buildReviewBody(sections: Record<string, string | undefined>): string {
  return Object.values(sections)
    .filter(Boolean)
    .join("\n");
}

const displayText = buildReviewBody({
  markerStart,
  baseheadMaker,
  header,
  summary,
  footer,
  modelMetadataFooter,
  markerEnd,
});

This groups related logic and makes it easier to maintain.

Comment thread pr-review/action.yml
Comment on lines 50 to 58
default: |
Create a kind alternative for the following term: Here's a helpful review of your code with support from AI. Some insights are predictions, not guaranteed facts, so feel free to use what works best for you. Your decisions lead the way—AI is just here to assist.
Do not use more than 80 words. Do not use quotation marks.
summary-prompt:
description: "The prompt that is used to generate the summary shown in the review body."
required: false
default: "Append a brief summary of the key review findings. No lists, no headings."

display-mode:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure that the new summary-prompt input is nested under the top-level inputs: key and follows the same indentation/style as the other inputs. Also, consider adding an example under action.yml comments section:

inputs:
  prompt:
    description: "The base prompt..."
    default: |-
      ...
  summary-prompt:
    description: "The prompt that is used to generate the summary shown in the review body."
    required: false
    default: "Append a brief summary of the key review findings. No lists, no headings."
  display-mode:
    description: "Defines where the review will be posted..."
    default: "${{ github.api_url }}"

Comment thread pr-review/README.md
Comment on lines 98 to 104
| `prompt` | The base prompt that is used to generate the review. <br /> Default: See [action.yml](action.yml#L36-L43) |
| `prompt-addition` | The addition to the base prompt that is used to generate the review. |
| `disclaimer-prompt` | The prompt that is used to generate the disclaimer. <br /> Default: See [action.yml](action.yml#L51-L53) |
| `summary-prompt` | The prompt that is used to generate the summary shown in the review body. <br /> Default: `Append a brief summary of the key review findings. No lists, no headings.` |
| `header-text` | Text to be inserted before the review. |
| `footer-text` | Text to be inserted after the review. |
| `previous-results` | Define what to do with previous results. Possible values are `keep` or `hide`. <br /> Default: `keep` |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the README parameter table, clarify the default value for summary-prompt and ensure consistent markdown formatting. For example:

| `summary-prompt` | The prompt that is used to generate the summary shown in the review body. Default: `Append a brief summary of the key review findings. No lists, no headings.` |

Also, consider adding a note about how the summary output can be consumed programmatically.

@github-actions
Copy link
Copy Markdown

This update aligns CI workflows with the latest Node setup action, brings core libraries and tooling dependencies up to date in both the PR review and summary packages, and streamlines configuration parsing by removing unnecessary type assertions.

Walkthrough

  • Chore: Upgraded actions/setup-node in GitHub Actions workflows from v6.3.0 to v6.4.0 for faster, more secure builds
  • Chore: Bumped key dependencies (@actions/core, axios, zod, tsx, @types/node, typescript-eslint, yaml) to their latest versions
  • Refactor: Simplified config.ts parsing logic by dropping redundant as z.infer<T> casts

Model: o4-mini-2025-04-16 | Prompt Tokens: 1893 | Completion Tokens: 487 | Diff Range: 699875e...68c8f91

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant