feat: Add PR review reminder workflow#1012
Open
Flashl3opard wants to merge 8 commits intoalphaonelabs:mainfrom
Open
feat: Add PR review reminder workflow#1012Flashl3opard wants to merge 8 commits intoalphaonelabs:mainfrom
Flashl3opard wants to merge 8 commits intoalphaonelabs:mainfrom
Conversation
Contributor
WalkthroughAdds a new "PR Review Reminder" GitHub Actions workflow and two documentation files. The workflow scans open PRs, aggregates human reviewers from reviews and comments, and posts maintainer notifications / review requests for PRs with only bot reviews older than a configurable threshold. Changes
Sequence Diagram(s)sequenceDiagram
participant GHEvents as GitHub Events
participant Workflow as Actions Workflow
participant API as GitHub API
participant Maintainers as Maintainers
GHEvents->>Workflow: Trigger (review / cron / manual)
Workflow->>Workflow: Load config (MAINTAINERS, threshold, exempt bots, marker)
Workflow->>API: List open PRs (paginated)
API-->>Workflow: Return PR list
loop per PR
Workflow->>API: Fetch PR reviews
API-->>Workflow: Reviews
Workflow->>API: Fetch review comments & issue comments
API-->>Workflow: Comments
Workflow->>Workflow: Aggregate non-bot, non-author human reviewers
alt No human reviewers AND PR age > threshold AND no existing marker
Workflow->>API: Post comment mentioning maintainers + marker
API-->>Workflow: Comment created
Workflow->>API: Request reviews from maintainers (exclude author)
API-->>Workflow: Review requests created / warnings if failed
end
end
Workflow->>Workflow: Emit summary (checked / notified / errors)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/PR_REVIEW_REMINDER_SETUP.md:
- Around line 31-43: The documentation references incorrect line numbers for the
workflow config; update the guidance in .github/PR_REVIEW_REMINDER_SETUP.md so
it points to the correct locations in .github/workflows/pr-review-reminder.yml:
change the MAINTAINERS reference from line 26 to line 27 and change the
DEFAULT_HOURS_THRESHOLD reference from line 29 to line 30, and adjust the
proposed fix block text accordingly so the MAINTAINERS and
DEFAULT_HOURS_THRESHOLD entries are pointed to the correct lines.
In @.github/workflows/pr-review-reminder.yml:
- Around line 71-76: The code makes duplicate calls to
github.rest.issues.listComments for the same PR (first populating issueComments
and later re-calling to build commenters); remove the second call and reuse the
first result (issueComments) everywhere you currently re-fetch comments
(including the logic that collects commenters at lines around the second call),
ensuring any downstream code references issueComments.data (or the same shape)
instead of making a new API request; adjust variable names if needed so
functions that currently expect the second-call response accept the original
issueComments variable.
- Around line 9-14: This workflow uses the pull_request_review trigger and can
run concurrently with peer-review-reminder.yml and
check-unresolved-conversations.yml, risking race conditions when posting
comments or requesting reviewers; add a GitHub Actions concurrency block (e.g.,
group: pr-review-workflows-${{ github.event.pull_request.number || github.run_id
}} and cancel-in-progress: false) at the top of this workflow to serialize
executions of pull_request_review-triggered workflows so API interactions are
not performed concurrently.
- Around line 136-142: The loop over allIssueComments may throw when
comment.body is null and miscounts bots as reviewers; update the logic where
allIssueComments.data.forEach(comment => ...) to first guard that comment.body
is a non-null string before calling comment.body.includes(notificationMarker),
and broaden bot filtering by checking the comment author is not a bot (use
comment.user.type === 'Bot' or filter logins that end with "[bot]" or a small
known-bot list) in addition to the existing comment.user.login !== pr.user.login
check before adding to allReviewers; ensure you reference allIssueComments,
comment.body, comment.user.login, notificationMarker, and allReviewers when
applying these guards.
- Around line 219-224: The current use of github.rest.pulls.list with per_page:
100 can miss PRs beyond the first page; replace the call with github.paginate to
iterate all pages (use github.paginate(github.rest.pulls.list, { owner:
context.repo.owner, repo: context.repo.repo, state: 'open', per_page: 100 }) or
equivalent) so you collect every open pull request, and update any variable
names like pulls to accept the aggregated array returned by paginate instead of
a single page response.
- Around line 119-147: The set allReviewers currently collects logins from
reviews.data, reviewComments.data and allIssueComments.data but only explicitly
deletes coderabbitBot; update the collection/filtering so all bot accounts are
excluded (e.g., any login that endsWith "[bot]" or matches a bot regex) before
logging and before final decisions; adjust logic around
allReviewers.delete(coderabbitBot) to instead perform a generic bot-filter step
using the same notificationMarker/pr.user.login checks so human reviewers remain
while dependabot[bot], github-actions[bot], etc., are removed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 384b09e2-9df7-40d1-b50b-0826f947f2d2
📒 Files selected for processing (2)
.github/PR_REVIEW_REMINDER_SETUP.md.github/workflows/pr-review-reminder.yml
There was a problem hiding this comment.
Pull request overview
This PR adds an automated GitHub Actions workflow (pr-review-reminder.yml) that monitors open PRs and notifies maintainers when a PR has been reviewed only by the CodeRabbit AI bot and no human review has occurred within 24 hours. It complements the existing peer-review-reminder.yml by targeting the specific scenario of AI-only reviews. A setup/documentation file is also included.
Changes:
- New workflow file that runs on a 6-hour schedule, on PR review events, and on manual dispatch to detect and notify about PRs lacking human review
- New setup documentation file explaining configuration, usage, and troubleshooting
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
.github/workflows/pr-review-reminder.yml |
Main workflow automation: fetches open PRs, checks for human reviewers, and posts reminder comments/review requests |
.github/PR_REVIEW_REMINDER_SETUP.md |
Setup and usage documentation for the new workflow |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
.github/workflows/pr-review-reminder.yml (1)
151-152:⚠️ Potential issue | 🟠 MajorStatic bot exemptions still miss other automation accounts.
Removing only the logins listed in
EXEMPT_BOTSmeans any new bot account is treated as a human reviewer and blocks notifications. Filter generic bot identities (user.type === 'Bot'or[bot]suffix) when collecting reviewers instead of relying solely on a hard-coded list.Suggested fix
+ const isBotUser = (user) => + user && + (user.type === 'Bot' || (user.login && user.login.endsWith('[bot]'))); + // Add review submitters reviews.data.forEach(review => { - if (review.user && review.user.login) { + if (review.user && review.user.login && !isBotUser(review.user)) { allReviewers.add(review.user.login); } }); // Add review comment authors reviewComments.data.forEach(comment => { - if (comment.user && comment.user.login) { + if (comment.user && comment.user.login && !isBotUser(comment.user)) { allReviewers.add(comment.user.login); } }); @@ - if (comment.user && comment.user.login && + if (comment.user && comment.user.login && + !isBotUser(comment.user) && comment.user.login !== pr.user.login && comment.body && !comment.body.includes(notificationMarker)) { allReviewers.add(comment.user.login); } }); - - // Remove all bot accounts from the set - exemptBots.forEach(bot => allReviewers.delete(bot));🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/pr-review-reminder.yml around lines 151 - 152, When removing exempt reviewers, also exclude generic automation accounts instead of only using the hard-coded exemptBots list: when you build or prune allReviewers (the block that currently calls exemptBots.forEach(bot => allReviewers.delete(bot))), iterate the reviewer identities and delete any whose login indicates a bot (e.g., user.type === 'Bot' or login.endsWith('[bot]')) in addition to deleting entries in exemptBots/EXEMPT_BOTS; this ensures both explicit exemptions and any new GitHub App/bot accounts are filtered out.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/workflows/pr-review-reminder.yml:
- Around line 178-187: The PR title sanitation currently only escapes markdown
characters (sanitizedTitle) but leaves `@mentions` intact, allowing pings; update
the sanitization of pr.title to also neutralize any '@' mentions before building
commentBody by replacing or inserting a zero-width character after the '@'
(e.g., transform '@' into '@' + zero-width-space) so usernames/teams no longer
resolve, and use that new neutralizedTitle wherever sanitizedTitle is used in
commentBody.
- Around line 58-62: The code currently parses input into hoursThreshold using
parseInt (variables input, hoursThreshold) but doesn't validate the result;
update the logic to validate that the parsed value is a finite integer > 0 (use
Number.isInteger(hoursThreshold) && hoursThreshold > 0 or Number.isFinite check)
and reject/stop the run if validation fails (e.g., set the workflow to failed or
exit with a clear error message referencing DEFAULT_HOURS_THRESHOLD and the
invalid input) instead of proceeding with NaN or negative values.
- Around line 126-145: The PR author's own activity is currently being added to
allReviewers from both reviews.data and reviewComments.data; adjust the loops
that add review submitters (reviews.data.forEach) and review-comment authors
(reviewComments.data.forEach) to skip entries where review.user.login ===
pr.user.login (same check as used for allIssueComments.data). In practice,
update the conditional inside the reviews.data.forEach and
reviewComments.data.forEach blocks to also require comment.user.login !==
pr.user.login (or review.user.login !== pr.user.login) before calling
allReviewers.add so the PR author cannot suppress reminders by reviewing or
commenting on their own PR.
- Around line 76-118: The current calls to github.rest.issues.listComments,
github.rest.pulls.listReviews, and github.rest.pulls.listReviewComments use
per_page:100 and access .data, which misses pages; replace each with
github.paginate(...) for the corresponding REST method (e.g.,
github.paginate(github.rest.issues.listComments, {...}),
github.paginate(github.rest.pulls.listReviews, {...}),
github.paginate(github.rest.pulls.listReviewComments, {...})), remove the
`.data` accesses (so issueComments, reviews, reviewComments become plain
arrays), and update usages like issueComments.data.some(...) ->
issueComments.some(...), reviews.data.length -> reviews.length, and
reviewComments.data -> reviewComments so logic that checks notificationMarker,
reviewer counts, and comments works across all pages.
---
Duplicate comments:
In @.github/workflows/pr-review-reminder.yml:
- Around line 151-152: When removing exempt reviewers, also exclude generic
automation accounts instead of only using the hard-coded exemptBots list: when
you build or prune allReviewers (the block that currently calls
exemptBots.forEach(bot => allReviewers.delete(bot))), iterate the reviewer
identities and delete any whose login indicates a bot (e.g., user.type === 'Bot'
or login.endsWith('[bot]')) in addition to deleting entries in
exemptBots/EXEMPT_BOTS; this ensures both explicit exemptions and any new GitHub
App/bot accounts are filtered out.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5946ba2f-9c48-4e41-96be-a097f04d53b8
📒 Files selected for processing (2)
.github/PR_REVIEW_REMINDER_SETUP.md.github/workflows/pr-review-reminder.yml
github-actions
bot
dismissed
their stale review
All review conversations have been resolved.
github-actions
bot
added
files-changed: 3
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Contributor
👀 Peer Review RequiredHi @Flashl3opard! This pull request does not yet have a peer review. Before this PR can be merged, please request a review from one of your peers:
Thank you for contributing! 🎉 |
github-actions
bot
dismissed
their stale review
All review conversations have been resolved.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.github/PR_REVIEW_REMINDER_SETUP.md:
- Around line 11-14: Update the incorrect fragment identifiers in the
quick-links: change the "How It Works" link target from
WORKFLOWS.md#how-it-works-2 to WORKFLOWS.md#how-it-works-1 and change the
"Configuration" link target from WORKFLOWS.md#configuration-1 to
WORKFLOWS.md#configuration in the .github/PR_REVIEW_REMINDER_SETUP.md file so
the "How It Works" and "Configuration" anchors match the headings in
WORKFLOWS.md.
In @.github/workflows/pr-review-reminder.yml:
- Around line 160-163: The catch blocks that currently log errors and return
false (the reviewer-lookup catch and the notification-send catch) are turning
transient GitHub API failures into a silent “all clear”; update these blocks to
propagate failures instead of swallowing them: replace the return false behavior
with pushing a failure record into a shared failures array (include PR number,
action, and error) or rethrow the error so the main loop can detect it, and
update the final summary logic to include the failures array alongside successes
so API errors are reported separately; make this change for the reviewer lookup
catch and the notification-send catch (the two catch blocks that log the error
and return/continue).
- Around line 64-67: Update the marker-checking logic so the workflow only
treats a comment containing notificationMarker as authoritative if the comment
author is a trusted actor (e.g., "github-actions[bot]"); add an allowlist
(TRUSTED_ACTORS env or hardcoded set) and change the checks that currently just
inspect comment.body (the branches around the notificationMarker handling and
the allReviewers exclusion) to also verify comment.author.login is in the
trusted set before skipping/removing reviewers—use the existing
notificationMarker and exemptBots variables and apply this check at both places
that filter comments (the marker-detection branch and the allReviewers filter).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 84b46964-604f-4528-a87d-282fbbd617c4
📒 Files selected for processing (3)
.github/PR_REVIEW_REMINDER_SETUP.md.github/WORKFLOWS.md.github/workflows/pr-review-reminder.yml
…g, fix anchor links
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.github/workflows/pr-review-reminder.yml (1)
219-222:⚠️ Potential issue | 🟠 MajorError in
notifyMaintainersis logged but not propagated.When
createCommentfails at line 191, the catch block at lines 219-221 logs the error but does not throw, so the main loop treats notification failures as success. This is inconsistent withshouldNotifyPRwhich now throws on error. If posting the comment fails, the PR should be tracked as failed.🔧 Proposed fix
} catch (error) { console.error(`Error notifying maintainers for PR #${pr.number}:`, error); + throw error; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In @.github/workflows/pr-review-reminder.yml around lines 219 - 222, The catch in notifyMaintainers currently only logs the error (from createComment) and swallows it; change notifyMaintainers to rethrow the error after logging (or throw a new Error with context) so callers can detect failure—aligning behavior with shouldNotifyPR which throws on error; update the catch in notifyMaintainers (the block catching errors around createComment) to rethrow the caught error (or throw a wrapped error that includes the PR number and original error) so the main loop will treat the notification as failed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In @.github/workflows/pr-review-reminder.yml:
- Around line 219-222: The catch in notifyMaintainers currently only logs the
error (from createComment) and swallows it; change notifyMaintainers to rethrow
the error after logging (or throw a new Error with context) so callers can
detect failure—aligning behavior with shouldNotifyPR which throws on error;
update the catch in notifyMaintainers (the block catching errors around
createComment) to rethrow the caught error (or throw a wrapped error that
includes the PR number and original error) so the main loop will treat the
notification as failed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 0f21b30b-bb07-4125-bc17-e3b4f389b9f7
📒 Files selected for processing (2)
.github/PR_REVIEW_REMINDER_SETUP.md.github/workflows/pr-review-reminder.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements a GitHub Action workflow that automatically notifies maintainers when a Pull Request has only been reviewed by CodeRabbit AI and no human review has occurred within 24 hours.
Features
New Files
.github/workflows/pr-review-reminder.yml- Main workflow automation.github/PR_REVIEW_REMINDER_SETUP.md- Setup documentationConfiguration
A1L13N🚀 How It Works
Triggers:
For each open PR:
If all conditions met:
Summary by CodeRabbit
New Features
Documentation