Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions .github/workflows/manage-stale-items.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Manage stale issues and PRs

# Runs daily across all 12 public repositories to:
# 1. Mark issues and PRs with no activity for 90 days as "stale" with a notice.
# 2. Auto-close "stale" items with no activity after an additional 30 days.
# 3. Remove the "stale" label if new activity occurs.

on:
schedule:
# Daily at 09:00 UTC.
- cron: '0 9 * * *'
workflow_dispatch:
inputs:
dry_run:
description: 'Preview actions without posting comments or closing items'
type: boolean
default: true
max_writes:
description: 'Maximum comment/close actions per run (rate-limit safety)'
type: string
default: '150'

concurrency:
group: manage-stale-items
cancel-in-progress: false

permissions: {}

jobs:
stale:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
# Pinned to a commit hash to satisfy the org's mandatory zizmor policy.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0

- name: Manage stale issues and pull requests
env:
GH_TOKEN: ${{ secrets.PROJECT_SYNC_TOKEN }}
DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && (inputs.dry_run && '1' || '0') || '0' }}
MAX_WRITES: ${{ inputs.max_writes || '150' }}
run: ./scripts/manage_stale_items.sh
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ Org-level defaults and automation for the Data Commons GitHub organization.
| `SECURITY.md` | Security policy. GitHub falls back to this for every repo in the org that has no `SECURITY.md` of its own. |
| `scripts/sync_project_items.sh` | Keeps the triage project stocked and tagged. |
| `.github/workflows/sync-project-items.yml` | Runs that script every 30 minutes. |
| `scripts/manage_stale_items.sh` | Marks 90-day inactive issues/PRs as stale and closes them after 30 more days. |
| `.github/workflows/manage-stale-items.yml` | Runs the stale cleanup script daily. |

> [!NOTE]
> A `README.md` at the root of this repo is just this repo's readme. The org's public profile page comes from `profile/README.md`, which does not exist here. Don't move this file there.
Expand Down Expand Up @@ -36,6 +38,21 @@ DRY_RUN=1 ./scripts/sync_project_items.sh # report, change nothing
WAIT_FOR_BUDGET=1 ./scripts/sync_project_items.sh # sit through rate limits
```

## Stale issue and PR management

`scripts/manage_stale_items.sh` runs daily across all 12 public repos to keep the backlog actionable:

1. **Warning after 90 days:** Open issues and PRs with no activity for 90 days receive a polite notice comment and the `stale` label.
2. **Auto-close after 30 more days:** Items with the `stale` label that receive no activity for 30 days after the warning (120+ days total) are automatically closed with an invitation to reopen if still relevant.
3. **Un-stale on new activity:** If someone comments or updates an item after the warning comment, the `stale` label is automatically removed.
4. **Exemptions:** Items with an active milestone or labeled `keep-open`, `pinned`, or `security` are never marked stale.

```bash
DRY_RUN=1 ./scripts/manage_stale_items.sh # preview actions across all repos
DRY_RUN=1 ./scripts/manage_stale_items.sh mixer # preview actions for one repo
./scripts/manage_stale_items.sh mixer # run live against mixer
```

## Routine tasks

### Someone joins or leaves the team
Expand Down
200 changes: 200 additions & 0 deletions scripts/manage_stale_items.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
#!/usr/bin/env bash
#
# Automated stale issue and pull request management across Data Commons public
# repositories.
#
# Policy:
# 1. Warning (90 days): Open issues or PRs with no activity for WARN_DAYS
# receive a polite comment and the "stale" label.
# 2. Auto-close (30 days after warning): Items with the "stale" label that
# have had no activity for CLOSE_DAYS (120+ days total) are closed with a
# friendly invitation to reopen if still relevant.
# 3. Un-stale: If someone comments or updates a "stale" item after the
# warning comment, the "stale" label is automatically removed.
# 4. Exemptions: Items with an active milestone or any exempt label
# (keep-open, pinned, security) are skipped.
#
# Usage:
# ./scripts/manage_stale_items.sh # all repos
# ./scripts/manage_stale_items.sh mixer website # specific repos
# DRY_RUN=1 ./scripts/manage_stale_items.sh # preview actions only

set -euo pipefail

ORG="datacommonsorg"
WARN_DAYS="${WARN_DAYS:-90}"
CLOSE_DAYS="${CLOSE_DAYS:-30}"
STALE_LABEL="stale"
EXEMPT_LABELS=("keep-open" "pinned" "security")
DRY_RUN="${DRY_RUN:-0}"
# Cap comment/state writes per run to avoid GitHub secondary rate limits.
MAX_WRITES="${MAX_WRITES:-150}"

REPOS=(
agent-toolkit api-python data datacommons deployment-engine docsite
import llm-tools mixer schema tools website
)
[[ $# -gt 0 ]] && REPOS=("$@")

MARKER="<!-- datacommons-stale-warning -->"

WARN_BODY="This issue or pull request has had no activity for ${WARN_DAYS} days and has been marked as \`${STALE_LABEL}\`. Is this still relevant?

If there is no activity in the next ${CLOSE_DAYS} days, it will be closed automatically. Leave a comment or apply the \`keep-open\` label if you would like to keep it open.
${MARKER}"

CLOSE_BODY="Closing this issue or pull request automatically because it has had no activity for ${CLOSE_DAYS} days since being marked \`${STALE_LABEL}\`.

If this is still relevant, please feel free to reopen it or leave a comment and we will take a look."

WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT

NOW_EPOCH="$(date -u +%s)"

iso_to_epoch() {
local iso="$1"
date -u -d "$iso" +%s 2>/dev/null \
|| date -u -j -f '%Y-%m-%dT%H:%M:%SZ' "$iso" +%s 2>/dev/null \
|| echo "$NOW_EPOCH"
}
Comment on lines +55 to +60

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since date parsing is now handled directly in jq, the iso_to_epoch function is no longer needed and can be removed.

Suggested change
iso_to_epoch() {
local iso="$1"
date -u -d "$iso" +%s 2>/dev/null \
|| date -u -j -f '%Y-%m-%dT%H:%M:%SZ' "$iso" +%s 2>/dev/null \
|| echo "$NOW_EPOCH"
}
# Date parsing is handled directly in jq


ensure_stale_label() {
local repo="$1"
if [[ "$DRY_RUN" == "1" ]]; then return 0; fi
# Create label if it does not exist; ignore 422 if already present.
gh api --method POST "/repos/$ORG/$repo/labels" \
-f name="$STALE_LABEL" \
-f color="ededed" \
-f description="No activity for ${WARN_DAYS}+ days" \
>/dev/null 2>&1 || true
}

warned=0
closed=0
unstaled=0
exempt=0
active=0
writes=0
unlisted=0

# Extract: number <TAB> html_url <TAB> updated_at <TAB> has_milestone <TAB> labels_csv
ITEM_JQ='.[] | [
.number,
.html_url,
.updated_at,
(if .milestone != null then "yes" else "no" end),
([.labels[].name | ascii_downcase] | join(","))
] | @tsv'
Comment on lines +82 to +88

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of parsing ISO 8601 dates using platform-specific date commands in a bash loop (which spawns a subshell for every single issue/PR), you can parse the date directly to an epoch timestamp using jq's built-in fromdateiso8601 function. This is cross-platform, highly efficient, and eliminates the need for the iso_to_epoch helper function.

We can also extract whether the item is a Pull Request here to fix the state_reason issue.

Suggested change
ITEM_JQ='.[] | [
.number,
.html_url,
.updated_at,
(if .milestone != null then "yes" else "no" end),
([.labels[].name | ascii_downcase] | join(","))
] | @tsv'
ITEM_JQ='.[] | [
.number,
.html_url,
(.updated_at | fromdateiso8601),
(if .milestone != null then "yes" else "no" end),
([.labels[].name | ascii_downcase] | join(",")),
(if .pull_request != null then "yes" else "no" end)
] | @tsv'


for repo in "${REPOS[@]}"; do
: > "$WORK/items.tsv"
if ! gh api "/repos/$ORG/$repo/issues?state=open&per_page=100" --paginate \
--jq "$ITEM_JQ" >> "$WORK/items.tsv"; then
echo " could not list items for $repo" >&2
unlisted=$((unlisted + 1))
continue
fi

ensure_stale_label "$repo"

while IFS=$'\t' read -r num url updated_at has_milestone labels_csv; do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the read command to match the new fields returned by ITEM_JQ (using the parsed epoch timestamp and the is_pr flag).

Suggested change
while IFS=$'\t' read -r num url updated_at has_milestone labels_csv; do
while IFS=$'\t' read -r num url upd_epoch has_milestone labels_csv is_pr; do

[[ -z "$num" ]] && continue

# 1. Check milestone exemption
if [[ "$has_milestone" == "yes" ]]; then
exempt=$((exempt + 1))
continue
fi

# 2. Check label exemptions and current stale status
is_exempt=0
has_stale=0
padded_labels=",$labels_csv,"
for ex in "${EXEMPT_LABELS[@]}"; do
if [[ "$padded_labels" == *",$ex,"* ]]; then
is_exempt=1
break
fi
done
if [[ "$is_exempt" -eq 1 ]]; then
exempt=$((exempt + 1))
continue
fi
if [[ "$padded_labels" == *",$STALE_LABEL,"* ]]; then
has_stale=1
fi

upd_epoch="$(iso_to_epoch "$updated_at")"
age_days=$(( (NOW_EPOCH - upd_epoch) / 86400 ))
Comment on lines +128 to +129

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since upd_epoch is now read directly as an integer from the TSV, we no longer need to call iso_to_epoch.

Suggested change
upd_epoch="$(iso_to_epoch "$updated_at")"
age_days=$(( (NOW_EPOCH - upd_epoch) / 86400 ))
age_days=$(( (NOW_EPOCH - upd_epoch) / 86400 ))


# 3. Item already has the stale label
if [[ "$has_stale" -eq 1 ]]; then
if [[ "$age_days" -ge "$CLOSE_DAYS" ]]; then
if [[ "$DRY_RUN" == "1" ]]; then
echo " would close (${age_days}d since stale) $url"
closed=$((closed + 1))
continue
fi
if [[ "$writes" -ge "$MAX_WRITES" ]]; then
echo "Reached MAX_WRITES ($MAX_WRITES); stopping to respect rate limits."
break 2
fi
gh api --method POST "/repos/$ORG/$repo/issues/$num/comments" \
-f body="$CLOSE_BODY" >/dev/null
gh api --method PATCH "/repos/$ORG/$repo/issues/$num" \
-f state="closed" -f state_reason="not_planned" >/dev/null
Comment on lines +145 to +146

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The GitHub REST API does not support state_reason for Pull Requests (it will return a 422 Unprocessable Entity error). Since this script manages both issues and PRs, you should check if the item is a PR before applying state_reason="not_planned".

Suggested change
gh api --method PATCH "/repos/$ORG/$repo/issues/$num" \
-f state="closed" -f state_reason="not_planned" >/dev/null
if [[ "$is_pr" == "yes" ]]; then
gh api --method PATCH "/repos/$ORG/$repo/issues/$num" \
-f state="closed" >/dev/null
else
gh api --method PATCH "/repos/$ORG/$repo/issues/$num" \
-f state="closed" -f state_reason="not_planned" >/dev/null
fi

echo " closed (${age_days}d since stale) $url"
closed=$((closed + 1))
writes=$((writes + 1))
sleep 1
else
# Check if updated after the warning comment by someone else
last_body="$(gh api "/repos/$ORG/$repo/issues/$num/comments?per_page=1&direction=desc" \
--jq '.[0].body // ""' 2>/dev/null || echo "")"
Comment on lines +153 to +154

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The GitHub REST API /repos/{owner}/{repo}/issues/{issue_number}/comments endpoint does not support the direction or sort query parameters. It always returns comments in ascending order (oldest first). As a result, direction=desc is ignored, and .[0] returns the oldest comment instead of the newest. This causes any stale issue with prior comments to be immediately un-staled on the next run.

To fix this, you can fetch the comments (up to 100) and use .[-1] in jq to get the last (newest) comment.

Suggested change
last_body="$(gh api "/repos/$ORG/$repo/issues/$num/comments?per_page=1&direction=desc" \
--jq '.[0].body // ""' 2>/dev/null || echo "")"
last_body="$(gh api "/repos/$ORG/$repo/issues/$num/comments?per_page=100" \
--jq '.[-1].body // ""' 2>/dev/null || echo "")"

if [[ -n "$last_body" && "$last_body" != *"$MARKER"* ]]; then
if [[ "$DRY_RUN" == "1" ]]; then
echo " would un-stale (new activity) $url"
unstaled=$((unstaled + 1))
continue
fi
gh api --method DELETE "/repos/$ORG/$repo/issues/$num/labels/$STALE_LABEL" \
>/dev/null 2>&1 || true
echo " un-staled (new activity) $url"
unstaled=$((unstaled + 1))
writes=$((writes + 1))
else
active=$((active + 1))
fi
fi
continue
fi

# 4. Item does not have the stale label yet
if [[ "$age_days" -ge "$WARN_DAYS" ]]; then
if [[ "$DRY_RUN" == "1" ]]; then
echo " would warn (${age_days}d inactive) $url"
warned=$((warned + 1))
continue
fi
if [[ "$writes" -ge "$MAX_WRITES" ]]; then
echo "Reached MAX_WRITES ($MAX_WRITES); stopping to respect rate limits."
break 2
fi
gh api --method POST "/repos/$ORG/$repo/issues/$num/comments" \
-f body="$WARN_BODY" >/dev/null
gh api --method POST "/repos/$ORG/$repo/issues/$num/labels" \
-f "labels[]=$STALE_LABEL" >/dev/null
echo " warned (${age_days}d inactive) $url"
warned=$((warned + 1))
writes=$((writes + 1))
sleep 1
else
active=$((active + 1))
fi
done < "$WORK/items.tsv"
done

echo
echo "warned $warned, closed $closed, un-staled $unstaled, exempt $exempt, active $active"
[[ "$unlisted" -eq 0 ]]
Loading