AgentScan Action

GitHub action that analyzes PR and issue authors’ recent activity patterns to detect automation signals.

Setup

Create a workflow file in your repository (e.g., .github/workflows/agentscan.yml):

name: AgentScan

on:
  pull_request_target:
    types:
      - opened
      - reopened
  issues:
    types:
      - opened

jobs:
  agentscan:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      issues: write
      contents: read
    steps:
      - name: AgentScan
        uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}

The action will run automatically on new and reopened pull requests, and on newly opened issues, analyzing the author’s activity patterns to detect automation signals.

Configuration

Inputs

• github-token (required): GitHub token for API access
• skip-members (optional): Comma-separated list of usernames to skip from scanning
• agent-scan-comment (optional): Enable/disable posting comments on PRs and issues (default: true). Set to false if you only want to use the outputs
• cache-path (optional): Path to cache directory for storing analysis results (e.g., .agentscan-cache). When provided, analysis results are cached and reused within the TTL period
• skip-comment-on-organic (optional): Skip posting PR or issue comment if analysis result is “organic” (default: false)
• label-community-flagged (optional): Label to add when an account is flagged by the community (default: agentscan:community-flagged)
• label-mixed (optional): Label to add when an account has mixed automation signals (default: agentscan:mixed-signals)
• label-automation (optional): Label to add when an account is classified as automated (default: agentscan:automated-account)

Skip Members

To skip specific team members from being scanned, add their usernames to the skip-members input:

- name: AgentScan
  uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    skip-members: "dependabot,renovate,my-trusted-bot"

Members in the skip list will be excluded from analysis without any PR comment or labels added.

Caching

To enable caching and avoid redundant API calls, use actions/cache@v5 and pass the cache path to the action:

steps:
  - name: Cache AgentScan analysis
    uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae
    with:
      path: .agentscan-cache
      key: agentscan-cache-${{ github.actor }}
      restore-keys: agentscan-cache-
  - name: AgentScan
    uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
    with:
      github-token: ${{ secrets.GITHUB_TOKEN }}
      cache-path: ".agentscan-cache"

How caching works:

1. Set up actions/cache with a path and unique key
2. Pass the same path to the action via cache-path input
3. The action stores analysis results in that directory
4. actions/cache persists the directory between workflow runs
5. On subsequent runs, cached results are reused if they’re within the TTL period

Cache Invalidation: Cached entries automatically expire after 2 days.

Skip Organic Comments

To skip posting a PR or issue comment when the analysis result is “organic” (clean, human-like activity), enable the skip-comment-on-organic option:

- name: AgentScan
  uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    skip-comment-on-organic: true

When enabled, the action will still output all analysis data (for downstream steps to use) but won’t post a comment on the PR or issue if the account is classified as organic.

Custom Labels

To customize labels added to PRs and issues, set any of the label inputs:

- name: AgentScan
  uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    label-community-flagged: "security:community-flagged"
    label-mixed: "needs-review:automation-signals"
    label-automation: "blocked:automated-account"

Disable Comments

To disable all PR and issue comments and only use the action’s outputs, set agent-scan-comment to false:

- name: AgentScan
  uses: MatteoGabriele/agentscan-action@f41545309db947a68e22ed2643f182e754f4d41a
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    agent-scan-comment: false

This is useful if you want to use the analysis outputs in downstream steps without posting comments.

Testing

Run tests with vitest:

pnpm run test

Tests cover the following scenarios:

• Normal Flow: Analyzes a user without cache, saves result with timestamp
• Cached Flow:
  • Fresh cache (< 2 days): Uses cached data, skips API calls
  • Stale cache (≥ 2 days): Invalidates cache, makes fresh API calls
  • Corrupted cache: Falls back to API calls with warning
• Skip-Member Flow: Members in skip list are not analyzed
• Label Assignment: Correct labels added based on classification (organic, mixed, automation, community-flagged)
• Issue Scanning: Analyzes issue authors with the same automation detection pipeline, posts comments and labels on issues

---

Stay safe out there, fellow human, and use AI responsibly.