Scan Diff Comparison

Compare two scans to see what’s been fixed, what’s new, and what remains unchanged. Track security improvement over time.

Overview

The diff feature compares findings between two scans of the same agent, showing:

• Fixed: Findings that existed in the previous scan but are gone now
• New: Findings that appeared since the last scan
• Unchanged: Findings that persist across both scans

This is essential for tracking remediation progress and catching security regressions.

How It Works

Fingerprinting

Each finding is uniquely identified by:

pattern_id + file + line

For example: prompt_injection:agent.py:42

This means:

• If you fix a vulnerability, it shows as “Fixed”
• If you add new vulnerable code, it shows as “New”
• If you refactor (move code to different lines), it may show as both “Fixed” and “New”

Column number is intentionally excluded from the fingerprint. This prevents false positives when code formatting changes (like auto-formatters) shift column positions without changing the underlying issue.

Using Diff in the Dashboard

1. Navigate to your scan results
2. Click the Compare button in the header
3. Select a previous scan to compare against
4. View the categorized findings

The diff view shows:

Using Diff via API

curl -X POST https://api.inkog.io/api/v1/diff \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "current_scan_id": "scan_abc123",
    "previous_scan_id": "scan_xyz789"
  }'

Response:

{
  "diff": {
    "fixed": [
      {
        "pattern_id": "prompt_injection",
        "file": "agent.py",
        "line": 42,
        "message": "System prompt vulnerable to injection"
      }
    ],
    "new": [],
    "unchanged": [
      {
        "pattern_id": "infinite_loop",
        "file": "workflow.py",
        "line": 15,
        "message": "Loop lacks termination guards"
      }
    ]
  },
  "summary": {
    "fixed_count": 1,
    "new_count": 0,
    "unchanged_count": 1,
    "improvement_rate": 0.5
  }
}

Best Practices

Track Progress Over Time

Run scans after each sprint or release to measure security debt reduction:

# Compare latest against baseline
inkog diff --current latest --previous v1.0.0

Use in CI/CD

Add diff comparison to your CI pipeline to catch regressions:

# GitHub Actions example
- name: Run Inkog Scan
  uses: inkog-io/inkog-action@v1
  with:
    path: .
    compare-to: ${{ github.base_ref }}

The action will:

1. Scan the current branch
2. Compare against the base branch scan
3. Fail if new critical/high findings are introduced
4. Show what was fixed in the PR

Establish Baselines

Before major refactoring, create a baseline scan:

inkog scan . --tag baseline-v2

Then compare after refactoring to ensure no security regressions:

inkog diff --current HEAD --previous baseline-v2

Understanding Results

”Fixed” Doesn’t Mean Secure

A finding marked “Fixed” means it’s no longer detected at that location. This could mean:

• The vulnerability was actually fixed
• The code was deleted
• The code moved to a different file/line

Always verify fixes are genuine remediations, not just code movement.

Multiple Findings Per Line

If the same line has multiple vulnerability types (e.g., both prompt injection and logging sensitive data), they are tracked separately. Fixing one doesn’t affect the others.

Line Number Changes

If you refactor code and the vulnerable line moves from line 42 to line 85:

• Old finding at line 42 appears as “Fixed”
• Same issue at line 85 appears as “New”

This is expected behavior. The issue wasn’t truly fixed, just relocated.

Related

• Scan History
• CI/CD Integration
• Scoring & Metrics