# V5 Workshop Discovery & Ranking Integrity

plan-id: `v5-workshop-discovery-integrity.v1`

Status: Draft for review — fills audit gap V1_V7_PLAN_SET_AUDIT_2026-06-12
§6.2. The workshop spec (features§"Workshop & Mods") defines discovery
surfaces (browse by tag/cell/type/rating/recency/subscriber count, featured
slots, main-menu showcase) and a **creator standing** score — but standing
covers *publishers only*. Nothing protects the ranking inputs themselves:
ratings, subscriber counts, and reports are all attacker-controllable, and on
a marketplace where rank drives 70%-share real-money revenue
(`year1_modder_marketplace_manifest.json`), ranking manipulation is a
financial exploit, not a vanity problem.

Owners: Trust & Safety Lead (policy + enforcement), Workshop service owner
(implementation in the `workshop` service), Anti-Cheat ML owner (shared
anomaly/sock-puppet models — the same telemetry-stream classifier
infrastructure, features§"Anti-Cheat"), Live-Ops Producer (appeal SLAs).

All thresholds below are launch values — planning assumptions adopted
2026-06-12, tuned against closed-beta data (the closed beta runs the workshop
live, `beta-program.md` Phase 2) and reviewed monthly thereafter.

---

## 1. Rating Eligibility and Rate Limits

Make each fake rating expensive before trying to detect it:

- **One rating per account per item** (revisable; revision replaces, never
  adds). Spec already uses five-star ratings.
- **Engagement gate**: rating eligibility requires owning/subscribing the item
  AND ≥30 minutes of in-session use of it (telemetry-verified via the
  mod-set session attribution that already exists for the data-only loader);
  paid items additionally require the 48-hour refund window to have opened
  (i.e., a real, charged purchase — refunded purchases lose their rating).
- **Velocity caps**: max 10 ratings per account per day, 40 per week; max 3
  ratings per day targeting the same creator.
- **Account maturity**: accounts < 7 days old or < 5 played hours cannot rate
  (they can subscribe and play; they just don't move rankings yet).
- **Report rate limits**: max 10 content reports/day/account; reports against
  the same item from accounts sharing a §4 cluster collapse into one weighted
  report.

## 2. Rater Reputation (the gap the spec leaves open)

A per-account **rater-reputation weight** `r ∈ [0, 1]`, distinct from the
existing creator standing (which throttles *publishers*). Displayed score and
rank use **weighted** ratings:

```
score(item) = Σ_i ( r_i x stars_i ) / Σ_i r_i        (+ Bayesian prior, §3)
```

- Initialization: new eligible raters start at **r = 0.3**.
- Earning weight: +0.05 per 30 days of account good standing (cap 1.0);
  ratings that *agree with the long-run consensus* of high-reputation raters
  on widely-rated items nudge +0.01 (cap respected) — agreement is measured
  only on items with ≥200 weighted ratings to avoid punishing taste on niche
  items.
- Losing weight: rating through a brigade window that is later confirmed
  (§3) → ×0.5; upheld report-abuse finding → ×0.25; §4 sock-puppet cluster
  membership → r := 0 (and worse, §4). Weight decay is account-scoped and
  recoverable (floor 0.05, recovery through the earning path) except for
  confirmed sock-puppets.
- `r` is **never displayed** (publishing it teaches attackers the model);
  the appeal path (§5) discloses *that* down-weighting occurred, not the
  score.

## 3. Brigading and Review-Bombing Countermeasures

**Display-score robustness (always on):**

- Bayesian prior: displayed score = weighted mean shrunk toward the
  catalog-category mean with prior strength `m = 25` weighted ratings —
  kills the "5 fake five-stars at birth" and "10 one-stars kill a new item"
  attacks symmetrically.
- Rank freshness uses time-decayed rating mass (half-life 90 days), so an
  ancient brigade can't permanently pin an item.

**Anomaly detection signals (evaluated hourly per item, ML pipeline shared
with the anti-cheat classifier infra):**

| Signal                                                                      | Threshold (launch values)                              | Action on trip |
| ---------------------------------------------------------------------------- | ------------------------------------------------------- | -------------- |
| Rating-velocity z-score vs the item's own 30-day baseline                   | z > 4 over a 6 h window (min 20 ratings in window)      | Auto-hold: window's ratings quarantined from the displayed score pending review |
| One-sided burst: ≥80% of a ≥30-rating 24 h window at 1★ (or 5★)             | trip                                                     | Auto-hold + T&S review queue (review SLA 48 h)         |
| Off-platform coordination correlate: burst onset within 24 h of a creator's drama spike (report-volume z > 4 on the *creator*, not the item) | trip | Review-bomb protocol below |
| New-rater share: >50% of a window's raters at r ≤ 0.3                        | trip                                                     | Window weight ×0.25 pending review                      |
| Refund-then-rate pattern on paid items: >10% of window raters refund within 24 h of rating | trip                                      | Ratings from refunders removed (per §1); cluster check (§4) |

**Review-bomb protocol** (confirmed off-platform-motivated bombing, the
"creator said something / patch broke a different mod" case):

1. Freeze the **displayed** score at its pre-window value; banner on the item
   page: "Recent ratings are under review" (honest, neutral — players are
   told, the score is not silently edited).
2. T&S review within 48 h classifies the window: legitimate quality collapse
   (e.g., an update genuinely broke the item — the linter/compat flags
   corroborate) → ratings restored and the score moves; coordinated bombing →
   window excluded, raters take §2 weight penalties.
3. Exclusions are logged per item and **auditable on appeal** (§5); a
   quarterly transparency note reports counts of windows held/excluded
   (consistent with the DSA-transparency posture the moderation pipeline
   already carries).

**Subscriber-count and showcase integrity:** subscriber counts used for
ranking/featured eligibility count only accounts that loaded the item in ≥1
session in the last 90 days ("active subscribers") — bot-subscribed ghosts
decay out automatically; featured-slot and main-menu-showcase eligibility
additionally require clean §3 history over the trailing 30 days.

## 4. Sock-Puppet Detection

Target: rings of accounts inflating one creator (or burying a competitor).
Detection layers, run as a daily batch over the rating/report graph:

- **Identity features**: shared payment instrument fingerprint, shared
  device/hardware fingerprint, IP /24 + timing co-occurrence, account-creation
  burst patterns (≥3 accounts, same /24, < 72 h apart).
- **Graph features**: rating-target overlap — pairwise Jaccard similarity of
  rated-item sets > 0.6 across ≥ 10 items flags a pair; connected components
  of flagged pairs with ≥ 3 accounts form a **cluster**. Bipartite
  spectral/co-clustering on the account×item matrix catches rings that
  randomize superficial identity features but still concentrate fire on the
  same targets.
- **Behavioral features**: rating-only accounts (ratings ≥ 10, played hours
  < 2 beyond the §1 engagement minimum), inter-rating intervals more regular
  than human (coefficient of variation < 0.2 across ≥10 events).

Actions (graduated, cluster-scoped):

1. Cluster confidence ≥ medium: all cluster ratings → weight 0, retroactively;
   affected item scores recompute (this is the main remedy — quiet and
   complete).
2. High confidence + a beneficiary link (cluster inflates one creator who has
   a §4-relevant connection to the cluster — shared payment instrument or
   device with the creator account): case to T&S for **creator-side**
   enforcement through the existing creator-standing system; on a paid item
   this is marketplace fraud — listing review and possible payout hold per
   `ugc-marketplace-compliance.md` §6.
3. Confirmed rings: account enforcement through the standard ban-tier ladder
   (features§"Cross-Platform Social, Reporting, and Bans") — discovery
   manipulation is a "griefing/cheating"-class offense, not a chat offense.
4. Ban-wave hygiene mirrors anti-cheat practice: enforcement in periodic
   waves (not instant) so rings can't A/B-test detection boundaries; every
   wave human-reviewed before execution (the spec's "flags route to human
   review rather than auto-banning" principle applies here too).

False-positive guards: households/LAN cafés share IPs and devices —
identity-feature hits alone (without graph or behavioral corroboration) never
exceed "low confidence" and trigger no action beyond logging; clusters require
at least one graph or behavioral feature to act.

## 5. Appeal Flow

Rides the existing appeal infrastructure (companion app, 7-day SLA — the same
channel the spec gives ban and moderation appeals):

| Who appeals          | What                                                            | Path |
| -------------------- | ---------------------------------------------------------------- | ---- |
| Creator              | Score freeze, window exclusion, featured-eligibility loss        | Companion app → T&S queue; 7-day SLA; outcome with reason (which §3 protocol step applied); second-level review by a different moderator on request (DSA-consistent) |
| Rater                | Rating quarantined/zeroed, rate-limit strike, cluster membership | Same channel; 7-day SLA; remedy = weight restoration + rating reinstatement; cluster-membership appeals get human review of the §4 evidence class (without disclosing thresholds) |
| Reported item owner  | Already covered by the spec's DSA-compliant moderation appeal    | Existing path (unchanged) |

- Upheld appeals are integrity inputs: each one back-tests the §3/§4 rule that
  fired (the false-positive feeds the monthly threshold review).
- Appeal-abuse guard: appeals are never rate-limited, but repeat frivolous
  appeals (3 consecutive rejections on the same matter) drop to a
  low-priority queue — answered, just not at the expense of the SLA pool.

## 6. Metrics and Review

Monthly integrity review (Trust & Safety Lead chairs): brigade windows
detected/held/confirmed, sock-puppet clusters actioned, appeal uphold rate
(target: < 10% of §3/§4 actions overturned — higher means thresholds too hot),
median time-to-detection for confirmed brigades (target < 12 h), and
rank-stability of the top-100 marketplace items (unexplained rank swings are
the residual signal that something got through). Thresholds in §1–§4 are
versioned in the `workshop` service config so every change is auditable
against the metric record.