V5 Workshop Discovery & Ranking Integrity

plan-id: v5-workshop-discovery-integrity.v1

Status: Draft for review — fills audit gap V1_V7_PLAN_SET_AUDIT_2026-06-12 §6.2. The workshop spec (features§"Workshop & Mods") defines discovery surfaces (browse by tag/cell/type/rating/recency/subscriber count, featured slots, main-menu showcase) and a creator standing score — but standing covers publishers only. Nothing protects the ranking inputs themselves: ratings, subscriber counts, and reports are all attacker-controllable, and on a marketplace where rank drives 70%-share real-money revenue (year1_modder_marketplace_manifest.json), ranking manipulation is a financial exploit, not a vanity problem.

Owners: Trust & Safety Lead (policy + enforcement), Workshop service owner (implementation in the workshop service), Anti-Cheat ML owner (shared anomaly/sock-puppet models — the same telemetry-stream classifier infrastructure, features§"Anti-Cheat"), Live-Ops Producer (appeal SLAs).

All thresholds below are launch values — planning assumptions adopted 2026-06-12, tuned against closed-beta data (the closed beta runs the workshop live, beta-program.md Phase 2) and reviewed monthly thereafter.

1. Rating Eligibility and Rate Limits#

Make each fake rating expensive before trying to detect it:

• One rating per account per item (revisable; revision replaces, never adds). Spec already uses five-star ratings.
• Engagement gate: rating eligibility requires owning/subscribing the item AND ≥30 minutes of in-session use of it (telemetry-verified via the mod-set session attribution that already exists for the data-only loader); paid items additionally require the 48-hour refund window to have opened (i.e., a real, charged purchase — refunded purchases lose their rating).
• Velocity caps: max 10 ratings per account per day, 40 per week; max 3 ratings per day targeting the same creator.
• Account maturity: accounts < 7 days old or < 5 played hours cannot rate (they can subscribe and play; they just don't move rankings yet).
• Report rate limits: max 10 content reports/day/account; reports against the same item from accounts sharing a §4 cluster collapse into one weighted report.

2. Rater Reputation (the gap the spec leaves open)#

A per-account rater-reputation weight r ∈ [0, 1], distinct from the existing creator standing (which throttles publishers). Displayed score and rank use weighted ratings:

score(item) = Σ_i ( r_i x stars_i ) / Σ_i r_i        (+ Bayesian prior, §3)

• Initialization: new eligible raters start at r = 0.3.
• Earning weight: +0.05 per 30 days of account good standing (cap 1.0); ratings that agree with the long-run consensus of high-reputation raters on widely-rated items nudge +0.01 (cap respected) — agreement is measured only on items with ≥200 weighted ratings to avoid punishing taste on niche items.
• Losing weight: rating through a brigade window that is later confirmed (§3) → ×0.5; upheld report-abuse finding → ×0.25; §4 sock-puppet cluster membership → r := 0 (and worse, §4). Weight decay is account-scoped and recoverable (floor 0.05, recovery through the earning path) except for confirmed sock-puppets.
• r is never displayed (publishing it teaches attackers the model); the appeal path (§5) discloses that down-weighting occurred, not the score.

3. Brigading and Review-Bombing Countermeasures#

Display-score robustness (always on):

• Bayesian prior: displayed score = weighted mean shrunk toward the catalog-category mean with prior strength m = 25 weighted ratings — kills the "5 fake five-stars at birth" and "10 one-stars kill a new item" attacks symmetrically.
• Rank freshness uses time-decayed rating mass (half-life 90 days), so an ancient brigade can't permanently pin an item.

Anomaly detection signals (evaluated hourly per item, ML pipeline shared with the anti-cheat classifier infra):

Review-bomb protocol (confirmed off-platform-motivated bombing, the "creator said something / patch broke a different mod" case):

1. Freeze the displayed score at its pre-window value; banner on the item page: "Recent ratings are under review" (honest, neutral — players are told, the score is not silently edited).
2. T&S review within 48 h classifies the window: legitimate quality collapse (e.g., an update genuinely broke the item — the linter/compat flags corroborate) → ratings restored and the score moves; coordinated bombing → window excluded, raters take §2 weight penalties.
3. Exclusions are logged per item and auditable on appeal (§5); a quarterly transparency note reports counts of windows held/excluded (consistent with the DSA-transparency posture the moderation pipeline already carries).

Subscriber-count and showcase integrity: subscriber counts used for ranking/featured eligibility count only accounts that loaded the item in ≥1 session in the last 90 days ("active subscribers") — bot-subscribed ghosts decay out automatically; featured-slot and main-menu-showcase eligibility additionally require clean §3 history over the trailing 30 days.

4. Sock-Puppet Detection#

Target: rings of accounts inflating one creator (or burying a competitor). Detection layers, run as a daily batch over the rating/report graph:

• Identity features: shared payment instrument fingerprint, shared device/hardware fingerprint, IP /24 + timing co-occurrence, account-creation burst patterns (≥3 accounts, same /24, < 72 h apart).
• Graph features: rating-target overlap — pairwise Jaccard similarity of rated-item sets > 0.6 across ≥ 10 items flags a pair; connected components of flagged pairs with ≥ 3 accounts form a cluster. Bipartite spectral/co-clustering on the account×item matrix catches rings that randomize superficial identity features but still concentrate fire on the same targets.
• Behavioral features: rating-only accounts (ratings ≥ 10, played hours < 2 beyond the §1 engagement minimum), inter-rating intervals more regular than human (coefficient of variation < 0.2 across ≥10 events).

Actions (graduated, cluster-scoped):

1. Cluster confidence ≥ medium: all cluster ratings → weight 0, retroactively; affected item scores recompute (this is the main remedy — quiet and complete).
2. High confidence + a beneficiary link (cluster inflates one creator who has a §4-relevant connection to the cluster — shared payment instrument or device with the creator account): case to T&S for creator-side enforcement through the existing creator-standing system; on a paid item this is marketplace fraud — listing review and possible payout hold per ugc-marketplace-compliance.md §6.
3. Confirmed rings: account enforcement through the standard ban-tier ladder (features§"Cross-Platform Social, Reporting, and Bans") — discovery manipulation is a "griefing/cheating"-class offense, not a chat offense.
4. Ban-wave hygiene mirrors anti-cheat practice: enforcement in periodic waves (not instant) so rings can't A/B-test detection boundaries; every wave human-reviewed before execution (the spec's "flags route to human review rather than auto-banning" principle applies here too).

False-positive guards: households/LAN cafés share IPs and devices — identity-feature hits alone (without graph or behavioral corroboration) never exceed "low confidence" and trigger no action beyond logging; clusters require at least one graph or behavioral feature to act.

5. Appeal Flow#

Rides the existing appeal infrastructure (companion app, 7-day SLA — the same channel the spec gives ban and moderation appeals):

• Upheld appeals are integrity inputs: each one back-tests the §3/§4 rule that fired (the false-positive feeds the monthly threshold review).
• Appeal-abuse guard: appeals are never rate-limited, but repeat frivolous appeals (3 consecutive rejections on the same matter) drop to a low-priority queue — answered, just not at the expense of the SLA pool.

6. Metrics and Review#

Monthly integrity review (Trust & Safety Lead chairs): brigade windows detected/held/confirmed, sock-puppet clusters actioned, appeal uphold rate (target: < 10% of §3/§4 actions overturned — higher means thresholds too hot), median time-to-detection for confirmed brigades (target < 12 h), and rank-stability of the top-100 marketplace items (unexplained rank swings are the residual signal that something got through). Thresholds in §1–§4 are versioned in the workshop service config so every change is auditable against the metric record.