Redaction: a vendor tied to a panel seat is fatal in prose too and the RQC's "Claude-position" reads "first-seat"; the boilerplate says human curators, not human editors
Browse filesThe RQC on the first external paper called one seat "the Claude-position
reviewers"; the jargon rewrite covered "slot" but not that, and reviewer
prose sits outside the fatal vendor scan by design (a paper's own AI-use
disclosure is legitimate content there). A vendor or model name tied to a
seat is never a disclosure, so it is now fatal wherever it appears, and the
observed phrasing is rewritten before the gate. "Rest with human editors"
and its siblings in the review boilerplate, the RQC status line, the rubric,
the legacy templates and the README now say human curators or the curation
team.
- README.md +2 -2
- redaction.py +17 -3
- rubrics/review_quality_control.md +1 -1
- templates/accept-comment.md +1 -1
- templates/accept.md +1 -1
README.md
CHANGED
|
@@ -146,7 +146,7 @@ not a general-purpose academic peer review platform. It is not a service you
|
|
| 146 |
can submit to without going through `icsacinstitute.org/submit`. It is not a
|
| 147 |
replacement for human editorial judgment β every panel outcome routes to a
|
| 148 |
human curator for the final verdict, and any author who disagrees with that
|
| 149 |
-
verdict can appeal to
|
| 150 |
|
| 151 |
It is also opinionated. The rubrics reflect the institute's editorial scope:
|
| 152 |
complexity science, information theory, persistence dynamics, and adjacent
|
|
@@ -244,7 +244,7 @@ to fork, adapt, and use as the basis for your own institute's review system.
|
|
| 244 |
|
| 245 |
If your paper was reviewed by this system and you disagree with the panel's
|
| 246 |
recommendation: write to `help@icsacinstitute.org` with your submission ID in
|
| 247 |
-
the subject line.
|
| 248 |
curator turns into the verdict, and that the editor overrides on appeal.
|
| 249 |
|
| 250 |
If your paper was accepted: the redacted review is published at
|
|
|
|
| 146 |
can submit to without going through `icsacinstitute.org/submit`. It is not a
|
| 147 |
replacement for human editorial judgment β every panel outcome routes to a
|
| 148 |
human curator for the final verdict, and any author who disagrees with that
|
| 149 |
+
verdict can appeal to the curation team.
|
| 150 |
|
| 151 |
It is also opinionated. The rubrics reflect the institute's editorial scope:
|
| 152 |
complexity science, information theory, persistence dynamics, and adjacent
|
|
|
|
| 244 |
|
| 245 |
If your paper was reviewed by this system and you disagree with the panel's
|
| 246 |
recommendation: write to `help@icsacinstitute.org` with your submission ID in
|
| 247 |
+
the subject line. The curation team reads every appeal. The panel is not the last word β it is a thorough first pass that the
|
| 248 |
curator turns into the verdict, and that the editor overrides on appeal.
|
| 249 |
|
| 250 |
If your paper was accepted: the redacted review is published at
|
redaction.py
CHANGED
|
@@ -89,6 +89,14 @@ SOFT_WARN_TOKENS: tuple[str, ...] = (
|
|
| 89 |
# Added 2026-04-18 after prompt-injection attack-surface audit. Triggered
|
| 90 |
# by file paths pointing at our hosts, env-var assignments, and known
|
| 91 |
# credential prefixes. Match anywhere in the redacted review text.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
FORBIDDEN_EXFIL_PATTERNS: tuple[str, ...] = (
|
| 93 |
# Absolute filesystem paths likely pointing at our hosts
|
| 94 |
r"/home/orangepi\b",
|
|
@@ -375,7 +383,7 @@ def build_public_markdown(parsed: ParsedReview) -> str:
|
|
| 375 |
(
|
| 376 |
"*Reviews at ICSAC are open and transparent. AI tooling helps "
|
| 377 |
"the panel draft and structure each review; final acceptance "
|
| 378 |
-
"decisions rest with human
|
| 379 |
"alongside acceptance for accountability; individual reviewer "
|
| 380 |
"identities are abstracted to keep focus on the assessment "
|
| 381 |
"rather than the tooling behind it.*"
|
|
@@ -452,6 +460,11 @@ _JARGON_REWRITES = (
|
|
| 452 |
(re.compile(r"\bslots\b"), "reviewers"),
|
| 453 |
(re.compile(r"\bSlot\b"), "Reviewer"),
|
| 454 |
(re.compile(r"\bslot\b"), "reviewer"),
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 455 |
)
|
| 456 |
|
| 457 |
|
|
@@ -493,6 +506,7 @@ def scan(text: str) -> ScrubReport:
|
|
| 493 |
fatal = _find_substring_hits(structural, FORBIDDEN_VENDOR_TOKENS)
|
| 494 |
fatal.extend(_find_substring_hits(text, FORBIDDEN_SECRET_PHRASES))
|
| 495 |
fatal.extend(_find_regex_hits(text, FORBIDDEN_EXFIL_PATTERNS))
|
|
|
|
| 496 |
warn = _find_wordboundary_hits(text, SOFT_WARN_TOKENS)
|
| 497 |
return ScrubReport(fatal_hits=fatal, warn_hits=warn)
|
| 498 |
|
|
@@ -742,7 +756,7 @@ def build_public_rqc_markdown(parsed: ParsedRQC) -> str:
|
|
| 742 |
enforces this before the redaction writes anything to the site.
|
| 743 |
"""
|
| 744 |
status_line = (
|
| 745 |
-
"Review Quality Control: flagged β reviewed by human
|
| 746 |
if parsed.flag
|
| 747 |
else "Review Quality Control: passed."
|
| 748 |
)
|
|
@@ -775,7 +789,7 @@ def build_public_rqc_markdown(parsed: ParsedRQC) -> str:
|
|
| 775 |
lines.extend([
|
| 776 |
(
|
| 777 |
"The audit surfaced a concern outside the four scholarly "
|
| 778 |
-
"dimensions above.
|
| 779 |
"before the acceptance decision was recorded."
|
| 780 |
),
|
| 781 |
"",
|
|
|
|
| 89 |
# Added 2026-04-18 after prompt-injection attack-surface audit. Triggered
|
| 90 |
# by file paths pointing at our hosts, env-var assignments, and known
|
| 91 |
# credential prefixes. Match anywhere in the redacted review text.
|
| 92 |
+
# Panel-composition talk in reviewer or RQC prose: a vendor or model name tied
|
| 93 |
+
# to a seat ("Claude-position reviewers", "the GPT slot"). A paper's own AI-use
|
| 94 |
+
# disclosure never takes this shape, so it is fatal wherever it appears, prose
|
| 95 |
+
# included (audit 2026-09-28 item 4).
|
| 96 |
+
FORBIDDEN_PANEL_PATTERNS: tuple[str, ...] = (
|
| 97 |
+
r"(?i)\b(?:claude|anthropic|gpt|chatgpt|openai|gemini|gemma|qwen|deepseek|llama|mistral|nemotron|sonnet|opus)[- ](?:position|seat|slot|reviewer|model)s?\b",
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
FORBIDDEN_EXFIL_PATTERNS: tuple[str, ...] = (
|
| 101 |
# Absolute filesystem paths likely pointing at our hosts
|
| 102 |
r"/home/orangepi\b",
|
|
|
|
| 383 |
(
|
| 384 |
"*Reviews at ICSAC are open and transparent. AI tooling helps "
|
| 385 |
"the panel draft and structure each review; final acceptance "
|
| 386 |
+
"decisions rest with human curators. Reviews are published "
|
| 387 |
"alongside acceptance for accountability; individual reviewer "
|
| 388 |
"identities are abstracted to keep focus on the assessment "
|
| 389 |
"rather than the tooling behind it.*"
|
|
|
|
| 460 |
(re.compile(r"\bslots\b"), "reviewers"),
|
| 461 |
(re.compile(r"\bSlot\b"), "Reviewer"),
|
| 462 |
(re.compile(r"\bslot\b"), "reviewer"),
|
| 463 |
+
# The RQC sometimes names a seat by the model that fills it ("the
|
| 464 |
+
# Claude-position reviewers"); the public record names seats, never
|
| 465 |
+
# vendors (2026-09-28, found on the first external paper).
|
| 466 |
+
(re.compile(r"(?i)\bclaude-position\b"), "first-seat"),
|
| 467 |
+
(re.compile(r"(?i)\b(?:gpt|chatgpt|openai|gemini|gemma|qwen|deepseek|llama|mistral|nemotron)-position\b"), "panel-seat"),
|
| 468 |
)
|
| 469 |
|
| 470 |
|
|
|
|
| 506 |
fatal = _find_substring_hits(structural, FORBIDDEN_VENDOR_TOKENS)
|
| 507 |
fatal.extend(_find_substring_hits(text, FORBIDDEN_SECRET_PHRASES))
|
| 508 |
fatal.extend(_find_regex_hits(text, FORBIDDEN_EXFIL_PATTERNS))
|
| 509 |
+
fatal.extend(_find_regex_hits(text, FORBIDDEN_PANEL_PATTERNS))
|
| 510 |
warn = _find_wordboundary_hits(text, SOFT_WARN_TOKENS)
|
| 511 |
return ScrubReport(fatal_hits=fatal, warn_hits=warn)
|
| 512 |
|
|
|
|
| 756 |
enforces this before the redaction writes anything to the site.
|
| 757 |
"""
|
| 758 |
status_line = (
|
| 759 |
+
"Review Quality Control: flagged β reviewed by human curators before acceptance."
|
| 760 |
if parsed.flag
|
| 761 |
else "Review Quality Control: passed."
|
| 762 |
)
|
|
|
|
| 789 |
lines.extend([
|
| 790 |
(
|
| 791 |
"The audit surfaced a concern outside the four scholarly "
|
| 792 |
+
"dimensions above. Human curators reviewed the panel output "
|
| 793 |
"before the acceptance decision was recorded."
|
| 794 |
),
|
| 795 |
"",
|
rubrics/review_quality_control.md
CHANGED
|
@@ -131,7 +131,7 @@ The model emits JSON with this exact shape. The editorial workflow serializes it
|
|
| 131 |
|
| 132 |
The public markdown/HTML pair at `src/data/public-reviews/<record_id>_review_quality_control.{md,html}` carries:
|
| 133 |
|
| 134 |
-
- A one-line status: `Review Quality Control: passed.` or `Review Quality Control: flagged β reviewed by human
|
| 135 |
- A short paragraph naming which scholarly dimensions were audited (rubric adherence, internal consistency, specificity, institutional voice) and the audit's purpose.
|
| 136 |
- A condensed per-slot table showing only the four scholarly dimensions, with positional reviewer labels.
|
| 137 |
- No `injection_indicators` column. No reference to prompt injection, security, adversarial content, or security architecture.
|
|
|
|
| 131 |
|
| 132 |
The public markdown/HTML pair at `src/data/public-reviews/<record_id>_review_quality_control.{md,html}` carries:
|
| 133 |
|
| 134 |
+
- A one-line status: `Review Quality Control: passed.` or `Review Quality Control: flagged β reviewed by human curators before acceptance.`
|
| 135 |
- A short paragraph naming which scholarly dimensions were audited (rubric adherence, internal consistency, specificity, institutional voice) and the audit's purpose.
|
| 136 |
- A condensed per-slot table showing only the four scholarly dimensions, with positional reviewer labels.
|
| 137 |
- No `injection_indicators` column. No reference to prompt injection, security, adversarial content, or security architecture.
|
templates/accept-comment.md
CHANGED
|
@@ -10,7 +10,7 @@ The landing page above carries:
|
|
| 10 |
- A direct citation reference
|
| 11 |
- A community signup link
|
| 12 |
|
| 13 |
-
ICSAC's review process is open and transparent. Reviews are published alongside acceptance for accountability; AI tooling helps the panel draft and structure each review while final acceptance decisions rest with human
|
| 14 |
|
| 15 |
Welcome to the community.
|
| 16 |
|
|
|
|
| 10 |
- A direct citation reference
|
| 11 |
- A community signup link
|
| 12 |
|
| 13 |
+
ICSAC's review process is open and transparent. Reviews are published alongside acceptance for accountability; AI tooling helps the panel draft and structure each review while final acceptance decisions rest with human curators.
|
| 14 |
|
| 15 |
Welcome to the community.
|
| 16 |
|
templates/accept.md
CHANGED
|
@@ -27,7 +27,7 @@ Or reference your inclusion in your own words:
|
|
| 27 |
|
| 28 |
## About the review
|
| 29 |
|
| 30 |
-
Your submission was evaluated through ICSAC's open review process β a multi-reviewer panel scoring domain fit, methodological transparency, internal consistency, citation integrity, novelty, and AI Provenance Signal β with human curator oversight. AI tooling helps the panel draft and structure each review; final acceptance decisions rest with human
|
| 31 |
|
| 32 |
## Rights and licensing
|
| 33 |
|
|
|
|
| 27 |
|
| 28 |
## About the review
|
| 29 |
|
| 30 |
+
Your submission was evaluated through ICSAC's open review process β a multi-reviewer panel scoring domain fit, methodological transparency, internal consistency, citation integrity, novelty, and AI Provenance Signal β with human curator oversight. AI tooling helps the panel draft and structure each review; final acceptance decisions rest with human curators. A copy of your review report is available upon request.
|
| 31 |
|
| 32 |
## Rights and licensing
|
| 33 |
|