EduEvidence: No Evidence, No New Study Designeduevidence：没有证据，就不设计新研究.

Someone asked: should we give students an AI coding assistant?

Ask a model directly and it’ll give you something that sounds smooth: efficiency gains, personalization, the future is here. The literature tells a different story — CHI 2023 has controlled studies at 1.15× / 1.8×, but also “no significant difference after a week”; PNAS 2025 measured −17% in unsupervised exams.

AI doesn’t lack search. It lacks evidence discipline.

What EduEvidence does is turn a “decision question” into an answer that is evidence-backed, pilotable, and evaluable — not a longer summary.

The README’s product line is: From Research Questions to Evidence-Based Decisions.

It ships as an AI Agent Skill; inside the Skill runs the EduEvidence Research Engine — persistent, auditable, compressing decision questions into evidence-grounded answers.

Three frozen principles:

1. Retrieved snippets are locators, not evidence.
   OpenAlex / Semantic Scholar / Sciverse give you leads; to become evidence, something must enter a traceable citation chain.

2. Without an evidence-grounded Knowledge Gap, no new study gets designed.
   A frozen scientific rule: study design must explicitly reference a grounded Gap ID.

3. result.json / HTML / Markdown are projections, not the fact store.
   The real fact layer is versioned, immutable revisions of the Evidence Graph.

The output isn’t a binary “allow / ban” either. It’s four states:

ADOPT / PILOT / REJECT / INSUFFICIENT EVIDENCE, plus an executable intervention and evaluation plan.

The Problem

Education decisions have three traps:

• Citations that look real: the year, journal, and conclusion all check out, but the details don’t survive scrutiny
• Correlation passed off as causation: “scores went up after adoption” is usually just selection bias
• Skipped applicability boundaries: who it works for, under what conditions it fails — often erased

Real studies like CHI 2023 and PNAS 2025 remind us: AI-assisted coding may speed up short tasks, but it can also drag in transfer and unsupervised settings. “Faster across the board” with no boundary conditions is unusable as decision input.

The Method

The engine splits the full research cycle open:

• Research Core: frame the question → retrieve → grade evidence → Gap → synthesize
• Decision Extension: option → pilot design → evaluate → update the decision

Three public workflows:

Multi-domain support works through contracts, not engine copies: education and policy each declare their frame schema, outcome taxonomy, and method list; unknown tokens fail closed.

How It Works

After install, hosts like Claude Code / Cursor / Codex / OMP can auto-load the Skill on “teaching decision” type questions.

npm install -g eduevidence
eduevidence skill --host claude

Native Core only depends on the Python standard library — no Agent MCP, no daemon. That’s deliberate: a research engine shouldn’t be welded to one orchestration framework.

Retrieval channels come in zero-config (OpenAlex / Semantic Scholar / CrossRef…) and keyed (Sciverse etc.). Sciverse is for citation-level locating; then things go into the Graph for structuring.

Results

• Benchmark simulation ≠ evidence. benchmarks/results/ is just a simulation harness; the first real empirical round is B2 vs B3 (10 questions × 3 repeats) — it can’t be written up as “dominates across the board.”
• Four states are harder to sell than two, but more useful. “Insufficient evidence” is a legitimate conclusion, not a failure.
• Projections can change; the fact layer can’t. Reports can be reordered; once a Graph revision lands, it shouldn’t be quietly rewritten.

Currently at 6.2.0. The Landing / Research Studio / Deep Research comparison pages all open directly. Sample report:

open examples/ai-coding-assistant-evidence/EduEvidence_Report.html

It won’t make education decisions for you. It only guarantees: before you say “go” or “no go” — where the evidence comes from, what the Gap is, how the pilot gets verified — all written down.

Main repo: 37chengshan/eduevidence.

FAQ

How is EduEvidence different from just asking an AI?

Ask an AI directly and you get something that sounds smooth. EduEvidence compresses a decision question into an answer that is evidence-backed, pilotable, and evaluable — ending in ADOPT, PILOT, REJECT, or INSUFFICIENT EVIDENCE, plus an executable intervention and evaluation plan.

Does a retrieved paper abstract count as evidence?

No. Retrieved snippets are only locators — leads. To become evidence, something must enter a traceable citation chain. That’s the first frozen principle.

Will it just tell me whether students should get an AI coding assistant?

It won’t make the decision for you. It only guarantees: before you say “go” or “no go” — where the evidence comes from, what the knowledge gap is, how the pilot gets verified — all written down.

Do I have to install Agent MCP to use it?

No. Native Core only depends on the Python standard library and isn’t welded to any single orchestration framework; hosts like Claude Code, Cursor, Codex, and OMP can all auto-load the Skill.

Is “insufficient evidence” a failure?

No. It’s a legitimate conclusion, not a failure. Four states are harder to sell than two, but more useful.