Abstract
WildGraphBench evaluates GraphRAG performance in realistic scenarios using Wikipedia's structured content to assess multi-fact aggregation and summarization capabilities across diverse document types.
Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.
Community
hi
Good!
A Good Work worthy of attention, Starrrr!
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- SentGraph: Hierarchical Sentence Graph for Multi-hop Retrieval-Augmented Question Answering (2026)
- N2N-GQA: Noise-to-Narrative for Graph-Based Table-Text Question Answering Using LLMs (2026)
- Orion-RAG: Path-Aligned Hybrid Retrieval for Graphless Data (2026)
- A2RAG: Adaptive Agentic Graph Retrieval for Cost-Aware and Reliable Reasoning (2026)
- Graph-Anchored Knowledge Indexing for Retrieval-Augmented Generation (2026)
- A Navigational Approach for Comprehensive RAG via Traversal over Proposition Graphs (2026)
- Relink: Constructing Query-Driven Evidence Graph On-the-Fly for GraphRAG (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper