QUASAR: A Query-Oriented System for Multi-hop Knowledge Graph Data Summarisation

To quantify information loss when a knowledge graph is verbalized, summarized, and reconstructed, measuring surviving QA capability.

The project compares rule-based vs. neural verbalization and extractive vs. abstractive summarization, identifying reconstruction as the bottleneck.

Implications for knowledge graph storage, transfer, and augmentation via large language models.

Knowledge Graphs (KGs) store structured factual information as RDF triples, but their util- ity depends on preservation through NLP transformations. This thesis presents QUASAR (Query-oriented System for Multi-hop Knowledge Graph Data Summarisation), a systematic investigation into information loss when KG triples are verbalized into natural language text, optionally summarized, and then reconstructed back into a KG. We implement a complete six-stage round-trip evaluation pipeline on the Web QSP dataset over Freebase, testing 6 pipeline variants formed by combining 2 verbalization methods (rule-based templatesvs. neural Flan-T 5-XL) with 3 summarizationlevels(extractive Text Rank, abstractive Flan-T 5-XL, andnosummarization). Eachvariantproducesareconstructed KG viathe REBEL relation extraction model, which is then evaluated by executing the original 1,639 test SPARQL queries and comparing predicted answers against ground truth. Our key (cid:28) ndings are: (1) the reconstruction stage is the dominant information bottleneck, with REBEL recoveringonly∼12%oftriples(ℓ 5 ≈ 0.88);(2) rule-basedverbalizationconsistentlyout- performs neural across all downstream metrics (Answer F 1: 0.0295 vs. 0.0275 for best variants); (3) extractive summarization preserves 93% of recoverable information while abstractive loses 44%; (4) 2-hop queries involving CVT intermediate nodes are essentially unrecoverable across all approaches; and (5) even the identity baseline (original 810 K-triple KG) can only answer 28.8% of cross-split test queries, establishing a hard ceiling. The cumulative preservation through the best pipeline variant (nosumm_rb) is 7.77%, demon- strating that text remains a highly lossy compression format for structured knowledge. These (cid:28) ndingshaveimplicationsfor KG storage, transfer, andaugmentationvialargelanguagemodels.