Logo image
Automated Digitization of Scientific Graphs for Retrieval-Augmented Language Model Analysis
Conference proceeding

Automated Digitization of Scientific Graphs for Retrieval-Augmented Language Model Analysis

Daniel Walczak, Montgomery Longtin, Kundan Nepal, Cheol-Hong Min, Hassan Salamy and Abe Kazemzadeh
2026 IEEE World AI IoT Congress (AIIoT), pp.0187-0193
05/20/2026

Abstract

Convolutional Neural Networks (CNNs) Graphical Analysis Indexes Indexing Labeling Large Language Models (LLMs) Modeling Object Detection Optical character recognition Optical Character Recognition (OCR) Pipelines Pixel Printing Retrieval augmented generation Retrieval-Augmented Generation (RAG) Signal detection You Only Look Once (YOLO)
This paper presents an automated system for scientific graph detection, classification, and data extraction from technical documents using a computer vision classification pipeline and optical character recognition (OCR). The system processes published research papers by detecting graph regions within PDF documents, isolating plots using an object detection model, and reconstructing numerical datasets from visual data points. Axis labels and tick values are extracted using OCR, while image processing techniques via LineFormer identify plot series data. Testing across 900 graphs achieved a Mean Average Precision (mAP) of 86.6% and a normalized average plot-value reconstruction error of 1.65%. The extracted datasets were applied, in conjunction with technical paper text documentation, through a retrieval-augmented large language model for further research insights. This process demonstrates reliable automated digitization of graphical data from scientific literature and supports the use of large language models for reasoning-based research recommendations.

Metrics

1 Record Views

Details

Logo image