For millions of people who are blind or have low vision, the Portable Document Format (PDF) is often not readable. The PDF is a digital document widely used in healthcare, government, business and education. It was developed by Adobe in the early 1990s because there was no universal way to read digital documents across different computers and programs without the document being distorted or looking broken. Adobe packed all the fonts, images, colors and more into one document “package” that only needed a free Acrobat Reader program to view.
The original PDF was not readable by people who used screen reader assistive technology programs. So, in the early 2000s, Adobe expanded the capability of PDFs so that they could be made accessible and readable by people who are blind or have low vision. However, this new accessible PDF was rare since the new format required additional skills and additional formatting work. The lack of accessible PDFs has continued to present time, with the vast majority not being accessible—up to 95% as reported in a Allyant’s recent PDF benchmark report.
While the PDF has become a global standard for sharing documents, it frequently serves as a digital dead end for those relying on screen-reading technology. Without proper internal formatting, the PDF cannot communicate with a screen reader about the different elements of the document. To be read by a screen reader, a PDF needs coded elements called tags to alert the reader of important information like headings, lists, charts, tables, and images. Without these tags also known as “semantic information,” a PDF is essentially a flat image and is unreadable by a screen reader program.
The impact is particularly heightened in Science, Technology, Engineering, and Mathematics (STEM) fields. In these disciplines, researchers predominantly use a typesetting system called LaTeX to produce complex layouts when publishing their research. “It is very hard to get an accessible PDF from a LaTeX authoring tool,” explained Aaron Steinfeld, a research professor at the Robotics Institute in the Carnegie Mellon School of Computer Science. Steinfeld noted that most STEM publications are done in LaTeX, yet “it does not preserve those (tags) when it exports it to PDF and so you get a PDF with no tags. And that’s, as you can imagine, very painful”.
Products & Services
The Architecture of the iTagPDF Solution
To bridge this gap, Peya Mowar, a PhD student at the Carnegie Mellon Robotics Institute, developed iTagPDF, an application designed to automate the tagging process for LaTeX-generated documents. Mowar, who previously worked at Microsoft Research India on making print newspapers accessible to people with visual impairments, recognized that the information needed for accessibility already exists within the author’s original code.
“Authors are essentially required to write the syntax in a way that all of those semantics are already asked by the author when they are creating their PDFs. But all that structure is lost when the PDF is rendered in a visual format,” Mowar stated. The iTagPDF application works by requiring a user to upload both the final PDF and the original LaTeX source files. It then uses generative artificial intelligence (AI), through its use of OpenAI, to re-insert the missing semantic information into the document.
Unlike traditional auto-tagging tools which rely primarily on the visual appearance of a page to guess its structure, iTagPDF uses the source code’s information as a reference for whatever it is labeling or tagging. “We are using a combination of both the visual and the semantics part,” Mowar noted. “We can identify things like tables and figures using the visual understanding that these models have and then we can add on to those semantics using the source document which the authors write”.
Products & Services
Redefining Accuracy through AI
The move toward an AI-driven solution was born out of necessity. Steinfeld and his colleague Jeff Bigham had spent over a decade attempting to work with the LaTeX community to build accessibility directly into the software engine, but they found little traction with the program’s creators. (LaTex is open source software and not owned by a single company.) As a result, they shifted their focus to repairing the document after its creation. “Jeff and his team have tried a few different things over the years but it has always been very painful and not particularly effective,” Steinfeld explained. “Peya had the clever idea to try and use AI to solve this problem”.
Evaluations suggest that Mowar’s approach significantly outperforms existing industry standards. In an assessment of 40 research papers, the iTagPDF system produced higher accuracy than Adobe Acrobat’s automated features. Mowar noted that the system “is also easier to use in the sense that you do not have to make a lot of manual corrections”. Steinfeld added that “the system outperforms some of these human authors” who are often required to tag documents manually for academic conferences.
While iTagPDF is currently in a beta phase and optimized only for LaTeX, the underlying principles could eventually be applied to other authoring tools. Steinfeld emphasized, “The key part is it has to be a tool where people are applying that semantic information instead of just formatting text with bold and font size.” Steinfeld provided the example of a Microsoft Word document, “You may have interacted with people who generate Word documents, and, instead of applying the Heading Style 1 for Heading 1, they just bolded and increased the font. So, you don’t know that that’s Heading Level 1. If you’re blind and you’re accessing the Word document, you don’t know that’s Heading Level 1 because they didn’t use the style to mark it that way.”
Although the iTagPDF system has been successful, using AI to make “flat” PDFs accessible without original source files remains a significant hurdle. While researchers are pushing toward more automated solutions, the current technology still relies heavily on the presence of underlying code to ensure 100% accuracy. To answer the question whether AI could currently transform a document with no source data into an accessible format, Mowar commented, “I think we are not there yet, but we are working towards [that]”.
Aaron Steinfeld further explained that without the source documents to provide semantic labels, the AI must guess the author’s intent based solely on visual cues, which is prone to error. While Steinfeld noted that a fully autonomous remediation of flat files is “theoretically possible,” both researchers agree that the field is still in the process of gathering the massive datasets required to train models for that level of inference. Mowar explained, “The next steps for iTagPDF could also be to collect this huge corpus of both the source documents and the visual rendered PDFs and train systems on a combination but then run inference on just the PDF. So, we could get to a point where the model could start learning from prior semantics of the papers which are published prior and then applying those for inference purposes.”
Products & Services
Opening to the World of STEM
The broader impact of this technology extends beyond convenience. For many students and professionals with disabilities, the lack of accessible academic materials has long been a barrier to entry in scientific fields. Steinfeld observed that colleagues who are blind or have low vision frequently have to request alternative copies of things or request help converting documents.
“There has been an under representation of people with disabilities in these fields,” Steinfeld said. “Some of this is also the lack of access to the articles and the academic knowledge because the PDFs are not accessible”. He believes that Mowar’s work “has the potential to improve access in these STEM fields for people who are blind”.
By automating the remediation of scholarly articles, iTagPDF aims to create a future where information is universal. As the system moves toward wider deployment, it offers a glimpse into a world where technology no longer creates barriers but actively dismantles them.
Resources:
