How to remove Renderable Text from .PDF files to allow OCR

For all those people out there - students, academics, archivists, and eBooks readers - who have been stymied by Adobe® Acrobat's® stubborn refusal to perform optical character recognition (OCR) on a document, claiming: "Acrobat could not perform recognition (OCR) on this page because: This page contains renderable text." - I believe I have found a workable solution. Notice, I am not saying it is "The" solution. That would be for Adobe® to fix their software. I just think this is a workable solution which is much better than the "save to TIFF and rebuild from there" solution offered by Adobe®. Using this technique, it is possible to obtain a searchable and text-select-able document while preserving the original image of the scanned document, if desired.

Basics:

1. Print the "malfunctioning" .PDF file to the "Microsoft XPS Document Writer" printer driver (which you will need to install).
2. Convert the resulting .XPS file to an Acrobat® .PDF file.
3. Perform OCR in Acrobat® using one of the three available output styles depending on the type of document you have and the results you want.

Preliminary Notes:

• You need the full (or "Pro") version of Adobe® Acrobat® to complete this procedure. However, as this same program is required to perform OCR from within Acrobat®, and anyone reading this is doing so because they normally would have been able to do the OCR but can't for some some specific document(s), I assume the reader has access to this "Pro" version of Adobe® Acrobat®; henceforth to be referred to simply as "Acrobat®." I use Acrobat 9 Pro®, but these procedures will likely work on any relatively recent version of the product.
• This trick can only be done on Windows® computers, but the resulting file can then be used anywhere.
• Although this trick does not require a lot of tedious manual labor, it does take up a lot of computer time and processing power.
• I recommend testing these procedures out on individual - extracted - pages of your document, both to ensure you understand the process and to allow you to quickly try different variations so you can decide which result you like best.
  • To extract a single page in Acrobat®:
    1. Open the thumbnail pane.
    2. Select a sample page.
    3. Right-click and choose "Extract Pages" and follow the prompts.
       • (Name the files appropriately so you can better judge the results of your experiments.)
  • (You may want to choose three different pages - text only, line drawing or graphics heavy, and photographic image heavy - to experiment around with.)
• This process generates some really large transitional files. Your final files are likely to be somewhat larger than the original file, depending on the original document and which OCR output style you choose. However, they will also be a lot more useful.

Full Procedure:

Install the XPS printer driver if you don't already have it on your computer:

XPS is Microsoft's® answer to the Adobe® Acrobat® file format. It stands for "XML Paper Specification," following Microsoft's® habit of using generic naming for their products, as if they were the only product of their type in existence. From what I have read, it is supposedly similar to Acrobat® except that everything is in XML and can therefore be read by humans. It also makes for some extraneously large files. Fortunately we don't have to leave our files in this format. It is merely used as a transitional format, the conversion to which, strips out the bothersome "renderable text."

1. Download the XPS printer driver here: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=b8dcffdd-e3a5-44cc-8021-7649fd37ffee&displaylang=en.
2. Save the file where you can find it then double-click it to start the install. Follow the prompts to complete the install.
   • This will create a new printer in your "Printers and Faxes" folder. To print to it, you simply choose that printer instead of your regular printer when you print a document.

Print the .PDF file to the .XPS "printer.":

1. Open the file in question using the latest version of Acrobat Reader and follow these GCGUINS instructions: { File / Print ; Printer, Name = Mocrpsoft XPS Document Writer[v] ; [Properties] ; <Layout> ; [Advanced] ; Microsoft XPS .../Document Options/ Interleaving: = Off...[v] ; Images: = PNG-Lossless compression[v] ; [OK] ; [OK] ; Page Handling, Page Scaling: = None[v] ; [ ] Auto-Rotate... ; [OK] } to "print" it to the "Microsoft XPS Document Writer" printer driver just as you would when "printing" to an Acrobat® .PDF file. The printer driver will open up a "File Save" dialog asking where to save the .XPS file.

This could take quite some time depending on how much "rendered text" (i.e. selectable text) is in the document. Text that is actually only an image should convert rather quickly because this process seems to simply move the image portions of the documents straight over without any conversion or alteration whatsoever. Though I am not positive, the little bit of poking around in the document I did, causes me to speculate that the .XPS printer driver converts each and every character in the document into a vector graphic, similar to an Adobe postscript file. As you can imagine, this makes for an incredibly large file (see the table below) and it takes a really long time. I would suggest you start this process and then go off to a long lunch or meeting. If you have a separate computer on which you can run these processes, more's the better.

Convert the .XPS file back into a .PDF file.

Now this step is really going to take a long time, perhaps hours. If you have a large document with lots of "rendered text," I recommend that you start the process before going to bed or before leaving the office for the night. In addition, once you have started this process, it will look as if your computer isn't doing anything at all for almost the entire time. This is because Acrobat® does not display any user interface until it has completed the conversion and has a .PDF document to show.

1. Right click on the file and choose the appropriate context menu option:
   • Some installations of Acrobat® place an item in the Windows® file explorer context menu (pops up when you right-click on a file) that says "Combine supported files in Acrobat..." when you right-click on any file that Acrobat® knows how to convert to .PDF format. If you see this option in your context menu when you right-click on a .XPS file then choose it because this gives you the most control. (Yes, it works even though you only selected one file.)
     • In the 'Combine Files' dialog, in the lower right corner: Choose the largest document icon to choose the largest file size, and click [Combine Files].
   • If the above option is not available look for 'Convert to Adobe PDF.' This function will not open any dialog or the Acrobat Pro window until the file has been completely converted. It will look as if your computer is either not doing anything or is locked up. Don't reboot like I did the first few times interrupting the process. Just be patient.
   • If you don't see either of the above options then - from the context menu - choose: { Open With / Adobe Acrobat x } or choose { Open With / Choose Program... } and select Adobe Acrobat® from the list. Be sure not to select Acrobat Reader®.
     • (I wouldn't recommend selecting the "Always use the selected program to open this kind of file" option because you only want to open .XPS files in Acrobat when you really want to convert them to .PDF format. If you just want to view the file quickly, you really should just use the XPS viewer. It is a lot faster.)
2. (Optional) If you had to use either of the last two options above then you may want to double check that things have actually started processing. As I stated earlier, Acrobat® may not display anything for hours. The best way to check on this is to use the Windows® Task Manager.
   1. Right-click in the task bar and choose "Task Manager" (XP) or "Start Task Manager" (Windows 7).
   2. Select the "Processes" tab and look for "acrobat.exe." (If you click the "CPU" column header twice (not double-click) then acrobat.exe should be at about the top of the list.) The acrobat.exe process should be using about 50% of your CPU time.
   3. Now look in the "Memory ..." column (normally the fourth one). acrobat.exe should be using up ever increasing amounts of memory.
      • Believe it or not, that is how you know Acrobat® is processing your file. Essentially, Acrobat® is building up a complete .PDF file in memory before displaying it to you. Considering that the .XPS file has a separate vector graphic for each separate character in the file, that is a lot of data. And, until you do the OCR, all that data is in the .PDF file too.
3. Go to bed and get some sleep. Research shows this is very important to your overall productivity and health.
4. Save the file.
   • Acrobat® does not generate a file on disk. It was only generated in memory. You must save the file to disk yourself. Choose an appropriate file name and DO NOT overwrite your original file.

I do have to admit that this conversion does seem to produce slightly blurier images for scanned documents. It appears that either Acrobat or the XPS driver does a little bit of antialiasing of the jagged edges.

Perform the OCR:

Most people who have used Acrobat® to do OCR know there are three different output styles to choose from: Searchable Image, Searchable Image (exact), and ClearScan. Which you choose depends on the original document and the intended use for the final document.

1. To select the desired output style and start the OCR process: in Acrobat®: { Document / OCR Text Recognition / Recognize Text using OCR ; Settings, [Edit] ; PDF Output Style = your chosen method as elaborated below [v] ; Downsample Images = Lowest (600 dpi) [v] ; [OK] ; [OK] }
   • (These instructions are in GCGUINS format for concision.)
2. Save the file using yet another file name.
   • Until you are completely satisfied with the results, you should not delete or overwrite any of these files.

Mostly-Image (scanned) Documents:

Most academics will be dealing with scanned documents, where the "document" is actually just a series of images of pages stored in the .PDF file. These become a problem for OCR when the scanning software did not already do the OCR but did insert some computer printed ("rendered") text, thus causing Acrobat® to choke and show the dreaded "... could not perform recognition (OCR) ..." error dialog.

Now, said academic may want to preserve the original image of the document for possible scrutinizing or grabbing snapshots from in the future. In which case, said academic should choose the "Searchable Image (exact)" OCR output style. Acrobat® recognizes the text but hides the recognized text behind the image, which it does not disturb at all. This produces a pretty large file. However, if the file was really just a series of images to begin with, then the resulting file may not be much larger than the original.

On the other hand, our imaginary academic may want to produce the smallest possible file size, or may have hopes of producing a file that is easier to read than the scanned original. In this case he or she should choose the ClearScan OCR output style. This causes Acrobat® to replace the image file with a set of custom-generated fonts, designed to look as close as possible to the original fonts, but with clean edges instead of blurry, scanned edges. It is easier to read, but, if Acrobat® guessed wrong for some words while doing its OCR magic, then all you are left with is the bad guess. It also sometimes completely gives up and just places a small image of the word - or just a couple of letters - in the spot where those letters should have gone. It is acceptably readable but it looks weird and those words or letters aren't selectable.

The plain "Searchable Image" output style is a decent middle of the road option, but it does modify the look of the page images because they are compressed. You should experiment to make sure you can tolerate the results.

Mostly-Text Documents:

Some of the documents that cause the "renderable text" error look as if they were generated by a computer ("born digital," as some are saying these days) but either some of the text is not selectable or it is selectable but the copied text is gibberish. Many people suspect this is meant to prevent people from copying any of the document for use elsewhere. It also makes the document practically useless for any academic or business purpose. For these kinds of documents, the .XPS file can be ginormous; ten to twenty times the size of the original .PDF file.

The "Searchable Image (exact)" output style does produce the best looking result - the final document looks exactly like the original - but the final .PDF file size is only slightly less ginormous than the .XPS file. This is because all the vector images of all the individual characters in the document are retained when using this OCR output style. While that isn't a problem for a mostly-image (scanned) document because there is a relatively small amount of "rendered text," it is a nightmare for mostly-text documents because of the vast quantity of individual vectors they contain. So, only use the "Searchable Image (exact)" output style if the document also contains images which you absolutely must retain in their original quality. If the most important images are on separate pages from the text then one could selectively OCR only the pages with text using the ClearScan output style.

I do not recommend the plain "Searchable Image" output style because it produces really poor quality character renderings. It is readable and selectable but it is much more difficult to read than documents produced using either the "Searchable Image (exact)" or the "ClearScan"output style.

The ClearScan output style results in very nice looking text as well as files that are usually less than twice the size of the original, sometimes even smaller than the original. However, the images within the document may not look as good as the originals. Again, some selective OCRing may produce a more optimum result, but that requires more manual labor, which we are trying to avoid.

Comparison Chart:

I have performed this conversion on three different types of pages taken from a mostly-text document: a page with all text, one with some text and a single B&W photograph, and one with some text but also some line drawings. The chart below shows the resulting file sizes. If there is nothing in a cell, that means I didn't think it was worth trying that conversion.

Mostly Text - File Size Comparison

	Text	Photo	Drawings
Original File:	44k	273k	109k
.XPS File:	767k	336k	381k
Converted, Pre-OCR:	611k	290k	323k
Searchable Image:	397k
Searchable Image (exact):	597k	287k	334k
ClearScan:	44k	89k	127k

 

As you can see, the results vary dramatically. Note, however, that pages with the most text produced the greatest increase in size when printing to the .XPS file. When I processed a 350-page, mostly-text, 10MB document: the XPS file was 175MB, and the resulting document came out to 15MB using the ClearScan OCR method.

I haven't performed similar tests on mostly-image documents at this time. Perhaps I will do so later. Such is the luxury of doing all this only for my own edification and sharing the information completely free (without any ads even).

Hopefully, this article will be a big help for: A) all those students out there trying to OCR all those papers they have collected in their research so they can pull quotes out of them without retyping everything, as well as B) those archivists out there who are trying to make the documents in their collections searchable. Though I have not done so, it should also be possible to write some kind of script that would completely automate this process for batch-processing lots of files at the same time. If this helps you, please let me know. If you have any questions or suggestions, please don't hesitate to contact me.

---

How to remove Renderable Text from .PDF files to allow OCR by Grant Sheridan Robertson is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available here.