Make PDFs searchable

I searched for a good way to make scanned documents searchable. Most newer scanning software already has some OCR built-in, but what about all the old documents? Using pdfsandwich and Tesseract, we recover the text from each page of a PDF and put it behind each page as an invisible layer. That way, we can search the PDF with a normal PDF reader or upload it to Google translate to get a translated version. To get a text-only version, pdftotext can be used.

First, install the missing packages (tested on Ubuntu 12.04):

# we use tesseract-ocr-deu for German
apt-get install tesseract-ocr tesseract-ocr-deu poppler-utils
apt-get install exactimage imagemagick ghostscript

Second, we download and install pdfsandwich:

wget http://downloads.sourceforge.net/project/pdfsandwich/pdfsandwich%200.0.7/\
pdfsandwich_0.0.7_amd64.deb
dpkg -i pdfsandwich_0.0.7_amd64.deb

Finally, we run pdfsandwich and pdftotext on a PDF:

pdfsandwich -resolution 240x240 -rgb -lang deu german_document.pdf
# creates german_document_ocr.pdf with colors and 240dpi

pdftotext german_document_ocr.pdf
# gives german_document_ocr.txt

To process all PDFs in the current directory, find can be used:

find . -name "*.pdf" -exec pdfsandwich -resolution 240x240 -rgb -lang deu {} \;

Read More →

Create PDF invoices with HTML5 and PhantomJS

Creating invoices in PDF is always a bit tricky: there are many libraries to create PDF documents with PHP, but most can't handle complex layouts and require a lot of memory and CPU time. Things like Unicode characters and line/page breaks are often difficult to program and the source of many bugs and memory problems. Using logos as vector graphics or embedding TrueType fonts is often required, but not possible with most libraries.

The good thing is that all these features are included in HTML5. Since most companies offer their invoices also in HTML, it would be great to convert them directly to PDF. Tools like html2ps and ps2pdf can produce PDFs with 200+ pages without problems, but only support HTML4 and some very limited CSS. Also, the layout is never the same as in a web browser.

So we need a real web browser to convert HTML pages to PDFs. From testing web applications, we know that PhantomJS is a headless WebKit browser that runs on the commandline and can automate things very easily. The great thing is that it can also create screenshots and PDFs with selectable texts and embedded fonts! So we can use all kinds of CSS (Truetype fonts, round borders, etc.), SVG images, DIVs and tables to create great looking invoices.

Here is a PhantomJS script that converts an HTML file into a PDF:

// html2pdf.js
var page = new WebPage();
var system = require("system");
// change the paper size to letter, add some borders
// add a footer callback showing page numbers
page.paperSize = {
  format: "Letter",
  orientation: "portrait",
  margin: {left:"2.5cm", right:"2.5cm", top:"1cm", bottom:"1cm"},
  footer: {
    height: "0.9cm",
    contents: phantom.callback(function(pageNum, numPages) {
      return "<div style='text-align:center;'><small>" + pageNum +
        " / " + numPages + "</small></div>";
    })
  }
};
page.zoomFactor = 1.5;
// assume the file is local, so we don't handle status errors
page.open(system.args[1], function (status) {
  // export to target (can be PNG, JPG or PDF!)
  page.render(system.args[2]);
  phantom.exit();
});

Let's start the conversion:

time phantomjs html2pdf.js invoice.html invoice.pdf
real    0m0.189s
user    0m0.096s
sys     0m0.020s

time html2ps invoice.html | ps2pdf - invoice.pdf
...
real    0m3.521s
user    0m0.656s
sys     0m0.208s

(phantomjs 1.6.1, html2ps 1.0b7, 3.4 GHz single core, QEMU)

Here is the example invoice.html containing a SVG logo, tables, a TrueType font and CSS styles:

And it gets a perfect PDF with 2 pages in less than 1 second:

The size of the PDF is 65 KB. Using LibreOffice for the conversion gives 130 KB, html2ps produces 8 KB without the logo and the fonts.

The source files and the target PDF can be downloaded here.

Note: manual page breaks can be added with:

<div style="page-break-before:always;"></div>
or
<div style="page-break-after:always;"></div>

Note: Support for the WebOpenFontFormat (woff) will be available in future releases, so currently only TrueType fonts are supported.

Read More →