Avoid quadratic placeholder replacement in PDFObject::formatContent()#831
Avoid quadratic placeholder replacement in PDFObject::formatContent()#831bernemann wants to merge 2 commits into
Conversation
formatContent() restores every extracted string/dict placeholder with a full-content str_replace() in a loop, and does the extraction with a preg_quote()+preg_replace() per operand; both scan the whole content stream once per operand, so parsing is quadratic in the number of operands. Restore the placeholders in a single strtr() pass and use strpos()+substr_replace() for the initial extraction
|
Thank your for the PR. Could you provide some test code which demonstrates the statement:
Ideally the check code is located somewhere in https://github.com/smalot/pdfparser/tree/master/tests/Performance. We already have a bunch of performance tests. Could you also add at least one PHPUnit test which demonstrates that your new code works as expected? |
Add a performance regression test in tests/Performance and PHPUnit tests covering the changed formatContent()
b11b09b to
289ed83
Compare
|
Thanks for the review. I added tests for the Unit tests (PDFObjectTest.php)
Performance test (KernedTjArrayFormatContentTest.php) Builds a content stream with 20,000 kerned string operands and extracts its text. The old restoration ran one full-content Rough numbers:
I have a standalone script to reproduce the figures which I left out of the commit. I can provide it if requested. |
Type of pull request
About
This is a follow-up to the performance issue I opened in #712.
As described there, parsing gets slow on PDFs whose text is encoded as bracket-notation TJ arrays like [(Xxxx)1.7(a)1.2(tt )-10.1(\374)...], because every string operand becomes a placeholder in PDFObject::formatContent() and is then replaced back with its own str_replace() over the whole content stream. With many small operands this becomes quadratic and dominates the parsing time.
Instead of shortening the placeholder (which I first suggested, but which could collide with real stream content), I kept the uniqid() placeholders and only changed how they are handled:
First, the placeholders are now restored in a single strtr() pass instead of one str_replace() per placeholder. strtr() does all replacements in one traversal, so the cost no longer grows with the number of operands.
Second, the initial extraction now uses strpos() + substr_replace() instead of preg_quote() + preg_replace(), which avoids compiling a regex for every operand.
In my measurements this brings the affected PDFs down from about 8.3 seconds to about 2.1 seconds (roughly 3.5-4x), while documents that were already fast stay the same. The extracted text from getTextArray() is byte-identical before and after on all my sample files.
Thank you for your time!
Checklist for code / configuration changes
See CONTRIBUTING.md for all essential information about contributing.