Comments (5)
Hi @mikejokic. Thank you for flagging. Unfortunately, I can't seem to reproduce your findings. If anything, it runs slightly faster on 0.10.4
than 0.10.3
for me. Here's the exact code I'm running:
import pdfplumber
import time
import sys
start = time.time()
with pdfplumber.open(sys.stdin.buffer) as pdf:
for page in pdf.pages:
results = page.search(r'word', regex=True,return_chars=False)
if hasattr(page, "close"):
page.close()
else:
page.flush_cache()
end = time.time() - start
print(round(end, 3))
And then python test.py < documentation.pdf
. On 0.10.3
, I'm seeing times of around 7.9 seconds; on 0.10.4
, I'm seeing closer to 7.6 seconds.
If you run the same, what do you see?
from pdfplumber.
Thanks for the reply @jsvine. I ran your code block in Docker and I found similar results to yours. But I have been able to reproduce my issue with the provided pdf.
Here is code I have been able to run in Docker changing just the pdfplumber version number.
I look for a set of relevant keywords/regex patterns (repeated keywords for simplicity), and then take the surrounding line info as well. 0.10.3 runs in around 30-36seconds, and 0.10.4 takes around 90-96 seconds.
keywords = ['capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS''capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS','capabilities','BHRS', 'Intervention', 'assessment','IMD', 'Affidavit', 'subclass', 'IHBS']
import time
import pdfplumber
start = time.time()
with pdfplumber.open('documentation.pdf') as pdf:
for page in pdf.pages:
print(page,flush=True)
for key in keywords:
results = page.search(r'.*\b' + key + r'\b.*', regex=True,case=False,return_chars=False)
if hasattr(page, "close"):
page.close()
else:
page.flush_cache()
end = time.time() - start
print(round(end, 3))
from pdfplumber.
Big thanks, @mikejokic — that extra detail about looping through a bunch of .search(...)
calls per page helped me (a) reproduce your observation, (b) figure out what the problem was, and (c) fix it.
Turns out 0bfffc2 introduced a bug in which the page layout calculations (necessary for .search(...)
) were no longer getting cached. The fix in efca277 resolves that, restoring the prior speed/performance. Now available on the develop
branch and will be in the next release.
from pdfplumber.
Thanks @jsvine. Out of curiosity, does .search() run .extract_text() on each run or is the text also cached?
from pdfplumber.
.search(...)
uses the text-layout cache, which is based on the layout-dependent parameters you pass. E.g., if you run page.search("q1", x_tolerance=5)
and page.search("q2", x_tolerance=5)
, then the .extract_text(...)
is only run once, on the first search; but if you then call page.search("q2", x_tolerance=10)
, then .extract_text(...)
is called again.
from pdfplumber.
Related Issues (20)
- Any way to detect formatting? HOT 1
- Add `autodetect_direction` option to text-extraction methods
- Table extraction bug when lines are just barely end-to-end
- About paragraph recognition
- Custom deduppe_chars char properties HOT 4
- Got different result of "page.to_image()" on MacOS and Linux HOT 7
- TypeError: argument of type 'PDFObjRef' is not iterable HOT 5
- table_settings support draw line per page
- function "extract_words" extract words that don't exist in a pdf. HOT 2
- Extracting text from PDFs with encodings- Identity-H, Roman fails, gives a blank response. HOT 1
- Nothing founded from a pdf. No pages, no chars, nothing. HOT 3
- Text with imaginary lines is being treated as a table
- char need an attr linewidth HOT 4
- Concatenating cropped page objects HOT 1
- page.to_image() PDFium: Data format error HOT 3
- Demonstrations / Examples - links are not available HOT 2
- 逐个获取page.chars方法无法得到有些table里的chars,格式一模一样的table有的就不能获取
- page.img 好像没有图片的key,只有一些图片信息?怎么获取PDF中的图片对象
- When I set repair=true,there is an error:'utf-8' codec can't decode byte 0xae in position 239: invalid start byte.Because of the original PDF? HOT 1
- I got a bug when i parsing a pdf!!! HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pdfplumber.