Giter VIP home page Giter VIP logo

unpdf's Introduction

unpdf

A collection of utilities to work with PDFs. Designed specifically for Deno, workers and other nodeless environments.

unpdf ships with a serverless build/redistribution of Mozilla's PDF.js for serverless environments. Apart from some string replacements and mocks, unenv does the heavy lifting by converting Node.js specific code to be platform-agnostic. See pdfjs.rollup.config.ts for all the details.

This library is also intended as a modern alternative to the unmaintained but still popular pdf-parse.

Features

  • ๐Ÿ—๏ธ Works in Node.js, browser and workers
  • ๐Ÿชญ Includes serverless build of PDF.js (unpdf/pdfjs)
  • ๐Ÿ’ฌ Extract text and images from PDFs
  • ๐Ÿงฑ Opt-in to legacy PDF.js build
  • ๐Ÿ’จ Zero dependencies

PDF.js Compatibility

The serverless build of PDF.js provided by unpdf is based on PDF.js v4.3.136. If you need a different version, you can use another PDF.js build.

Installation

Run the following command to add unpdf to your project.

# pnpm
pnpm add unpdf

# npm
npm install unpdf

# yarn
yarn add unpdf

Usage

Extract Text From PDF

import { extractText, getDocumentProxy } from "unpdf";

// Fetch a PDF file from the web
const buffer = await fetch(
  "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
).then((res) => res.arrayBuffer());

// Or load it from the filesystem
const buffer = await readFile("./dummy.pdf");

// Load PDF from buffer
const pdf = await getDocumentProxy(new Uint8Array(buffer));
// Extract text from PDF
const { totalPages, text } = await extractText(pdf, { mergePages: true });

Access the PDF.js API

This will return the resolved PDF.js module and gives full access to the PDF.js API, like:

  • getDocument
  • version
  • โ€ฆ and all other methods

Especially useful for platforms like ๐Ÿฆ• Deno or if you want to use the PDF.js API directly. If no custom build was defined beforehand, the serverless build bundled with unpdf will be initialized.

import { getResolvedPDFJS } from "unpdf";

const { getDocument } = await getResolvedPDFJS();
const data = Deno.readFileSync("dummy.pdf");
const doc = await getDocument(data).promise;

console.log(await doc.getMetadata());

for (let i = 1; i <= doc.numPages; i++) {
  const page = await doc.getPage(i);
  const textContent = await page.getTextContent();
  const contents = textContent.items.map((item) => item.str).join(" ");
  console.log(contents);
}

Use Official or Legacy PDF.js Build

Generally speaking, you don't need to worry about the PDF.js build. unpdf ships with a serverless build of the latest PDF.js version. However, if you want to use the official PDF.js version or the legacy build, you can define a custom PDF.js module.

Warning

The latest PDF.js v4.3.136 uses Promise.withResolvers, which may not be supported in all environments, such as Node < 22. Consider to use the bundled serverless build, which includes a polyfill, or use an older version of PDF.js.

// Before using any other method, define the PDF.js module
// if you need another PDF.js build
import { configureUnPDF } from "unpdf";

await configureUnPDF({
  // Use the official PDF.js build (make sure to install it first)
  pdfjs: () => import("pdfjs-dist"),
});

// Now, you can use the other methods
// โ€ฆ

Config

interface UnPDFConfiguration {
  /**
   * By default, UnPDF will use the latest version of PDF.js compiled for
   * serverless environments. If you want to use a different version, you can
   * provide a custom resolver function.
   *
   * @example
   * // Use the official PDF.js build (make sure to install it first)
   * () => import('pdfjs-dist')
   */
  pdfjs?: () => Promise<PDFJS>;
}

Methods

configureUnPDF

Define a custom PDF.js module, like the legacy build. Make sure to call this method before using any other methods.

function configureUnPDF(config: UnPDFConfiguration): Promise<void>;

getResolvedPDFJS

Returns the resolved PDF.js module. If no build is defined, the latest version will be initialized.

function getResolvedPDFJS(): Promise<PDFJS>;

getMeta

function getMeta(
  data: DocumentInitParameters["data"] | PDFDocumentProxy,
): Promise<{
  info: Record<string, any>;
  metadata: Record<string, any>;
}>;

extractText

Extracts all text from a PDF. If mergePages is set to true, the text of all pages will be merged into a single string. Otherwise, an array of strings for each page will be returned.

function extractText(
  data: DocumentInitParameters["data"] | PDFDocumentProxy,
  { mergePages }?: { mergePages?: boolean },
): Promise<{
  totalPages: number;
  text: string | string[];
}>;

renderPageAsImage

Note

This method will only work in Node.js and browser environments.

To render a PDF page as an image, you can use the renderPageAsImage method. This method will return an ArrayBuffer of the rendered image.

In order to use this method, you have to meet the following requirements:

  • Use the official PDF.js build
  • Install the canvas package in Node.js environments

Example

import { configureUnPDF, renderPageAsImage } from "unpdf";

await configureUnPDF({
  // Use the official PDF.js build
  pdfjs: () => import("pdfjs-dist"),
});

const pdf = await readFile("./dummy.pdf");
const buffer = new Uint8Array(pdf);
const pageNumber = 1;

const result = await renderPageAsImage(buffer, pageNumber, {
  canvas: () => import("canvas"),
});
await writeFile("dummy-page-1.png", Buffer.from(result));

Type Declaration

declare function renderPageAsImage(
  data: DocumentInitParameters["data"],
  pageNumber: number,
  options?: {
    canvas?: () => Promise<typeof import("canvas")>;
    /** @default 1 */
    scale?: number;
    width?: number;
    height?: number;
  },
): Promise<ArrayBuffer>;

FAQ

Why Is canvas An Optional Dependency?

The official PDF.js library depends on the canvas module for Node.js environments, which doesn't work inside worker threads. That's why unpdf ships with a serverless build of PDF.js that mocks the canvas module.

However, to render PDF pages as images in Node.js environments, you need to install the canvas module. That's why it is a peer dependency.

License

MIT License ยฉ 2023-PRESENT Johann Schopplich

unpdf's People

Contributors

eltigerchino avatar himself65 avatar johannschopplich avatar moinulmoin avatar pserrer1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

unpdf's Issues

Struggling to get it to work in a Supabase Edge Function

Environment

Using via esm: https://esm.sh/[email protected]/
Deno version: 1.38.4 (I'm guessing this because it's not easy to see the supabase edge function environment, but they say they use the latest stable version.

Reproduction

Sorry, it's a bit hard to reproduce since it's only failing in the deployed Supabase edge function. When I run it locally in my docker container, it works fine. Here's the relevant code though of my edge function:

import { configureUnPDF, getResolvedPDFJS } from 'https://esm.sh/[email protected]';
import * as pdfjs from 'https://esm.sh/[email protected]/dist/pdfjs.mjs';

configureUnPDF({
  // deno-lint-ignore require-await
  pdfjs: async () => pdfjs,
});
const resolvedPdfJs = await getResolvedPDFJS();
const { getDocument } = resolvedPdfJs;

export async function convertPdfToText(
  arrayBuffer: ArrayBuffer
): Promise<string> {
  try {
    const data = new Uint8Array(arrayBuffer);

    // Get the document
    const doc = await getDocument(data).promise;
    let allText = '';

    // Iterate through each page of the document
    for (let i = 1; i <= doc.numPages; i++) {
      const page = await doc.getPage(i);
      const textContent = await page.getTextContent();

      // Combine the text items with a space (adjust as needed)
      const pageText = textContent.items
        .map((item) => {
          if ('str' in item) {
            return item.str;
          }
          return '';
        })
        .join(' ');
      allText += pageText + '\n'; // Add a newline after each page's text
    }

    return allText;
  } catch (error) {
    console.error('Error converting PDF to text', error);
    throw error;
  }
}

Describe the bug

In the supabase edge functions log, it consistently throws this error:

event loop error: Error: PDF.js is not available. Please add the package as a dependency.
    at f (https://esm.sh/v135/[email protected]/deno/unpdf.mjs:2:574)
    at async h (https://esm.sh/v135/[email protected]/deno/unpdf.mjs:2:230)
    at async file:///home/runner/work/tl.ai/tl.ai/supabase/functions/process/index.ts:12:23

Originally, I followed the base setup instructions. Then, I tried to use getResolvedPDFJS. Finally, I tried to first configureUnPDF and pointing pdfjs specifically to the one exported from your package. However, all still failed in the production environment.

I'm mainly wondering if I'm not following the instructions correctly for configuring pdfjs. Thanks in advance for your help!

Additional context

No response

Logs

No response

Resolving fails with latest pdfjs-dist version

Environment


  • Operating System: Linux
  • Node Version: v20.12.2
  • Nuxt Version: -
  • CLI Version: 3.12.0
  • Nitro Version: 2.9.6
  • Package Manager: [email protected]
  • Builder: -
  • User Config: -
  • Runtime Modules: -
  • Build Modules: -

Reproduction

I set up a nitro project:

https://github.com/mwohlan/undpdf-issue

Describe the bug

This seems to lead to an error, when using the latest version of pdfjs-dist

  await configureUnPDF({
    // Use the official PDF.js build
    pdfjs: () => import('pdfjs-dist'),
  });

giving the following error message: Resolving failed. Please check the provided configuration.

It works again when downgrading the pdfjs-dist version to for example 4.0.379

Additional context

No response

Logs

No response

Missing `pdfjs-dist` types

Environment

  • unpdf v0.10.1
  • node v18.19.0

Reproduction

The types should be exported here so we can use them.

import * as PDFJS from './types/src/pdf'
declare function resolvePDFJS(): Promise<typeof PDFJS>
export { resolvePDFJS }

Hence, the currently generated declaration file looks like this:

import * as PDFJS from './types/src/pdf'
declare function resolvePDFJS(): Promise<typeof PDFJS>
export { resolvePDFJS } // no types are included

unpdf/pdfjs type not exported error

Describe the bug

The types are not exported together with unpdf/pdfjs. This prevents typing variables / function params when composing with the library.

Additional context

No response

Logs

No response

renderPageAsImage missing fonts

Describe the feature

Depending on the fonts used for PDFs, some fonts won't render at all when converted to an image.

To fix this issue we can disable the FontFace option and provide a standardFontDataUrl (through getDocument or getDocumentProxy options object) pointing to the standard_fonts folder.

Here is a working fix, since I might not be the only one facing this issue:

import { Buffer } from 'node:buffer'
import { dirname, resolve } from 'pathe'
import { configureUnPDF, getDocumentProxy,, renderPageAsImage } from 'unpdf'
import type { TypedArray } from 'pdfjs-dist/types/src/display/api'

export async function convertPdfToImg(buffer: ArrayBuffer | TypedArray, width) {
  try {
    await configureUnPDF({
      // Use the official PDF.js build
      pdfjs: () => import('pdfjs-dist'),
    })

    const packagePath = dirname(resolve('node_modules/pdfjs-dist/package.json'))

    const pdf = await getDocumentProxy(buffer, {
      isEvalSupported: false,
      useSystemFonts: false,
      disableFontFace: true,
      standardFontDataUrl: `${packagePath}/standard_fonts/`,

    })


    const pagenumber = 1
   
    const result = await renderPageAsImage(pdf, pageNumber, {
        canvas: () => import('canvas'),
        width,
   })

     

    return result
  }
  catch (error) {
    console.error('Error converting PDF to images:', error)
    throw new Error(`Failed to convert PDF to images: ${error.message}`)
  }
}

I don't know if pdfjs-dist has to be installed separately for this to work. Maybe @johannschopplich can consider integrating this fix.

Additional information

  • Would you be willing to help implement this feature?

Does not work in BGSW using Plasmo framework

Environment

Framework: Plasmo 0.84.0
Client side Chrome Browser Extension

Reproduction

Can be reproduced by creating a BGSW in the Plasmo framework and importing unpdf. Error message is:

๐Ÿ”ด ERROR | Build failed. To debug, run plasmo dev --verbose.
๐Ÿ”ด ERROR | Failed to resolve 'unpdf/pdfjs' from './node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/index.mjs'

Describe the bug

Bug is as aforementioned: unpdf seems to be looking for a pdfjs dependency that is inaccessible.

Additional context

No response

Logs

No response

PDF Generator

Describe the feature

It would be nice addon a PDF Generator (something like adding pdf-lib )

Additional information

  • Would you be willing to help implement this feature?

Mention that `dom` must be added to tsconfig `compilerOptions`

Describe the change

It should be mentioned in the documentation that dom must be added to tsconfig.

{
    "compilerOptions": {
        "lib": [
            "es2022",
            "dom" // ๐Ÿ‘ˆ Should be be added
        ]
    }
}

Otherwise type checking would fail, i.e tsc --noemit will give:

node_modules/unpdf/dist/index.d.ts:33:60 - error TS2304: Cannot find name 'HTMLCanvasElement'.

33     _createCanvas(width: number, height: number): Canvas | HTMLCanvasElement;
                                                              ~~~~~~~~~~~~~~~~~

node_modules/unpdf/dist/index.d.ts:35:26 - error TS2304: Cannot find name 'HTMLCanvasElement'.

35         canvas: Canvas | HTMLCanvasElement;
                            ~~~~~~~~~~~~~~~~~

.....

Found 81 errors in 17 files.

Errors  Files
     5  node_modules/unpdf/dist/index.d.ts:33
    10  node_modules/unpdf/dist/types/src/display/annotation_layer.d.ts:9
     9  node_modules/unpdf/dist/types/src/display/api.d.ts:172
     5  node_modules/unpdf/dist/types/src/display/display_utils.d.ts:61
     6  node_modules/unpdf/dist/types/src/display/editor/annotation_editor_layer.d.ts:9
     2  node_modules/unpdf/dist/types/src/display/editor/color_picker.d.ts:7
    10  node_modules/unpdf/dist/types/src/display/editor/editor.d.ts:72
     4  node_modules/unpdf/dist/types/src/display/editor/freetext.d.ts:43
     7  node_modules/unpdf/dist/types/src/display/editor/ink.d.ts:21
     1  node_modules/unpdf/dist/types/src/display/editor/toolbar.d.ts:4
     6  node_modules/unpdf/dist/types/src/display/editor/tools.d.ts:59
     4  node_modules/unpdf/dist/types/src/display/text_layer.d.ts:9
     2  node_modules/unpdf/dist/types/src/display/worker_options.d.ts:8
     2  node_modules/unpdf/dist/types/src/display/xfa_layer.d.ts:6
     1  node_modules/unpdf/dist/types/src/shared/message_handler.d.ts:40
     2  node_modules/unpdf/dist/types/web/interfaces.d.ts:62
     5  node_modules/unpdf/dist/types/web/text_accessibility.d.ts:17

URLs

No response

Additional information

  • Would you be willing to help?

Cannot find module '../build/Release/canvas.node'

Environment

System:
OS: macOS 13.3.1
CPU: (12) arm64 Apple M2 Pro
Memory: 89.58 MB / 16.00 GB
Shell: 5.9 - /bin/zsh

Reproduction

Using the exact code snippet from the README for renderPageAsImage

Describe the bug

node:internal/modules/cjs/loader:1075
  const err = new Error(message);
              ^

Error: Cannot find module '../build/Release/canvas.node'
Require stack:
- /Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/lib/bindings.js
- /Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/lib/canvas.js
- /Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/index.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
    at a._resolveFilename (/Users/powella/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/tsx/dist/cjs/index.cjs:1:1729)
    at Module._load (node:internal/modules/cjs/loader:920:27)
    at Module.require (node:internal/modules/cjs/loader:1141:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at Object.<anonymous> (/Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/lib/bindings.js:3:18)
    at Module._compile (node:internal/modules/cjs/loader:1254:14)
    at Object.S (/Users/powella/Library/pnpm/global/5/.pnpm/[email protected]/node_modules/tsx/dist/cjs/index.cjs:1:1292)
    at Module.load (node:internal/modules/cjs/loader:1117:32)
    at Module._load (node:internal/modules/cjs/loader:958:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/lib/bindings.js',
    '/Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/lib/canvas.js',
    '/Users/code/node_modules/.pnpm/[email protected]/node_modules/canvas/index.js'
  ]
}

Node.js v18.15.0

Additional context

No response

Logs

No response

Unpdf can't render pages with images

Environment

Node.js v20.9.0
PNPM v8.10.0
UnPDF v0.10.0

Reproduction

Example Code: CodeSandBox

Describe the bug

When I am ready to render a page with unpdf, renderPageAsImage will report an error if there is an image embedded in the pdf.

Additional context

No response

Logs

TypeError: r.createCanvas is not a function
    at NodeCanvasFactory._createCanvas (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1316062)
    at NodeCanvasFactory.create (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1153693)
    at CachedCanvases.getCanvas (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1163909)
    at CanvasGraphics.paintInlineImageXObject (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1198408)
    at CanvasGraphics.paintImageXObject (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1197232)
    at CanvasGraphics.executeOperatorList (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1172983)
    at InternalRenderTask._next (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1152632)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.