Giter VIP home page Giter VIP logo

puppeteer-lambda-starter-kit's Introduction

Puppeteer Lambda Starter Kit

Starter Kit for running Headless-Chrome by Puppeteer on AWS Lambda.

It can use alone and also supported Serverless Framework.

Download

Use alone

This is simple and don't need IAM role but you have to deploy package by yourself. Don't worry, even if you will become to use Serverless in the future, what you should do for migration is little.

$ git clone -o starter-kit https://github.com/sambaiz/puppeteer-lambda-starter-kit.git your_project_name

Use with Serverless Framework

Serverless Framework can manage settings with CloudFormation and deploy.

$ serverless install --url https://github.com/sambaiz/puppeteer-lambda-starter-kit --name your_project_name

Run on local

By executing SLOWMO_MS=250 npm run local, you can check the operation while actually viewing the chrome (non-headless, slowmo).

Packaging & Deploy

Lambda's memory needs to be set to at least 384 MB, but the more memory, the better the performance of any operations.

512MB -> goto(youtube): 6.481s
1536MB -> goto(youtube): 2.154s

chrome in package (recommended)

If you use alone, run npm run package, and deploy the package.zip.

If you use with Serverless, run serverless deploy (this runs npm run package when packaging).

chrome NOT in package

Due to the large size of Chrome, it may exceed the Lambda package size limit (50MB) depending on the other module to include. In that case, put Chrome in S3 and download it at container startup so startup time will be longer.

Run npm run package-nochrome, deploy the package.zip, and set following env valiables on Lambda.

  • CHROME_BUCKET(required): S3 bucket where Chrome is put
  • CHROME_KEY(optional): S3 key. default: headless_shell.tar.gz

Build Headless-Chrome (optional)

This kit includes Chrome built by myself because official build Chrome installed by Puppeteer has problems about running on Lambda (missing shared library etc.).

If you want to use latest chrome, run chrome/buildChrome.sh on EC2 having at least 16GB memory and 30GB volume. See also serverless-chrome. Once you build it, link to headless_shell.tar.gz in chrome dir.

Article

Lambda上でPuppeteer/Headless Chromeを動かすStarter Kitを作った - sambaiz-net

puppeteer-lambda-starter-kit's People

Contributors

joshbeckman avatar sambaiz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppeteer-lambda-starter-kit's Issues

Compression Algorithms

In a effort to reduce the file size of the bundled headless_shell I was testing different algorithms:

File                Algorithm   Level   Bytes       MiB         %           Decompression
-----------------------------------------------------------------------------------------
headless_shell      -           -       136964856   130.62      -           -
headless_shell.gz   Gzip        1       51662087    49.27       62.28%      1.035s
headless_shell.gz   Gzip        2       50438352    48.10       63.17%      1.016s
headless_shell.gz   Gzip        3       49428459    47.14       63.91%      0.968s
headless_shell.gz   Gzip        4       47873978    45.66       65.05%      0.950s
headless_shell.gz   Gzip        5       46929422    44.76       65.74%      0.938s
headless_shell.gz   Gzip        6       46522529    44.37       66.03%      0.919s
headless_shell.gz   Gzip        7       46406406    44.26       66.12%      0.917s
headless_shell.gz   Gzip        8       46297917    44.15       66.20%      0.916s
headless_shell.gz   Gzip        9       46270972    44.13       66.22%      0.968s
headless_shell.gz   Zopfli      5       45089855    43.00       67.08%      0.931s
headless_shell.gz   Zopfli      10      45089161    43.00       67.08%      0.919s
headless_shell.gz   Zopfli      15      45086375    43.00       67.08%      0.919s
headless_shell.gz   Zopfli      20      45085868    43.00       67.08%      0.919s
headless_shell.gz   Zopfli      25      45085640    43.00       67.08%      0.929s
headless_shell.gz   Zopfli      30      45085003    43.00       67.08%      0.925s
headless_shell.gz   Zopfli      40      45084328    43.00       67.08%      0.921s
headless_shell.gz   Zopfli      50      45084098    43.00       67.08%      0.935s
headless_shell.br   Brotli      0       55401211    52.83       59.55%      0.778s
headless_shell.br   Brotli      1       54429523    51.91       60.26%      0.757s
headless_shell.br   Brotli      2       46436126    44.28       66.10%      0.659s
headless_shell.br   Brotli      3       46122033    43.99       66.33%      0.616s
headless_shell.br   Brotli      4       45050239    42.96       67.11%      0.692s
headless_shell.br   Brotli      5       40813510    38.92       70.20%      0.598s
headless_shell.br   Brotli      6       40116951    38.26       70.71%      0.601s
headless_shell.br   Brotli      7       39302281    37.48       71.30%      0.615s
headless_shell.br   Brotli      8       39038303    37.23       71.50%      0.668s
headless_shell.br   Brotli      9       38853994    37.05       71.63%      0.673s
headless_shell.br   Brotli      10      36090087    34.42       73.65%      0.765s
headless_shell.br   Brotli      11      34820408    33.21       74.58%      0.712s

Right now, the bundled version appears to be compressed with gzip level 6 at 44.37 MiB.

By switching to the gzip-compatible Zopfli we could shave an extra 1.37 MiB.
On the other hand, gzip-incompatible Brotli would save us an extra 11.16 MiB!

Not getting element when executing on Lambda

Hi, could anyone explain the main differences when running local vs on lambda?

It seems when running on Lambda it cannot find the elements but when I run local it does work.

Local Nodejs: v8.10.0
runtime: nodejs8.10

So, I would like to understand how to debug this properly.

{
    "errorMessage": "Node is either not visible or not an HTMLElement",
    "errorType": "Error",
    "stackTrace": [
        "ElementHandle.<anonymous> (/var/task/node_modules/puppeteer/node6/lib/ExecutionContext.js:635:13)",
        "next (native)",
        "step (/var/task/node_modules/puppeteer/node6/lib/ExecutionContext.js:610:24)",
        "Promise.resolve.then.value (/var/task/node_modules/puppeteer/node6/lib/ExecutionContext.js:621:15)",
        "process._tickDomainCallback (internal/process/next_tick.js:135:7)"
    ]
}

My code looks like this:

async login(page, url) {
        console.log('login on '+url);
        await page.goto(url, {waitUntil: 'networkidle0'});
        await page.waitFor(1 * 1000);
        const LOGIN = await page.$x('//*[@id="loginForm"]/div[1]/input');
        console.log(LOGIN);
        if (LOGIN.length > 0) {
            await LOGIN[0].click();
        }
    };

I I've printed the element to make sure it was finding on the page. here is the output.

2018-11-11T17:18:31.416Z	2f2ebb24-4837-5f9e-b625-ef462ebab245	[ ElementHandle {
_context: 
ExecutionContext {
_client: [Object],
_frame: [Object],
_contextId: 2,
_isDefault: true },
_client: 
CDPSession {
domain: null,
_events: [Object],
_eventsCount: 27,
_maxListeners: undefined,
_lastId: 133,
_callbacks: Map {},
_connection: [Object],
_targetType: 'page',
_sessionId: '19D40695F65E926625F0B1A1C1FF03FC:1',
_sessions: Map {} },
_remoteObject: 
{ type: 'object',
subtype: 'node',
className: 'HTMLInputElement',
description: 'input.font-size-1.col-12.ng-pristine.ng-untouched.ng-valid.ng-empty',
objectId: '{"injectedScriptId":2,"id":7}' },
_disposed: false,
_page: 
Page {
domain: null,
_events: [Object],
_eventsCount: 2,
_maxListeners: undefined,
_closed: false,
_client: [Object],
_target: [Object],
_keyboard: [Object],
_mouse: [Object],
_touchscreen: [Object],
_accessibility: [Object],
_networkManager: [Object],
_frameManager: [Object],
_emulationManager: [Object],
_tracing: [Object],
_pageBindings: Map {},
_ignoreHTTPSErrors: false,
_coverage: [Object],
_javascriptEnabled: true,
_viewport: [Object],
_screenshotTaskQueue: [Object],
_workers: Map {} },
_frameManager: 
FrameManager {
domain: null,
_events: [Object],
_eventsCount: 3,
_maxListeners: undefined,
_client: [Object],
_page: [Object],
_networkManager: [Object],
_defaultNavigationTimeout: 30000,
_frames: [Object],
_contextIdToContext: [Object],
_mainFrame: [Object] } } ]

Protocol error (Runtime.enable): Target closed.

Each 5th invocation getting errors similar to the one below:

{
  "errorMessage": "Protocol error (Runtime.enable): Target closed.",
  "errorType": "Error",
  "stackTrace": [
    "Session._onClosed (/var/task/node_modules/puppeteer/node6/Connection.js:313:23)",
    "Connection._onClose (/var/task/node_modules/puppeteer/node6/Connection.js:168:15)",
    "emitTwo (events.js:106:13)",
    "WebSocket.emit (events.js:191:7)",
    "WebSocket.emitClose (/var/task/node_modules/ws/lib/WebSocket.js:224:10)",
    "_receiver.cleanup (/var/task/node_modules/ws/lib/WebSocket.js:210:41)",
    "Receiver.cleanup (/var/task/node_modules/ws/lib/Receiver.js:520:15)",
    "WebSocket.finalize (/var/task/node_modules/ws/lib/WebSocket.js:210:22)",
    "emitNone (events.js:91:20)",
    "Socket.emit (events.js:185:7)"
  ]
}

Anyone seen this before?

NOTE: originally posted here: puppeteer/puppeteer#603 (comment)

The same 'one in five' error happens in on the serverless-chrome project as well as documented here: adieuadieu/serverless-chrome#41

Example not working when I publish

Hi everyone,

I can't make it run the basic sample at index script on a lambda function, here's what I did:

Clone this repo and then run npm install... test locally and works.

Then I run serverless deploy, everything get published, but when I run I always have this issue:
"errorMessage": "No node found for selector: #lst-ib", (to test, I simply go to the lambda function, and run a test)

Actually, I can't inspect any element, If I use waitForSelector I have a timeout, so I assume that page never get loaded.

Here's part of log:

START RequestId: 031bea5a-d60d-11e8-94d7-015d775a53cf Version: $LATEST
2018-10-22T15:13:22.647Z 031bea5a-d60d-11e8-94d7-015d775a53cf setup local chrome
2018-10-22T15:13:24.574Z 031bea5a-d60d-11e8-94d7-015d775a53cf setup done
2018-10-22T15:13:25.359Z 031bea5a-d60d-11e8-94d7-015d775a53cf launch done: HeadlessChrome/67.0.3361.0

Is there an issue with this build or I'm missing something?

Thanks in advance.

Node is either not visible or not an HTMLElement | page.click

exports.run = async (browser) => {
// implement here
// this is sample
const page = await browser.newPage();
await page.setViewport({width: 1024, height: 768});

page.on('dialog', async (dialog) => {
console.log(dialog.message());
await dialog.dismiss();
});

await page.goto('http://efisco.sefaz.pe.gov.br/sfi_trb_gpf/PREmitirCertidaoNegativaNarrativaDebitoFiscal',
{waitUntil: ['domcontentloaded', 'networkidle0']}
);
await page.select('select#cdTipoDocumento', '2');
await page.select('select#cdOrgaoEmissor', '1');
await page.type('#nuDocumento', '17358877000151');

console.log('about to click');
await page.waitFor('#btt_emitir');
await Promise.all([
page.waitForNavigation(),
page.click('#btt_emitir'),
]);

await page.close();
return 'done';
};


the code above executes using `npm run local` with success with headless mode on or off but on lambda I got this error on page.click function. 

2018-07-26 14:12:25.742 (-03:00) 0d39e55d-90f7-11e8-9154-f5bdacb2b61e about to click
2018-07-26 14:12:27.782 (-03:00) 0d39e55d-90f7-11e8-9154-f5bdacb2b61e {"errorMessage":"Node is either not visible or not an HTMLElement","errorType":"Error","stackTrace":["ElementHandle. (/var/task/node_modules/puppeteer/node6/lib/ElementHandle.js:190:13)","next (native)","step (/var/task/node_modules/puppeteer/node6/lib/ElementHandle.js:165:24)","Promise.resolve.then.value (/var/task/node_modules/puppeteer/node6/lib/ElementHandle.js:176:15)","process._tickDomainCallback (internal/process/next_tick.js:135:7)"]}


Any clues why I can run it locally but on lambda it's fail?

Thanks.

Error: WebSocket is not open: readyState 3 (CLOSED)

setupLocalChrome is generating an error when I invoke the Lambda function more than once. Here's my trace.

Error: WebSocket is not open: readyState 3 (CLOSED)
    at WebSocket.send (/var/task/node_modules/ws/lib/websocket.js:314:19)
    at WebSocketTransport.send (/var/task/node_modules/puppeteer/lib/WebSocketTransport.js:57:14)
    at Connection.send (/var/task/node_modules/puppeteer/lib/Connection.js:71:21)
    at Browser._getVersion (/var/task/node_modules/puppeteer/lib/Browser.js:233:29)
    at Browser.version (/var/task/node_modules/puppeteer/lib/Browser.js:208:32)
    at Browser.<anonymous> (/var/task/node_modules/puppeteer/lib/helper.js:145:23)
    at isBrowserAvailable (/var/task/starter-kit/setup.js:31:19)
    at Object.exports.getBrowser (/var/task/starter-kit/setup.js:15:51)
    at exports.handler (/var/task/index.js:14:31)

Afterward, the function logs launch done: HeadlessChrome/67.0.3361.0 and runs the function as expected...Let me know if I can provide more information to help reproduce this error.

Error when running package in windows

I am trying to start with this starter kit but when running 'npm run package', I get this error. `

@ package C:\Users\msarc\coding\puppeteer\project
npm run package-prepare && cp chrome/headless_shell.tar.gz dist && cd dist && zip -rq ../package.zip .

@ package-prepare C:\Users\msarc\coding\puppeteer\project
npm run lint && npm run babel && cp -r package.json dist && cd dist && PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm instal l --production

@ lint C:\Users\msarc\coding\puppeteer\project
eslint src

@ babel C:\Users\msarc\coding\puppeteer\project
rm -rf dist && mkdir dist && ./node_modules/.bin/babel src --out-dir dist

'.' is not recognized as an internal or external command,
operable program or batch file.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ babel: rm -rf dist && mkdir dist && ./node_modules/.bin/babel src --out-dir dist
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ babel script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\msarc\AppData\Roaming\npm-cache_logs\2017-11-06T20_13_55_486Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ package-prepare: npm run lint && npm run babel && cp -r package.json dist && cd dist && PUPPETEER_SKIP_CHR OMIUM_DOWNLOAD=1 npm install --production
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ package-prepare script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\msarc\AppData\Roaming\npm-cache_logs\2017-11-06T20_13_55_607Z-debug.log
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ package: npm run package-prepare && cp chrome/headless_shell.tar.gz dist && cd dist && zip -rq ../package. zip .
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the @ package script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\msarc\AppData\Roaming\npm-cache_logs\2017-11-06T20_13_55_667Z-debug.log

`

How can I fix this error and use this starter kit?

Memory keeps on increasing on every request

I have noticed my lambda function memory consumption keeps on increasing on every subsequent request. I have set my max memory of lambda function to 3008mb. The first call consumes around 400 MB and it will keep on increasing until 3008mb after that I will start getting timeout error more frequently.

Task timed out after 300.07 seconds

{
"errorMessage": "2018-02-07T13:38:33.285Z 7d79ae5e-0c0b-11e8-9c34-071d38a89c78 Task timed out after 300.07 seconds"
}

Getting above error when try to run below code

const mpromise = require('promise');

const setupS3Chrome = () => {
return new mpromise(function (resolve, reject) {
const params = {
Bucket: config.remoteChromeS3Bucket,
Key: config.remoteChromeS3Key
};
console.log(params);
s3.getObject(params).createReadStream().on('error', err => reject(err)).pipe(tar.x({
C: config.setupChromePath
})).on('error', err => reject(err)).on('end', () => resolve());
});
};

Having a bit of trouble getting past browser.newPage() on AWS Lambda

Using the latest version (1.1.2), I am unable to get past const page = await browser.newPage(); using an uploaded package.zip

I have added some debugging to see what's going on and it doesn't dump anything to the AWS log.

I even went as far as downloading the @latest for every package used and am now trying to use Node 8.10 after changing the Babel version to Node 8.10.

My index starts off with:

const setup = require('./starter-kit/setup');

exports.handler = async (event, context, callback) => {
	context.callbackWaitsForEmptyEventLoop = false;
	const browser = await setup.getBrowser();
	exports.run(browser).then(
		(result) => callback(null, result)
	).catch(
		(err) => callback(err)
	);
};

exports.run = async (browser) => {
	try {
		console.log('new page');
		const page = await browser.newPage();

		console.log('emulate screen');
...

The Amazon log is:

START RequestId: <guid> Version: $LATEST
2018-08-24T18:38:41.789Z	<guid>	setup local chrome
2018-08-24T18:38:45.726Z	<guid>	setup done
2018-08-24T18:38:47.545Z	<guid>	new page
END RequestId: <guid>
REPORT RequestId: <guid>	Duration: 5821.84 ms	Billed Duration: 5900 ms 	Memory Size: 512 MB	Max Memory Used: 335 MB	

It's bailing out at const page = await browser.newPage(); and never getting to dump out console.log('emulate screen');. The catch is just a console.log(e) for this try block.

I've attempted to use various versions of chrome and the only variant response I've gotten is with this version:
https://github.com/adieuadieu/serverless-chrome/releases/tag/v1.0.0-54

In which the log was:

START RequestId: <guid> Version: $LATEST
2018-08-24T18:34:33.046Z	<guid>	setup local chrome
2018-08-24T18:34:37.184Z	<guid>	setup done
2018-08-24T18:34:37.208Z	<guid>	Error: Failed to launch chrome! spawn /tmp/headless_shell ENOENT


TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

at onClose (/var/task/node_modules/puppeteer/lib/Launcher.js:299:14)
at ChildProcess.helper.addEventListener.error (/var/task/node_modules/puppeteer/lib/Launcher.js:290:64)
...

Alternative language packs

Question:
What's the most efficient way to include alternative language packs like Japanese and Arabic?
Is there a way to build with the included build script before going all out and including a full chome instance hosted separately on S3.

Cant handle multiple redirects

Kind of a weird one but when have a page goto a url that uses multiple redirects I will get a 200 OK response instead of the 302 redirect that i need. The strange thing is that it works fine when ran locally but on lambda it stops and ends with the 200 after the first redirect.

Too clarify:

When I run it locally it does what it should so which is this
Request:GET -> Response:302 -> Request:GET -> Response:302 -> Request:GET -> Response:302 -> Request:GET -> Response:302 -> Request:GET -> Response:200 OK (the page it should be on)

When I run it on a lambda It does this
Request:GET -> Response:302 -> Request:GET -> Response:200 OK (blank html page)

Is there a limit to redirects in the chrome binary? I have tried the binary provided as well as the one that is build using the build.sh script that is provided.

The code I am using is just a simple page.goto(url)

Any help would be vastly appreciated

Unable to start Chrome on Lambda

I uploaded headless_shell.tar.gz to S3. Lambda returns error:

`START RequestId: 4279d41e-d99e-11e7-8d7c-754f3d6a60fb Version: $LATEST
2017-12-05T09:25:40.241Z 4279d41e-d99e-11e7-8d7c-754f3d6a60fb /tmp/headless_shell
2017-12-05T09:25:40.241Z 4279d41e-d99e-11e7-8d7c-754f3d6a60fb [ '--no-sandbox', '--disable-gpu', '--single-process' ]
2017-12-05T09:25:40.246Z 4279d41e-d99e-11e7-8d7c-754f3d6a60fb (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: Failed to launch chrome! spawn /tmp/headless_shell ENOENT

TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

END RequestId: 4279d41e-d99e-11e7-8d7c-754f3d6a60fb
REPORT RequestId: 4279d41e-d99e-11e7-8d7c-754f3d6a60fb Duration: 91.17 ms Billed Duration: 100 ms Memory Size: 512 MB Max Memory Used: 40 MB `

Failed to load libosmesa.so even with --disable-gpu flag

Chromium still requires osmesa even with --disable-gpu flag. It worked fine on 0.10.2 version, but fails when I use recent versions (v.1.0.0 or v1.1.1) with headless_shell binaries for that versions from this repo. I tried to use all these 'disable-gpu'-related flags with no luck:

		'--no-sandbox',
		'--disable-gpu',
		'--single-process',
		'--disable-webgl',
		'--disable-software-rasterizer',
		'--disable-accelerated-compositing',
		'--disable-force-compositing-mode',
		'--disable-accelerated-compositing',
		'--disable-accelerated-layers',
		'--blacklist-webgl',
		'--disable-accelerated-2d-canvas',
		'--disable-gpu-vsync',
		'--blacklist-accelerated-compositing',
		'--blacklist-webgl',
		'--disable-gpu-sandbox',
		'--disable-accelerated-video',
		'--disk-cache-size=0',
		'--homedir=/tmp',
		'--data-path=/tmp/data-path',
		'--disk-cache-dir=/tmp/cache-dir',
		'--user-data-dir=/tmp/user-data',

logs:

[0301/115533.993711:WARNING:resource_bundle.cc(368)] locale_file_path.empty() for locale
[0301/115533.999319:WARNING:resource_bundle.cc(368)] locale_file_path.empty() for locale
[0301/115534.002328:ERROR:gpu_process_transport_factory.cc(1017)] Lost UI shared context.
DevTools listening on ws://127.0.0.1:43536/devtools/browser/f8530605-40nc-420a-a05b-dad9c2367629
[0301/115534.003021:ERROR:devtools_http_handler.cc(277)] Error writing DevTools active port to file
Fontconfig warning: "/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf
[0301/115534.024673:WARNING:histograms.cc(40)] Started multiple compositor clients (Browser, Renderer) in one process. Some metrics will be disabled.
[0301/115534.146301:ERROR:gl_implementation.cc(292)] Failed to load /opt/lambdas/screenshot/bin/libosmesa.so: /opt/lambdas/screenshot/bin/libosmesa.so: cannot open shared object file: No such file or directory

which headless_shell file is the last version of chrome?

I'm trying to deploy pupeeteer on lambda.

I'm currently using pupeeteer 1.7.0. so I found your git project.

when I run 'npm run package' the zip file is over 50MB.

if it correct to use under chrome folder headless_shell-70.0.3531.0.tar.gz or headless_shell.tar.gz ??

could you tell me which one tar.gz file should I use?

Cannot Create Package: 'eslint' is not recognized as an internal or external command

When following installation instructions, I am unable to create a package using 'npm run package', even with the initial, simple index.js script that is included. I get the error:

''eslint' is not recognized as an internal or external command'
pup-lambda-error

Is eslint required to be installed for this utility? Even after installilng eslint, I am met with more errors:

pup-lambda-error2

This is all before even trying to do anything complex with puppeteer! What am I missing?

Running on Windows 10, using Node version 8.

Where to get new binaries from?

Hi,

Puppeteer v1.12.2 requires Chromium 73.0.3679.0 (r624492). I'd like to know, is there a reliable place where I can get a binary that will just work on Lambda? serverless-chrome doesn't seem to have it

Serverless framework

Is it possible to integrate it with the serverless framework? Also right now we have to deploy almost a 44MB file for each time we change the script (unless i'm missing something).

Is it possible to split apart the puppeteer like chromeless using a remote proxy setup? I have some ideas on how we can do this.

If anybody wants to collaborate please let me know.

San

Stops on page.goto of exports.run without error

Has anyone else seen this start fine, run without error and then stop in the middle of the exports.run. If I modify the example code as follows:

exports.run = async (browser) => {
  console.log('in exports.run..');
  const page = await browser.newPage();
  await page.goto('https://www.google.co.jp',
   {waitUntil: ['domcontentloaded', 'networkidle0']}
  );
  console.log('after goto..');
  console.log((await page.content()).slice(0, 500));

I get the first console.log and not the second...

START RequestId: 07699f81-73fe-11e8-8b00-6f64b92f92ad Version: $LATEST
2018-06-19T20:19:16.966Z	07699f81-73fe-11e8-8b00-6f64b92f92ad	calling exports.run..
2018-06-19T20:19:16.967Z	07699f81-73fe-11e8-8b00-6f64b92f92ad	in exports.run..
END RequestId: 07699f81-73fe-11e8-8b00-6f64b92f92ad
REPORT RequestId: 07699f81-73fe-11e8-8b00-6f64b92f92ad	Duration: 4174.92 ms	Billed Duration: 4200 ms 	Memory Size: 1024 MB	Max Memory Used: 280 MB	

Any ideas?

Cannot cd into '/tmp'

In trying to setup local chome in this block,

const setupLocalChrome = () => {
    return new Promise((resolve, reject) => {
        fs.createReadStream(config.localChromePath)
            .on('error', (err) => reject(err))
            .pipe(tar.x({
                C: config.setupChromePath,
            }))
            .on('error', (err) => reject(err))
            .on('end', () => resolve());
    });
};

I get this error, has anyone else bumped into this and found a good solution? I suppose I can chmod the /tmp directory but that seems like a hack.

Cwd Error ----------------------------------------------
 
  ENOTDIR: Cannot cd into '/tmp'
 
     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
 
  Stack Trace --------------------------------------------
 
CwdError: ENOTDIR: Cannot cd into '/tmp'
    at CwdError (/Users/mark/dev/repos/spr-renderer/node_modules/tar/lib/mkdir.js:26:5)
    at fs.lstat (/Users/mark/dev/repos/spr-renderer/node_modules/tar/lib/mkdir.js:74:14)
    at FSReqWrap.oncomplete (fs.js:123:15)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.