Comments (7)
I've made sure that puppeteer-extra is a useless thing.
Nothing is working.
from puppeteer-extra.
It seems like the problem is that it can't fool the devtools detectors
https://cdn.jsdelivr.net/npm/devtools-detector
https://github.com/AEPKILL/devtools-detector
from puppeteer-extra.
I've written a simple script to bypass these detectors. It works with this website; I'm not sure about the others
You should evaluate this script on a new document like this:
page.evaluateOnNewDocument(script)
from puppeteer-extra.
@Overmiind
I added your script but it is still detecting and redirecting to https://brbeast.com/player/includes/template/no_video.html
i.e. having the same problem
//npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
const puppeteer = require('puppeteer-extra')
const sleep = ms => new Promise(res => setTimeout(res, ms));
const script = `
() => {
function isDevToolsScript() {
var stack = new Error().stack;
return stack.includes('devtool');
}
Date.prototype.originalGetTime = Date.prototype.getTime;
Date.prototype.getTime = function () {
if (!isDevToolsScript()) {
return this.originalGetTime();
}
return 0;
}
const originalOnMessageSetter = Object.getOwnPropertyDescriptor(Worker.prototype, 'onmessage').set;
Object.defineProperty(Worker.prototype, 'onmessage', {
set: function (fn) {
if (!isDevToolsScript()) {
originalOnMessageSetter.call(this, fn);
return;
}
newFn = (ev) => {
ev.data.time = 0;
fn(ev);
}
originalOnMessageSetter.call(this, newFn);
}
});
}
`;
// puppeteer-extra is a drop-in replacement for puppeteer,
// it augments the installed puppeteer with plugin functionality
const url = 'https://brbeast.com/video/f79921bbae40a577928b76d2fc3edc2a';
// add stealth plugin and use defaults (all evasion techniques)
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
// puppeteer usage as normal
puppeteer.launch({ headless: true }).then(async browser => {
console.log('Running tests..')
const page = await browser.newPage()
page.evaluateOnNewDocument(script)
page.goto(url)
await sleep(5000)
await page.screenshot({ path: 'testresult.png', fullPage: true })
await browser.close()
console.log(`All done, check the screenshot. ✨`)
})
from puppeteer-extra.
You can block devtools scripts from being downloaded using puppeteer request interception. I tried that on a puppeteer c# version and it works
from puppeteer-extra.
You can block devtools scripts from being downloaded using puppeteer request interception. I tried that on a puppeteer c# version and it works
Can you give more information on how to block devtools scripts download?
from puppeteer-extra.
I did this in Dart and it works
import 'dart:async';
import 'dart:convert';
import 'dart:io';
import 'dart:math';
import 'package:http/http.dart';
import 'package:puppeteer/puppeteer.dart';
//https://brbeast.com/video/e4bb4c5173c2ce17fd8fcd40041c068f
final customUA =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36';
final script = '''
() => {
window.onbeforeunload = function() { return "sorry, Your some work will be lost - really sorry."; };
function isDevToolsScript() {
var stack = new Error().stack;
return stack.includes('devtool');
}
Date.prototype.originalGetTime = Date.prototype.getTime;
Date.prototype.getTime = function () {
if (!isDevToolsScript()) {
return this.originalGetTime();
}
return 0;
}
const originalOnMessageSetter = Object.getOwnPropertyDescriptor(Worker.prototype, 'onmessage').set;
Object.defineProperty(Worker.prototype, 'onmessage', {
set: function (fn) {
if (!isDevToolsScript()) {
originalOnMessageSetter.call(this, fn);
return;
}
newFn = (ev) => {
ev.data.time = 0;
fn(ev);
}
originalOnMessageSetter.call(this, newFn);
}
});
}''';
final blockUrls = [
//'https://pl20623807.toprevenuegate.com/20/41/ad/2041ad026c42ce264b91586de1c33c6e.js',
'https://cdn.jsdelivr.net/npm/devtools-detector',
//'https://code.jquery.com/jquery-1.12.4.min.js',
//'https://brbeast.com/player/assets/scripts.php?v=6'
];
void main() async {
try {
await run();
} catch (e, s) {
print('main $e $s');
}
}
Future<void> run() async {
final completer = Completer<void>();
final url = 'https://brbeast.com/video/f79921bbae40a577928b76d2fc3edc2a';
final browser = await puppeteer.launch(
//timeout: Duration(days: 4),
headless: false,
defaultViewport: DeviceViewport(width: 1280, height: 720),
);
// Set custom user agent
var page = await browser.newPage();
await page.evaluateOnNewDocument(script);
await page.setUserAgent(customUA);
// Setting page view { 'width': 1280, 'height': 720 }
// await page.setViewport(DeviceViewport(width: 1280, height: 720));
// await page.setExtraHTTPHeaders({
// 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36',
// 'upgrade-insecure-requests': '1',
// 'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8',
// 'accept-encoding': 'gzip, deflate, br',
// 'accept-language': 'en-US,en;q=0.9,en;q=0.8'
// });
await page.setRequestInterception(true);
//: https://mrbeast1.com/cdn/down/disk2/8e011e9e2e455c20b9a94345d0bc7af6/Video/720p/720p_020.html
page.onRequest.listen((req) async {
if (blockUrls.contains(req.url)) {
req.abort();
} else {
req.continueRequest();
}
});
final exp = RegExp(r'(?:(?:https?|ftp):\/\/)?[\w/\-?=%.]+\.[\w/\-?=%.]+');
page.onResponse.listen((response) async {
if (response.url.contains('https://brbeast.com/m3/')) {
var m3u8data = utf8.decode(await response.bytes);
await File('downloads/m3u8data.txt').writeAsString(m3u8data);
Iterable<RegExpMatch> matches = exp.allMatches(m3u8data);
List<Uri> urls = [];
for (var match in matches) {
final urlVideoTs = m3u8data.substring(match.start, match.end);
if (urlVideoTs.startsWith('http')) {
//print('url: $urlVideoTs');
final uri = Uri.tryParse(urlVideoTs);
if (uri != null) {
urls.add(uri);
}
}
}
await browser.close();
await downloadFile(urls);
completer.complete();
}
// a url que tem master.txt é o arquivo m3u8
// if (response.url.contains('master.txt')) {
// final name = sanitizeFilename(response.url.split('/').last);
// //print(response.headers);
// final data = utf8.decode(await response.bytes);
// var match = exp.allMatches(data).last;
// var urlMaster = data.substring(match.start, match.end);
// print(urlMaster);
// await File('downloads/$name').writeAsString(data);
// downloadFile(urlMaster);
// }
});
await page.goto(url, wait: Until.domContentLoaded);
//await Future.delayed(Duration(seconds: 10));
//await page.screenshot();
// await myPage.pdf();
// await page.evaluate<String>('() => document.title');
// Gracefully close the browser's process
// browser.close();
// await Future.delayed(Duration(days: 4));
return completer.future;
}
Future<void> downloadFile(List<Uri> urlV) async {
final videoFile =
await File('downloads/video.ts').open(mode: FileMode.writeOnlyAppend);
final audioFile =
await File('downloads/audio.ts').open(mode: FileMode.writeOnlyAppend);
//
for (var item in urlV) {
if (item.path.contains('Video')) {
print('downloadFile $item');
final request = await get(item);
for (var byte in request.bodyBytes) {
await videoFile.writeByte(byte);
}
await videoFile.flush();
}
if (item.path.contains('Audio')) {
print('downloadFile $item');
final request = await get(item);
for (var byte in request.bodyBytes) {
await audioFile.writeByte(byte);
}
await audioFile.flush();
}
}
await videoFile.close();
await videoFile.close();
}
/// Replaces characters in [input] that are illegal/unsafe for filenames.
///
/// You can also use a custom [replacement] if needed.
///
/// Illegal Characters on Various Operating Systems:
/// / ? < > \ : * | "
/// https://kb.acronis.com/content/39790
///
/// Unicode Control codes:
/// C0 0x00-0x1f & C1 (0x80-0x9f)
/// https://en.wikipedia.org/wiki/C0_and_C1_control_codes
///
/// Reserved filenames on Unix-based systems (".", "..")
/// Reserved filenames in Windows ("CON", "PRN", "AUX", "NUL", "COM1",
/// "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9",
/// "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", and
/// "LPT9") case-insesitively and with or without filename extensions.
///
/// Each results have a maximum length of 255 characters:
/// https://unix.stackexchange.com/questions/32795/what-is-the-maximum-allowed-filename-and-folder-size-with-ecryptfs
///
/// Example:
/// ```dart
/// import 'package:sanitize_filename/sanitize_filename.dart';
///
/// const unsafeUserInput = "~/.\u0000ssh/authorized_keys";
///
/// // "~.sshauthorized_keys"
/// sanitizeFilename(unsafeUserInput);
///
/// // "~-.-ssh-authorized_keys"
/// sanitizeFilename(unsafeUserInput, replacement: "-");
/// ```
String sanitizeFilename(String input, {String replacement = ''}) {
if (input == '') {
return '_vazio_' + Random().nextInt(10000).toString();
}
final result = input
// illegalRe
.replaceAll(
RegExp(r'[\/\?<>\\:\*\|"]'),
replacement,
)
// controlRe
.replaceAll(
RegExp(
r'[\x00-\x1f\x80-\x9f]',
),
replacement,
)
// reservedRe
.replaceFirst(
RegExp(r'^\.+$'),
replacement,
)
// windowsReservedRe
.replaceFirst(
RegExp(
r'^(con|prn|aux|nul|com[0-9]|lpt[0-9])(\..*)?$',
caseSensitive: false,
),
replacement,
)
// windowsTrailingRe
.replaceFirst(RegExp(r'[\. ]+$'), replacement);
return result.length > 255 ? result.substring(0, 255) : result;
}
from puppeteer-extra.
Related Issues (20)
- Unable to detect verification code [Bug]
- [Bug] Extract Stealth Evasions stealth.min.js does not exist in gitcdn.xyz
- [Feature] npx extract-stealth-evasions in Docker
- missing error imports
- [Bug] Web Worker detect actual Navigation. HOT 2
- [Bug] Recaptchaplugin gets detected HOT 1
- Puppeteer v22.0.0 [Bug]
- userPrefs.protocol_handlers.exclude_schemes property set with puppeteer-extra-plugin-user-preferences not working
- ERROR : E: Unable to locate package google-chrome-stable while building Docker image HOT 1
- [Bug] Stealth is not working properly HOT 3
- This browser or app may not be secure [Login in Gmail] HOT 6
- [Bug] hCaptcha detect fails due to new hCaptcha URL format
- Playing audio on fake mic of puppeteer
- [Bug] TargetCloseError: Protocol error puppeteer-extra-plugin-stealth
- [Bug] REPL plugin giving error for spread syntax
- Inconsistent ad blocking behavior with puppeteer-extra-plugin-adblocker in Puppeteer[Bug] HOT 1
- [Bug] Puppeteer gets detected on cloudflare HOT 1
- [Bug] `RangeError: Maximum call stack size exceeded` because of hacks to `Function.prototype.toString` in `puppeteer-extra-plugin-stealth`
- TypeError: utils.isObject is not a function HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from puppeteer-extra.