3nock / spidersuite Goto Github PK

Where the source code?

我爬取了一个网站，想将其作为本地镜像，可以离线访问、使用代码程序进行分析、获取数据等。（I crawled a website and wanted to use it as a local mirror so that I can access it offline, analyze it using code programs, obtain data, etc.）

Repro steps

When opening the application and attempting to crawl a site using ssl (HTTPS). I am getting this error in logs;

WARNING: QSslSocket: cannot resolve EVP_PKEY_base_id
WARNING: QSslSocket: cannot resolve SSL_get_peer_certificate
DEBUG: Checking for Updates...
WARNING: QSslSocket: cannot call unresolved function SSL_get_peer_certificate

I am using Ubuntu and I have attempted to install OpenSSL and reinstall but nothing. I have also installed using the AppImage and the run installer. I am using the most up to date version.

How often does this bug happen?

Every time

The desktop OS you're using

Ubuntu 22.04

Details of other apps/devices

No response

Error screenshot

No response

Any other info?

Would really like to use this as it looks great.

Problem

How to browse the crawled website locally offline? I have crawled a website and I want to use it as a local mirror that can be accessed offline, analyzed using a code program, data fetched, etc.

Solution

Save the downloaded site with one click instead of selecting only one file at a time.

Alternatives

No response

Additional context

No response

I go to Application/Import From/katana/json
and when i load the json file it pops up says Importing from "D:/path/file.json" Complete

but i get nothing and it does not import it

Problem

spider suite doesn't deal with parameters

Solution

dealing with parameters like :
1- give the parameters in GET and POST requests and number of them in each request
2- alert reflected parameters
3- number of occurrence for every param

Alternatives

No response

Additional context

No response

Problem

Missing installer binary release for M1/M2 Apple silicon devices (AARCH64) and ARM64 device architectures?

Solution

Build and provide an installer release (binary) for macOS M1/M2 Apple silicon (AARCH64) and ARM64 CPU device architectures.

Alternatives

Provide OSS Source Code and instructions on how to build SpiderSuite natively.

Additional context

Cannot find instructions or documentation that supports for installation on macOS M1/M2 Apple silicon (AARCH64) or ARM64 device CPU architectures.