F95API

Unofficial Typescript API used for data scraping from the F95Zone platform.

These APIs have been developed to support this application and allow you to obtain data on games and mods on the platform F95zone (NSFW)

A simple usage example can be found in src/example.ts

Main features
Login
Data scraping
Classes
Logging
Guidelines for errors

Main features

Support for two-factor authentication
Structured and typed data of games, mods, comics, assets and animations
Structured and typed data of the current user (threads followed, favorites...)
Structured and typed data of other users (titles, donations, registration date...)
Standard search, through the "Latest Updates" page or advanced
Saving of cookies to maintain the session

Login

In addition to traditional login, two-factor authentication via callback that returns the OTP value is also supported.

// Normal login
let result = await F95API.login(username, password);

// 2FA login
let 2faCallback = function () => Promise<number>;
let result = await F95API.login(username, password, 2faCallback);

Refer to src/example.ts (lines 35-67) for a working example.

Data scraping

Each game, mod, comic, animation or asset is identified as handiwork and obtaining this element is generic, it will then be up to the user to decide which property to use based on the type of handiwork that has been requested.

Handiworks can be obtained by query or by URL.

// With query
const query = {
    category: "games";
    keywords = "Your game name";
    order = "likes";
};

let listOfFoundGames = await F95API.searchHandiwork<Game>(query);

// With URL
let specificGame = await F95API.getHandiworkFromURL<Game>("the URL of your game");

While user data (after authenticating) with

const userdata = new UserProfile();

// Fetch basic data (always necessary)
await userdata.fetch();

// Fetch all data (may take a while)
await userdata.fetch(true);

// Async properties
const threads = await userdata.watched;
const bookmarks = await userdata.bookmarks;
const alerts = await userdata.alerts;
const games = await userdata.featuredGames;

Classes

Handiwork

Information about games, mods, etc... are stored in a Handiwork object with the following fields:

Basic properties

This list of properties is common to every handiwork, be it a game, a comic, etc...

Property	Type	Description
`id`	`number`	Unique ID of the thread associated with the handiwork in the platform
`name`	`string`	The handiwork name
`overview`	`string`	Description of the handiwork
`authors`	`TAuthor[]`	The developer of this handiwork including the platforms on which it can be found
`category`	`TCategory`	The membership category
`changelog`	`TChangelog[]`	List of changes divided by version
`cover`	`string`	URL of the possible cover of the handiwork (in case of multiple images the first available is taken)
`lastThreadUpdate`	`Date`	Date of the last update of the thread associated with the handiwork
`prefixes`	`string[]`	List of prefixes
`rating`	`TRating`	Users' ratings on the handiwork (average, maximum, minimum, number of votes)
`tags`	`string[]`	List of tags
`threadPublishingDate`	`Date`	Publication date of the thread associated with the handiwork
`url`	`string`	The URL that leads to the handiwork's thread on F95Zone

Mixed properties

Each property is specific to a particular type of handiwork.

Property	Type	Handiwork type	Description
`genre`	`string[]`	`Animation`/`Comic`/`Game`	List of genres represented in this work
`pages`	`string`	`Animation`/`Comic`	Number of pages or elements of which the work is composed
`resolution`	`string[]`	`Animation`/`Comic`	List of resolutions available for the work
`installation`	`string`	`Animation`	Installation instructions
`language`	`string[]`	`Animation`	List of available languages
`length`	`string`	`Animation`	Length of the animation
`censored`	`boolean`	`Animation`/`Game`	Are the NSFW parts censored?
`engine`	`TEngine`	`Game`	Game engine (Unity, Ren'Py, RPGM...)
`lastRelease`	`Date`	`Game`	Date of the last update of this handiwork
`mod`	`boolean`	`Game`	Specify if it is a mod or a real game
`os`	`string[]`	`Game`	List of supported OS (Windows/Linux/Mac/Android...)
`status`	`TStatus`	`Game`	State of development (Completed/Abandoned/Ongoing/Onhold)
`version`	`string`	`Game`	Version of the handiwork
`assetLink`	`string`	`Asset`	External URL of the asset (es. Daz3D store)
`associatedAssets`	`string[]`	`Asset`	List of URLs of assets associated with the work (for example same collection)
`compatibleSoftware`	`string[]`	`Asset`	Software compatible with the work
`includedAssets`	`string[]`	`Asset`	List of assets url included in the work or used to develop it
`officialLinks`	`string[]`	`Asset`	List of official links of the work, external to the platform (es. Daz3D store)
`sku`	`string`	`Asset`	Unique SKU value of the work

The serialization/deserialization in JSON format of this object is possible through JSON.stringify()/JSON.parse().

Platform user

A generic user registered on the platform is represented by a PlatformUser object with the following fields:

Property	Type	Description
`id`	`number`	Unique user ID
`name`	`string`	Username
`title`	`string`	Title assigned to the user by the platform
`banners`	`string[]`	List of banners assigned by the platform
`messages`	`number`	Number of messages written by the user
`reactionScore`	`number`	Total number of reactions received from other users
`points`	`number`	Total number of points received after the acquisition of trophies
`ratingsReceived`	`number`	Number of ratings received
`joined`	`Date`	Date of joining the platform
`lastSeen`	`Date`	Date of the last connection to the platform
`followed`	`boolean`	Indicates whether the user is followed by the currently logged in user
`ignored`	`boolean`	Indicates whether the user is ignored by the currently logged on user
`private`	`boolean`	Indicates that the profile is private and not viewable by the user
`avatar`	`string`	URL of the image used as the user's avatar
`amountDonated`	`number`	Value ($) of donations made

User data

The user data currently connected through this API extends the PlatformUser class via the class UserProfile and adds:

Property	Type	Description
`watched`	`Promise<IWatchedThread[]>`	List of followed thread data
`bookmarks`	`Promise<IBookmarkedPost[]>`	List of bookmarked posts data
`alerts`	`Promise<IAlert[]>`	List of alerts
`conversations`	`Promise<IConversation[]>`	List of conversations (only without labels)
`featuredGames`	`Promise<Game[]>`	List of featured games from the platform (carousel, may not be available if disabled in settings)

Login results

The outcome of the authentication process is represented by the LoginResult object:

Property	Type	Description
`success`	`boolean`	List of followed thread data
`code`	`number`	Code associated with the result of the login operation
`message`	`string`	Possible error message (unrecognized user, wrong password...) or authentication successful message

Logging

To log the behavior of the application log4js-api is used with a default level of "warn". This option can be changed with the loggerLevel property.

Guidelines for errors

If you can, return a meaningful value
Return null only if the function should return a complex object (including strings)
Return an empty array if the function should return an array
Return false, -1 when the function should return boolean or number
Throw an exception only if it is an error or if a wrong value could mess up the functioning of the library
For all network operations, return a Result object

Site's new DDOS-GUARD DDOS protection is resulting in errors when scraping

I see this when running the example with the correct credentials. I can confirm that this worked a few days ago when was testing #257. Pretty sure this is caused by the newly added DDOS protection from DDOS-GUARD. Need to debug this further and probably look into bypassing DDOS-GUARD (could be something custom or a lib).

f95api: v2.0.0-beta.12
Node.js: v17.6.0
Windows 10: v19044.1586

PS G:\dev\code\F95API> npm run example

> @millenniumearl/[email protected] example     
> npm run build && node --inspect ./dist/example.js


> @millenniumearl/[email protected] prebuild
> node -p "'export const LIB_VERSION = ' + JSON.stringify(require('./package.json').version) + ';'" > src/version.ts


> @millenniumearl/[email protected] build
> npm run prebuild && tsc --project ./      


> @millenniumearl/[email protected] prebuild
> node -p "'export const LIB_VERSION = ' + JSON.stringify(require('./package.json').version) + ';'" > src/version.ts

Debugger listening on ws://127.0.0.1:9229/cedbfbf7-6837-4df6-ba8e-3fb0e50b85d3
For help, see: https://nodejs.org/en/docs/inspector
Authenticating...
G:\dev\code\F95API\dist\scripts\network-helper.js:111
        throw new errors_1.InvalidF95Token(`Invalid token for auth: ${credentials.token}`);
              ^

InvalidF95Token [Error]: Invalid token for auth: undefined
    at authenticate (G:\dev\code\F95API\dist\scripts\network-helper.js:111:15)        
    at loginInTheRemotePlatform (G:\dev\code\F95API\dist\scripts\login.js:105:58)     
    at login (G:\dev\code\F95API\dist\scripts\login.js:47:29)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async authenticate (G:\dev\code\F95API\dist\example.js:72:20)
    at async main (G:\dev\code\F95API\dist\example.js:143:9)

millenniumearl / f95api Goto Github PK

f95api's Introduction

F95API

Table of Contents

Main features

Login

Data scraping

Classes

Handiwork

Basic properties

Mixed properties

Platform user

User data

Login results

Logging

Guidelines for errors

f95api's People

Contributors

Stargazers

Watchers

Forkers

f95api's Issues

Description

Occurrences

Description

Occurrences

Rate-Limited

Open

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org