Giter VIP home page Giter VIP logo

nexushub's Introduction

NexusHub

GitHub release build js-standard-style Discord


NexusHub is an online service which provides insights on game economics and other ingame data. We pride ourselves on the freedom that we take to gather our data - If it's there, we'll find some way to extract it. This repo contains the application logic for our servers, including all statistics calculations. For data-tracking via OCR, please see nexus-sentry.


You can find the live version of the latest features from the development branch at staging.nexushub.co


API

Documentation
API client


Want to help?

Want to file a bug, contribute some code, or improve documentation? Excellent! Read up on our guidelines for contributing and then open an issue regarding your concern.

NexusHub is entirely developed by community members of the games we support, so if you wanna join our team as a regular contributor, you're more than welcome!

If you need an introduction to our code base, feel free to ask us on Discord! Though, we generally recommend going through the "Getting started" tutorial for Cubic first. It's the framework that powers nearly everything on our webserver and web app.


Quickstart

Make sure you have Docker installed first - we recommend following the installation instructions in our docker repo. Once installed, locate this repo and run:

sudo bash docker.sh

This will automatically install and run all required services. Feel free to grab a coffee while waiting for the first install, it'll probably take a few minutes. Once done, check out localhost:3000 and start hacking! You're gonna find all API endpoints in /api while all sites and UI elements are in /ui.

To stop the service again, just run

docker stack rm nexus






Blobot needs stars to survive. Not kidding.

Like what we do? Leave a star to make this little robot happy โค๏ธ


License

CC BY-NC 4.0

nexushub's People

Contributors

greenkeeper[bot] avatar kaptard avatar lldglichigo2 avatar nakroma avatar ogg-technologies avatar robineijsbouts avatar tobitenno avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nexushub's Issues

Add median value to component intervals

Not sure why this isn't a thing yet, the code basis for this should still be there, just perform a median calculation in each interval with the available price stack.

DB Client Race Condition in Method.js

If method is routed but Method.js has not connected to the mongodb, unhandled promise rejection occurs because this.db inside Method is not defined.

Suggestion: Return 'Not ready' (202 - Accepted)

DB connection string

Presently your mongo connection string is in plain text.
Just for your own security, while it is a LAN-reserved IP, it'd still be safer to move it to either a configuration file or, better yet, a process file for something like PM2

Provide data for price-range with most requests

In the item statistics endpoint, add a key to each component which displays the price-range in which most requests are located in.

Possible approaches:

  • Procedural - Count offers in price distances of ~5% of the median, then check the surrounding price range of the distance with most offers for more precise start/end detection
  • Maths - I'm pretty sure there are mathematical approaches to solve this problem more accurately and efficiently. Determining if there are multiple "hot zones" would be very useful as well.

Combine with #36 (individual buying/selling statistics)

Add "price trust"

Add a percentage how much a price can be "trusted". E.g. if theres only one offer it should be about 2%~, if there are 10k offers with almost the same price 100%

Add startup/shutdown services

Add scripts and services that get run on startup or shutdown, sort of like middleware.
Example:

[{ name: "Ensure and create index", script: "/scripts/mongo_index.js", entry: "start" }, ... ]

Only process n amount of requests per stats query

In order to keep processing time constant, even on longer time-range queries, it would seem sensible to have an upper limit of requests processed. However, instead of capping the output with a limit, we'd have to skip every n'th request based on an estimate of how many requests are present.
The resulting supply/demand data would then be multiplied with the according factors.

Open-Source Bounty System

Feature Description

Adds a system that allows attaching bounties (in the shape of ingame assets) to Github issues. List of bounties will be on separate page on development hub. New listings may also get automatically posted on Discord and mention anyone who joins the specific role.

Which problem will your feature solve?

This feature will likely bring a lot of life, or at least attention, to our open source approach.

What scenario will your feature be used in?

Will be used by players who need ingame currency/items in a short time frame.
Also useful for developers that were interested anyway and needed that bit of extra convincing to get more involved into the respective issues.

Documentation

Due to the scope of this project, the documentation will be kept to an abstract minimum

Scraping
Issues with bounties will be scraped through Github's API. Issues will be filtered by the Bounty label and then by user (whether or not they're in the nexus-devs org). The title should include the bounty like Some Issue [Bounty: 30p] where everything between the last : and last ] will be taken as bounty.

User implementation
The resulting data will then be displayed on their own page on the nexus-stats website (simple sortable list) and updated through our nexus-bot on discord.

List of URLs that need to be redirected

  • Index should redirect to /warframe/
  • Any item link (prev. /category/item/)
  • Any image link (prev. /img/category/Something Prime-min.ext)
  • Nexusbot page (should also go to /warframe/*)
  • API page (/api -> api.nexus-stats.com)
  • Open Source page ('how do we get our stats' -> /open-source)

Add more if necessary

No Konami code.

Site doesn't have Konami Code. I r8 0/8, very bad m8.
Suggestion: Convert all characters into emojis.

Add /items/requests

List of items and their user requests in the last hour. Generated from /items/:item/requests, Would dynamically maintain the items collection.

Calculate time window average based on request count

Currently the average for the full time window is calculated by summing up each interval's avg and dividing by number of intervals.

The issue: One interval with extreme prices and only few offers that day affects the overall average too much.

To fix: Before calculating average for each interval, add the price sum to the main component. Later divide price sum of main component by valid offers.

Sidebar touch triggers random event when vertically panning on active panel

To reproduce:

  1. Open Sidebar
  2. Select panel
  3. Pan vertically

Expected result:
Nothing, vertical pan should be ignored

Actual result:
Seemingly "random" event is triggered with deltaX and deltaY being the diff of what seems to be the left screen center (not confirmed). At the same time, the event immediatley receives the isFinal property, which triggers the reset() function. Listening to the panend event won't work either (never gets triggered)

Edit: This may be an issue with hammer.js, not the implementation

Make use of async components

See here for reference. Currently the entire app is loaded in one chunk, which obviously isn't very efficient.
Integration details with vue-router can be found here

Use weighted min/max purges

Need to purge depending on supply/demand. Huge demand increase indicates sudden market change that would otherwise be purged.

No License

The codebase doesn't have a license, contributing information, or copying information. Providing those would allow for contributions from others, if the team is open to it.

Overhaul open source page

The current open source page is just using a very basic list layout. However, since the open source approach is integral to our project, we should do as good of a job explaining the concept as possible.

The core message of the page is supposed to be that we're an open community that welcomes anyone who wants to shape the future of this project.

Sub goals of the page include:

  • Full transparency for our users. Be it for trust or to serve their curiosity about the project.
  • Explain the soon to be implemented bounty system
  • Adding features is easy. Link to some basic examples that explain how it works for API endpoints and UI additions. (this should be the second-most important part next to the community aspect. Would make a good CTA.
  • At the end of each example, link to the 'Add a new Feature' page or our issue page if people wanna fix existing problems.

Some useful references:
https://opensource.google.com/

Price to Effort values

As suggested on Discord. 'Effort' would be defined by an item's ducat cost combined with their demand vs supply ratio and count. Higher demand with many offers indicates difficulty and obtainability.

How to implement?
Would be most interesting in a view to display top/worst performers -> When displayed as individual stat on the item page, it makes sense to display their relative ranking as well. Something like: 0.89 - #12.

From a programmer's point of view, we'll need to just take the 3 values we have (price, ducats, supply/demand) and arbitrarily see what makes most sense.

Who would benefit from this?
Regular players who aren't too big into the capitalism aspect (shows them the best places to farm). Combined with patch note events as discussed with @-stratosphere-#5814, this may be a nice set of data to have when analyzing trends/game meta.

Some components not getting picked up properly

See Venka Prime Set, Banshee Prime Set, Fragor Prime Set. No idea what causes this. Sometimes the sets do get through. But there definitely should be more.

(checked the requests collection, there's lots of valid offers)

Error Verbosity

Invalid requests return no information about rejection reason.

To fix: Return Error Object with proper reason. If return value instance of error -> send error back.

List of omitted UI features

I'll keep this list to keep track of every UI feature that has been omitted while building certain components. (Most likely since they're unnecessary for an MVP)

  • Calendar for date picker in search fields (index/sidebar)

Add usage example for adding an API endpoint

Not sure yet which feature works best with this, but the following features should be covered:

  • Dynamic URL & filepath-based URL
  • Query values and defaults
  • Authorization scope
  • Per-Endpoint rate limiting
  • Caching
  • Pub/Sub model

Required for #49 (link)

Edit: Might be a good idea to tie this with the notification example, so people have a transition between the two and we can explain the missing gaps that pop up when only focusing on the UI or API part.

Compress favicon

/view/dist/favicon.ico is currently ~350kb in size, seems f'n unnecessary. Compression is possible but seems to require some workarounds. Think GIMP supports compressed ico files.

Add rank query param for item stats

So users could query for R0-R10 individually. Default value should be 0, so the prices appears as it should be (unranked) on the website.

defining timeend returns all offers

http://localhost:3010/warframe/v1/items/ember%20prime/statistics?timeend=1507814464
returns data for all requests, although it's actually in the future.

Add individual buying/selling statistics in stats endpoint

This will likely require breaking changes. A sensible schema may look like this:

{
  item: "itemname", // key used to be called title, but for consistency item is preferred
  buying: {
    offers: {
       count: Number,
       percentage: Number,
    },
    components: [/*...*/] // original components array, but offers {} like above instead of supply/demand
  },
  selling: {/* same as buying */},
  combined: {/* same as original, but no supply/demand keys, use offers instead, offers.percentage is 100%*/}
}

Might have missed something, so it's best to look at other possibilities too.

Write unit tests

Required unit tests:

  • Requests
  • Item list
  • Statistics
    • General stats
    • Anti spoofing
    • Ignoring / Extreme behaviour

On startup index checking

Add a check on startup that verifies all necessary mongoDB indexes and creates them if they don't exist.

Timeout errors to Warframe's CDN

Caused in in /warframe/v1/game/updates due to lost connection (socket hang up)
Solution: Use request package instead of http (has built-in workaround)

Re-implement supply/demand field

API data already exists, just need to implement the Vue components based on old designs. (PM me for designs, can't access them at this time)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.