Giter VIP home page Giter VIP logo

server's Introduction

FA Forever - Server

Build Status codecov Codacy Badge docs license python

This is the source code for the Forged Alliance Forever lobby server.

Overview

The lobby server is the piece of software sitting at the very core of FAF, enabling players to discover and play games with each other. It is a stateful TCP server written in asyncio and implements a custom TCP protocol for communicating with clients. The main responsibilities of the lobby server are:

  • To manage the lifecycle of joining games

    (Note that Forged Alliance uses a distributed peer-to-peer networking model, so the simulation happens entirely on the player's machines, and NOT on any server)

  • To facilitate initial connection establishment when players join a game

  • To maintain a list of online players

  • To perform rating calculations and updates

Support development

Post a bounty on Issue Hunt. You can reward and financially help developers who work on your issue.

Issue hunt

Major Software Dependencies

The lobby server integrates with a few external services and has been tested with the following versions:

  • MariaDB 10.6
  • (optional) RabbitMQ 3.9

Contributing

Before opening a pull request, please take a moment to look over the contributing guidelines.

Setting up for development

For detailed instructions see the development guide.

Quickstart

This section assumes you have the necessary system dependencies installed. For a list of what those are see the development guide.

  1. Start up an instance of the FAF database. This is required to run the unit tests and development server.
$ git clone https://github.com/FAForever/faf-stack.git
$ cd faf-stack
$ ./scripts/init-db.sh
  1. Install the project dependencies with pipenv
$ pipenv sync --dev
  1. Run the unit tests or development server
$ pipenv run tests
$ pipenv run devserver

Network Protocol

NOTE: This section of the README is outdated. The QString based message format has been deprectaed and will be replaced by a UTF-8 json + newline format in version 2. Many commands are missing from here. For a more complete list see https://faforever.github.io/server/.

The protocol is mainly JSON-encoded maps, containing at minimum a command key, representing the command to dispatch.

The wire format uses QDataStream (UTF-16, BigEndian).

For the lobbyconnection, each message is of the form:

ACTION: QString

With most carrying a footer containing:

LOGIN: QString
SESSION: QString

Incoming Packages

Mod Vault
  • (deprecated) {command: modvault, type: start}: show the last 100 mods
  • (deprecated) {command: modvault, type: like, uid: <uid>}: check if user liked the mod, otherwise increase the like counter
  • (deprecated) {command: modvault, type: download, uid: <uid>}: notify server about a download (for download counter), does not start the download
Social
  • {command: social_add, friend|foe: <player_id>}: Add a friend or foe
  • {command: social_remove, friend|foe: <player_id>}: Remove a friend or foe
Avatar
  • {command: avatar, action: list_avatar}: Send a list of available avatars
  • {command: avatar, action: select, avatar: <avatar_url>}: Select a valid avatar for the player
ICE Servers
  • (deprecated) {command: ice_servers}: Send ICE TURN/STUN servers - Returns: {command: ice_servers, : <ice servers>, date_created: <date token was created in ISO 8601 format>, ttl: <ttl in seconds>}

Parties

  • {command: invite_to_party, recipient_id: <...>}: Invite this player to a party
  • {command: accept_party_invite, sender_id: <...>}: Accept the party invite from the given player
  • {command: kick_player_from_party, kicked_player_id: <...>}: Kick a player from a party you own
  • {command: leave_party}: Leave the party you are currently in
Misc
  • (deprecated) {command: ask_session}: response with a welcome command and a valid session (can be delayed)
  • {command: hello, version: <...>, login: <...>, password: <...>, unique_id: <...>, (session: <...>)}: Log in to the server

Stream (Deprecated)

The stream API is deprecated, but currently the following message types are supported:

  • PING: response with a PONG
  • PONG: internal state changed to ponged

server's People

Contributors

0x647262 avatar 46bit avatar ahsanbagwan avatar andrewmiko avatar askaholic avatar b0ns avatar baterflyrity avatar blackyps avatar brutus5000 avatar cheyans avatar chriskitching avatar cleborys avatar crotalus avatar duk3luk3 avatar eforgacs avatar eoinnoble avatar gatsik avatar geosearchef avatar iannaughton avatar idragonfire avatar imangerah avatar jeroendedauw avatar katharsas avatar kaukahan avatar martijnpieterse avatar micheljung avatar rackover avatar sheeo avatar sheikah45 avatar spikey84 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

server's Issues

Unable to Search Mod Vault

When performing a server search the lobby just hangs with the small window titled Connecting to Statistics Server poping up with the message Connecting to Server.

In my forever.log, these are the last lines that deal with the query:

2015-02-20 09:08:33,342 DEBUG    faf.modvault         Updating visibilities with sort 'date' and visibility 'all'
2015-02-20 09:08:56,108 ERROR    faf.secondaryServer  Operation timed out while waiting for info.

The ‘ERROR’ line just constantly repeats and in a short time just make the log file over 2GB.

Lobby version: 0.10.122

The last time I had tried this search and was certain it was successful was on Saturday February 7, 11:47 pm EST (Sunday February 8, 4:47 am GMT). This was the last time I had downloaded a mod from the vault.

The first time that I had run into this issue was Wednesday, February 18, 11:00am EST (Wednesday, February 18, 4:00pm GMT).

Looking at the Commits history, all the changes during that time period were under Server, faf.webapp, and fa.

Edit:

The search function works when I enter some text, but it hangs when I leave the text field blank. That is I want the query to return the latest mods uploaded to the vault.

Make database access not block the event loop

Currently, all database access that uses the QSql namespace classes block the entire eventloop (Both with and without using the asyncio module).

  • Use aiomysql in:
    • Game
    • GameConnection
    • LobbyConnection
  • Port the rest of the server to asyncio so it can gain these improvements too

Coop: 'LobbyConnection' object has no attribute 'ip'

I observed this multiple times now, alyways in "coop_list"

DEBUG:server.lobbyconnection.LobbyConnection:<<: {'command': 'coop_list'}
ERROR:server.lobbyconnection.LobbyConnection:'LobbyConnection' object has no attribute 'ip'
Traceback (most recent call last):
  File "/code/server/lobbyconnection.py", line 134, in on_message_received
    if not self.ensure_authenticated(cmd):
  File "/code/server/lobbyconnection.py", line 120, in ensure_authenticated
    self.abort("Message invalid for unauthenticated connection: %s" % cmd)
  File "/code/server/lobbyconnection.py", line 112, in abort
    self._logger.warning("Aborting %s. %s" % (self.ip, logspam))
AttributeError: 'LobbyConnection' object has no attribute 'ip'
ERROR:server.servercontext.ServerContext:'LobbyConnection' object has no attribute 'ip'
Traceback (most recent call last):
  File "/code/server/lobbyconnection.py", line 134, in on_message_received
    if not self.ensure_authenticated(cmd):
  File "/code/server/lobbyconnection.py", line 120, in ensure_authenticated
    self.abort("Message invalid for unauthenticated connection: %s" % cmd)
  File "/code/server/lobbyconnection.py", line 112, in abort
    self._logger.warning("Aborting %s. %s" % (self.ip, logspam))
AttributeError: 'LobbyConnection' object has no attribute 'ip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/code/server/servercontext.py", line 71, in client_connected
    await connection.on_message_received(message)
  File "/code/server/lobbyconnection.py", line 171, in on_message_received
    self.abort("Error processing command")
  File "/code/server/lobbyconnection.py", line 112, in abort
    self._logger.warning("Aborting %s. %s" % (self.ip, logspam))
AttributeError: 'LobbyConnection' object has no attribute 'ip'

ranked game 'dead' time issue

It seems that the part responsible for deciding whether a game is ranked or not based on game time (so the N min unrakned period) has a problem.
I've played two games:
3138425 - game time: 1:35 - played disconnected so we rehosted
3138461 - game time 4:22 - another disconnect
Not sure after which one, but one of these lowered my ranking.

Leaderboard avatars not updating right

Chat excerpt:

dukedickinsson:
no golden shiny avatar for me then xD, oh well its not like i'm gonna stay in the T2 league for much longer :3
Sheeo:
You didn't get the proper league avatar dukedickinsson?
dukedickinsson:
in the leaderboards it shows me as the first of my division and first of the T2 league, yet i only get the " third of league avatar "

Asyncio TestClient

Will allow us to get rid of the transport classes I made before the asyncio switch

Agreeing to draw on ladder counts as a real draw

In ladder there is a draw button, and if you do it then it counts as a real draw. This means that people lose/gain rating from it, and win percentage. Perhaps drawing in this manner should not count at all?

BaseGame.rate_game not implemented

BaseGame defines rate_game as abstract method. Only one of the concrete derivatives actually implements it though (CustomGame). If the other derivatives require no behaviour in this method, then perhaps the base class should not specify it as abstract and instead just define it as no-op?

Rethink/remove current "smurf-prevention"

Short description: It's too much pain for the gain.

Long description:

  • Smurfs are not preventable, except players have to identify themselves personally (webcam, passport, etc) against trustful admin
  • Therefore, all currently implemented protections are ineffective
  • Since FAF is open source, it's easy as pie to circumvent the protection

You may argue that it at least prevents totally unskilled (in terms of programming) people from smurfing - and I agree. But is it worth the effort just to prevent 10 out of 20 smurfs?

Is smurfing really a problem that needs to be solved? If so, the only way to do that is having people identify themselves before they can play ladder, or before they can play at all. Effective, but not very user-friendly.

Make replay search fields wildcards

Please make player name and map name fields a wildcard search.

At the moment they are exact match, which is unpleasant because lots of map and players names are quite hard.

Ignore these youtubers in fafbot (SWTOR casters)

swcallow98 Chaotic3volution Blizofoz45 MultiAwesome123 TheDkgr4y MrSuperDjavo elitenz imsobuffed devilfiresmile666 acort14 aagrippaa suicune2001 MarcanOlsson elitenichemarketing Rayfinders TheGrahamAker DathanielTDK syfy88man KidLee3ASF swtorPaladin BobaVoll hale32225 Shotgunchief38 maxulic Jedidavie0804 Zedcreek legoboy4567 TDGMMO FluffyNinjaLlama KiethSomataw99 SideStrafe SilveliaDarkmoon FastSoulSlayer pokemonmaster9416 ArtemysKnigh

Inorrect license specification in README

The README file specifes the code is GPL v1, while the repo contains a lot of GPL v3 code.

The LICENSE file also specifies GPL 3v. Is the README simply wrong?

create a public variable called "map of the day"

the idea is to pick from a map pool of good maps a (completely or relatively) random map to send to the server and in our new lobby (and the current one if you want as well) we could display in the games tab a suggested map of the day. Thereby inciting community interaction.

Better handling of corrupt maps

Originally reported at FAForever/fa#384 by @ChrisKitching.

Currently, it is possible for maps which are corrupt to be downloaded, only to fail at load-time.

We should have a verification step when uploading maps (which we should retroactively apply to the entire map database) to avoid this. It is probably not sensible to implement this verification at load-time, for reasons of performance and simplicity (and in any case the flavour of brokenness generally doesn't prevent the selection of an alternative map, except in the case that the failure occurs at game-start-time, in which case you are hosed).

See also: FAForever/fa#351

STUN: Pending task is being destroyed

DEBUG:server.lobbyconnection.LobbyConnection:>>: {'target': 'connectivity', 'command': 'SendNatPacket', 'args': ['46.126.105.89:1024', 'Hello from 21447']}
DEBUG:server.connectivity.Connectivity:<server.connectivity.Connectivity object at 0x7f6b001d59e8> sending NAT packet 1 to 46.126.105.89:1025
DEBUG:server.lobbyconnection.LobbyConnection:>>: {'target': 'connectivity', 'command': 'SendNatPacket', 'args': ['46.126.105.89:1025', 'Hello from 21447']}
DEBUG:server.connectivity.Connectivity:<server.connectivity.Connectivity object at 0x7f6b001d59e8> sending NAT packet 2 to 46.126.105.89:1026
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/logging/__init__.py", line 984, in emit
    self.flush()
  File "/usr/local/lib/python3.5/logging/__init__.py", line 964, in flush
    self.stream.flush()
RuntimeError: reentrant call inside <_io.BufferedWriter name='<stderr>'>
Call stack:
  File "./server.py", line 77, in <module>
    loop.run_until_complete(done)
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 330, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 301, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 1198, in _run_once
    handle._run()
  File "/usr/local/lib/python3.5/asyncio/events.py", line 125, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(value)
  File "/code/server/connectivity.py", line 112, in ProbePeerNAT
    self._logger.debug("{} sending NAT packet {} to {}:{}".format(self, i, ip, port))
  File "/usr/local/lib/python3.5/logging/__init__.py", line 1267, in debug
    self._log(DEBUG, msg, args, **kwargs)
  File "/usr/local/lib/python3.5/logging/__init__.py", line 1415, in _log
    self.handle(record)
  File "/usr/local/lib/python3.5/logging/__init__.py", line 1425, in handle
    self.callHandlers(record)
  File "/usr/local/lib/python3.5/logging/__init__.py", line 1487, in callHandlers
    hdlr.handle(record)
  File "/usr/local/lib/python3.5/logging/__init__.py", line 855, in handle
    self.emit(record)
  File "/usr/local/lib/python3.5/logging/__init__.py", line 984, in emit
    self.flush()
  File "/usr/local/lib/python3.5/logging/__init__.py", line 964, in flush
    self.stream.flush()
  File "/usr/local/lib/python3.5/asyncio/tasks.py", line 92, in __del__
    self._loop.call_exception_handler(context)
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 1057, in call_exception_handler
    self.default_exception_handler(context)
  File "/usr/local/lib/python3.5/asyncio/base_events.py", line 1034, in default_exception_handler
    logger.error('\n'.join(log_lines), exc_info=exc_info)
Message: 'Task was destroyed but it is pending!\ntask: <Task pending coro=<GameConnection._handle_lobby_state() done, defined at /code/server/gameconnection.py:143> wait_for=<Future pending cb=[Task._wakeup()
]>>'
Arguments: ()
DEBUG:server.lobbyconnection.LobbyConnection:>>: {'target': 'connectivity', 'command': 'SendNatPacket', 'args': ['46.126.105.89:1026', 'Hello from 21447']}
DEBUG:server.connectivity.Connectivity:<server.connectivity.Connectivity object at 0x7f6b001d59e8> sending NAT packet 3 to 46.126.105.89:1027
DEBUG:server.lobbyconnection.LobbyConnection:>>: {'target': 'connectivity', 'command': 'SendNatPacket', 'args': ['46.126.105.89:1027', 'Hello from 21447']}

Make sure games are ranked if an ACU was killed by the enemy team

The early unrankedness feature is there to prevent early disconnects from affecting games and allowing players to opt out of rankedness in such cases.

This shouldn't apply to early rush strategies, so we must make sure to rank these games in all circumstances.

Infinite login await loop

It appears that if the uniqueID of a user mismatches, sometimes the user is stuck in an infinite login sequence.

Ladder changes

As described on forums, ladder should be changed so players do not get a map that they have played within the latest 5 games.

Crash during connectivity check

DEBUG:server.lobbyconnection.LobbyConnection:<<: {'target': 'connectivity', 'args': [6112], 'command': 'InitiateTest'}
DEBUG:server.connectivity.ConnectivityTest:Testing PUBLIC
DEBUG:server.natpacketserver.NatPacketServer:>>('46.126.105.89', 6112)/udp: Are you public? 21447
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<Connectivity.initiate_test() done, defined at /code/server/connectivity.py:83> exception=AttributeError("'NoneType' object has no attribute 'call_exception_handler'",)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/asyncio/selector_events.py", line 1029, in sendto
    self._sock.sendto(data, addr)
AttributeError: 'NoneType' object has no attribute 'sendto'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(value)
  File "/code/server/connectivity.py", line 85, in initiate_test
    result = await self._test.determine_connectivity()
  File "/code/server/connectivity.py", line 199, in determine_connectivity
    public = await self.test_public()
  File "/code/server/connectivity.py", line 216, in test_public
    await send_natpacket(self.remote_addr, message)
  File "/code/server/connectivity.py", line 23, in send_natpacket
    _natserver.send_natpacket_to(msg, addr)
  File "/code/server/natpacketserver.py", line 69, in send_natpacket_to
    self.protocol.transport.sendto(("\x08"+msg).encode(), addr)
  File "/usr/local/lib/python3.5/asyncio/selector_events.py", line 1038, in sendto
    'Fatal write error on datagram transport')
  File "/usr/local/lib/python3.5/asyncio/selector_events.py", line 584, in _fatal_error
    self._loop.call_exception_handler({
AttributeError: 'NoneType' object has no attribute 'call_exception_handler'

Download issues

People are reporting issues with downloads. They appear non-recurring.

I've fixed some issues from the nginx error.log with bad links (Some people still using wrong webroots, for one), but it wouldn't explain downloads stopping half way.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.