Hi @Noctem

I ran: pip3 install mysqlclient but got:

_mysql.c(29): fatal error C1083: Cannot open include file: 'my_config.h': No such file or directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit

status 2

I am using MariaDB as the DB, hope that is OK. I have tried the above on two different computers.

Still have an issue where workers overly concentrate in one or two areas of the map:

http://i.imgur.com/Vhs7eKc.jpg

different area, same issue
http://i.imgur.com/sHsDZib.png

I decided to install this on my linux server instead of my computer (OS X) but now Im getting another error which I try to "import db"

lots of traceback and then the subject error

OSError: Could not find lib geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so'].

I installed geos, but that didn't work

Looking at the wiki and seeing the list of status outputs for actions, operations and exceptions is perfect. Is there any way to get the complete listing? for example S and Q are not listed but seen often.

Running on windows 12 R2 (if this is needed.)
python --version = Python 3.6.0
pip3 --version = pip 9.0.1

I dont know if is a bug or something in my cfg,but i have several works like 20% outside of my location http://prntscr.com/dwunha

My cfg:

http://pastebin.com/raw/AfqbqhtT

Since Im not a Python user, any ideas on why Im getting this error when trying to "import db" according to your directions? This is in OS X.

Traceback (most recent call last):
File "", line 1, in
File "/Users/pokemongo/Sites/git-repositories/pokeminer/db.py", line 14, in
import utils
File "/Users/pokemongo/Sites/git-repositories/pokeminer/utils.py", line 13, in
from pgoapi import utilities as pgoapi_utils
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/pgoapi/init.py", line 50, in
from pgoapi.pgoapi import PGoApi
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/pgoapi/pgoapi.py", line 37, in
from pgoapi.auth_google import AuthGoogle
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/pgoapi/auth_google.py", line 32, in
from gpsoauth import perform_master_login, perform_oauth
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/gpsoauth/init.py", line 4, in
from . import google
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/gpsoauth/google.py", line 4, in
from Cryptodome.PublicKey import RSA
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/PublicKey/RSA.py", line 79, in
from Cryptodome.IO import PKCS8, PEM
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/IO/PKCS8.py", line 73, in
from Cryptodome.IO._PBES import PBES1, PBES2, PbesError
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/IO/_PBES.py", line 44, in
from Cryptodome.Cipher import DES, ARC2, DES3, AES
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/Cipher/init.py", line 78, in
from Cryptodome.Cipher._mode_ecb import _create_ecb_cipher
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/Cipher/_mode_ecb.py", line 46, in
"""
File "/Users/pokemongo/Sites/git-repositories/pokeminer/venv/lib/python3.5/site-packages/Cryptodome/Util/_raw_api.py", line 168, in load_pycryptodome_raw_lib
raise OSError("Cannot load native module '%s'" % name)
OSError: Cannot load native module 'Cryptodome.Cipher._raw_ecb'

Address to use for manager, leave unset or set to None if you're note sure.

#MANAGER_ADDRESS = r'\.\pipe\pokeminer' # must be in this format for Windows
MANAGER_ADDRESS = 'pokeminer.sock' # the socket name for Unix systems
MANAGER_ADDRESS = ('127.0.0.1', 5005) # could be used for CAPTCHA solving and live worker maps on remote systems

i cant reach page http://127.0.0.1:5005/ page blanc "Failed to load resource: net::ERR_INVALID_HTTP_RESPONSE

I can't get the webhooks to work. I've tried adding a slack and a discord webhook to the config. the console output from scan.py says it's sent alerts, but nothing showed up in the channels I set the webhooks up for.

I'm trying to run web.py. However, I keep getting this error:

OSError: [WinError 10013] An attempt was made to access a socket in a way forbid
den by its access permissions

I'm not sure if this is a system problem, or a Monocle problem. Any help?

EDIT: Here is the code i'm using:

python web.py -H 127.0.0.1 -P 5000

I have installed Python 3.6 32 bit
I have installed requirements.
When running comman py -3 create_db.py, I am getting the following:

Traceback (most recent call last):
File "create_db.py", line 3, in
from pokeminer import db
File "C:\Users\Lars\Desktop\pokeminer\pokeminer\db.py", line 14, in
from . import utils
File "C:\Users\Lars\Desktop\pokeminer\pokeminer\utils.py", line 25, in
from . import config
ImportError: cannot import name 'config'

Please help, as I am a rookie on Python

• rm db.sqlite
• set new region in config.py
• rm pickles/spawns.pickle
• rm pickles/cells.pickle
• rm pickles/ranking.pickle

# python scan.py --bootstrap 2>>error.log
PokeMiner running for 0:00:00.022520
Known spawns: 0, unknown: 0
10 workers, 3 threads, 0 coroutines

Seen per worker: min 0, max 0, med 0
Visits per worker: min 0, max 0, med 0
Visit delay: min 0.0, max 0.0, med 0.0
Speed: min 0.0, max 0.0, med 0.0
Extra accounts: 19, CAPTCHAs needed: 0

Pokemon found count (10s interval):


Visits: 0, per second: 0.00
Skipped: 0, unnecessary: 0

Workers without sightings so far:
0, 1, 2, 3, 4, 5, 6, 7, 8, 9

I I I I I I I I I I
Exiting, please wait until all tasks finish
Killing workers.
Done.

Hello,thanks for you great job but i have same issues:
Now only this part make me nuts.
:/home/pokeminer# python3 worker.py
Traceback (most recent call last):
File "worker.py", line 21, in
class Worker:
File "worker.py", line 24, in Worker
network_executor = ThreadPoolExecutor(config.NETWORK_THREADS)
AttributeError: module 'config' has no attribute 'NETWORK_THREADS'

THIS IS FIXED :)
and if i try to run : python3 web.py
Traceback (most recent call last):
File "web.py", line 81, in
manager.connect()
File "/usr/local/lib/python3.6/multiprocessing/managers.py", line 489, in connect
conn = Client(self._address, authkey=self._authkey)
File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/usr/local/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
UPDATE : I fix python3 web.py by edit in db.py
Line 19: DB_ENGINE = 'mysql://user:password@localhost/pokeminer'
Thanks

the game sometimes collects the rewards while still in game

I would love http://Boxcar.io notifications. Free for iOS users and works really well.

Run it on windows 7.
Use 10 ptc accounts + hashing key + 2capcha.
When I run "python3 scan.py" everything looks good it about 5 minutes it writes that Pokemon found count ~ 500.
But when I run "python3 web.py" there are only about 10 of map. I check both "Pokemon" and "Trash".
If i try to see report - the same small numbers there. "During that session, 17 Pokemon have been seen on an area of about 2 square km." But 705 pokemon found printed in console.
Maybe I do something wrong with config?

I have them like 1-3 times / minute with 25 Workers think that isn't good

I ran python3 scan.py and python3 web.py -H 0.0.0.0 -P 80

I can't see any pokemons on the map.
Pokemon found count (10s interval):
42 42 44 49 49 54 57 57 66 66

messages like that have increased

Hey Noctem, do you plan on adding 2captcha support soon? I tried to implement it myself but can't seem to debug it using the "logger.info" method. I'm going to keep trying but I was wondering if you plan on implementing it yourself.

Thanks for your work!

I tried to run it for the first time and it throws this in the console

Traceback (most recent call last):
  File "C:\Users\David\Desktop\pokeminerNcotme\landmarks.py", line 72, in query_location
    geo = nom.geocode(query=query, geometry='geojson').raw
AttributeError: 'NoneType' object has no attribute 'raw'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "worker.py", line 22, in <module>
    import config
  File "C:\Users\David\Desktop\pokeminerNcotme\config.py", line 187, in <module>
    LANDMARKS.add('Rice Eccles Stadium', hashtags={'Utes'})
  File "C:\Users\David\Desktop\pokeminerNcotme\landmarks.py", line 192, in add
    landmark = Landmark(*args, **kwargs)
  File "C:\Users\David\Desktop\pokeminerNcotme\landmarks.py", line 27, in __init__
    self.query_location(query)
  File "C:\Users\David\Desktop\pokeminerNcotme\landmarks.py", line 75, in query_location
    raise FailedQuery('Query for ' + query + ' did not return results.')
landmarks.FailedQuery: Query for Rice Eccles Stadium CiudadDelCarmen did not return results.

Config.py

# It is recommended to store the LANDMARKS object in a pickle to reduce startup 
# time if you are using queries.
LANDMARKS = Landmarks(query_suffix=AREA_NAME)

# Landmarks to reference when Pokémon are nearby
# If no points are specified then it will query OpenStreetMap for the coordinates
# If 1 point is provided then it will use those coordinates but not create a shape
# If 2 points are provided it will create a rectangle with its corners at those points
# If 3 or more points are provided it will create a polygon with vertices at each point
# You can specify the string to search for on OpenStreetMap with the query parameter
# If no query or points is provided it will query with the name of the landmark (and query_suffix)
# Optionally provide a set of hashtags to be used for tweets about this landmark
# Use is_area for large neighborhoods or regions
# When selecting a landmark, non-areas will be chosen first if any are close enough
# the default phrase is 'in' for areas and 'at' for non-areas, but can be overriden for either.

# since no points or query is provided, the names provided will be queried and suffixed with AREA_NAME
LANDMARKS.add('Rice Eccles Stadium', hashtags={'Utes'})
LANDMARKS.add('the Salt Lake Temple', hashtags={'TempleSquare'})

# provide two corner points to create a square for this area
LANDMARKS.add('City Creek Center', points=((40.769210, -111.893901), (40.767231, -111.888275)), hashtags={'CityCreek'})

# provide a query that is different from the landmark name so that OpenStreetMap finds the correct one
LANDMARKS.add('the State Capitol', query='Utah State Capitol Building')

## area examples ##
# query using name, override the default area phrase so that it says 'at (name)' instead of 'in'
LANDMARKS.add('the University of Utah', hashtags={'Utes'}, phrase='at', is_area=True)
# provide corner points to create a polygon of the area since OpenStreetMap does not have a shape for it
LANDMARKS.add('Yalecrest', points=((40.750263, -111.836502), (40.750377, -111.851108), (40.751515, -111.853833), (40.741212, -111.853909), (40.741188, -111.836519)), is_area=True)

Argument action with value 1 unknown inside encounter_tutorial_complete_message (Exception: 'EncounterTutorialCompleteMessage' object has no attribute 'action')

pycairo wont install. I keep getting error message "could not find version that satisfies requirement pycairo (from version: ) no matching distribution for pycairo"

i'm constantly using up my 1000rpm hash key. since it's not possible to upgrade them, it would be great if we could add another one. it's cheaper too then buying bigger ones.

C:\Monocle>py C:\Monocle\create_db.py
Traceback (most recent call last):
File "C:\Monocle\create_db.py", line 3, in
from monocle import db
ModuleNotFoundError: No module named 'monocle'

New to all this, hope I don't other you guys too much :)

Sometimes scan.py is segfaulting - this only seams to happen when there still are mysterys
after segfaulting there still is one process with a thread left over
there are no extra error messages or something like that

here some times:
Tue 17 Jan 09:20:06 CET 2017
Tue 17 Jan 09:58:31 CET 2017
Tue 17 Jan 10:51:27 CET 2017
Tue 17 Jan 11:38:59 CET 2017
Tue 17 Jan 12:06:26 CET 2017

I am using MySQL for the database and I have

SPAWN_ID_INT = False

and I am seeing a lot of problems in my scan.log like these:

SELECT max(seen_range)\n FROM mystery_sightings\n WHERE spawn_id = [REMOVED, but it's alphaneumeric]\n AND first_seen > 1484092800

Which because there are no quotes around the spawn_id it is failing.. since it's not a number...

my scan.log has lots of these inside of it..

[2017-01-23 15:46:03,975][ ERROR][worker-14] A wild exception appeared!
Traceback (most recent call last):
File "/path/to/pokeminer/pokeminer/worker.py", line 533, in visit
return await self.visit_point(point, bootstrap=bootstrap)
File "/path/to/pokeminer/pokeminer/worker.py", line 630, in visit_point
responses = await self.call(request)
File "/path/to/pokeminer/pokeminer/worker.py", line 475, in call
if not response:
UnboundLocalError: local variable 'response' referenced before assignment

I think is not a victory for mystic? Mystic and Valor have both 22 gyms. And Valor have more prestige.

seams like it failes to read the items correctly(?)

(venv) bash-3.2$ python3 ./web.py
Traceback (most recent call last):
File "./web.py", line 90, in
manager.connect()
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/managers.py", line 489, in connect
conn = Client(self._address, authkey=self._authkey)
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/connection.py", line 487, in Client
c = SocketClient(address)
File "/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory

I can't tell which file or directory it is looking for, since python error messages are not my thing.

so that we find more the rare Pokemon

I don't really yet understand the scanner algo so I can't do this my self currently

Feature request. Some kind of filtering. To hide items (and un-hide them) while live mapping. not using the trash_ids but in conjunction with that as well. My map is kinda cluttered and I'm not savvy in coding enough to work that out.

To remove the need to edit config, stop and start scan.py. could possibly integrate into the database somehow (I'm using MySQL)

The commented line in the config.example.py is as follows, that raises an error when the user uncomments it and tries to use the landmark feature since it can't import landmarks.

from landmarks import Landmarks

it needs to be the following:

from monocle.landmarks import Landmarks

I have cloned the New Monocle.
Now the Scanner has Errors in the Logs Like This.

[2017-01-28 05:05:27,601][   ERROR][eventloop] A wild exception appeared!
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/sslproto.py", line 628, in _process_write_backlog
    ssldata = self._sslpipe.shutdown(self._finalize)
  File "/usr/lib/python3.5/asyncio/sslproto.py", line 151, in shutdown
    raise RuntimeError('shutdown in progress')
RuntimeError: shutdown in progress
[2017-01-28 05:05:27,603][   ERROR][eventloop] {'message': 'Fatal error on SSL transport', 'protocol': 
	<asyncio.sslproto.SSLProtocol object at 0x7fbb80d5f240>, 'exception': RuntimeError('shutdown in progress',), 'transport': <_SelectorSocketTransport closing fd=17 read=idle write=<idle, bufsize=0>>}

SELECT
            fs.fort_id,
            fs.id,
            fs.team,
            fs.prestige,
            fs.guard_pokemon_id,
            fs.last_modified,
            f.lat,
            f.lon
        FROM fort_sightings fs
        JOIN forts f ON f.id=fs.fort_id

            WHERE (fs.fort_id, fs.last_modified) IN (
                SELECT fort_id, MAX(last_modified)
                FROM fort_sightings
                GROUP BY fort_id
            )

is taking a very long time...

select count(*) from fort_sightings; returns 12813
select count(*) from forts; returns 1221

okay, i see, you store every sighting of those gyms, you could display a history of those. one solution would be to only keep the most current sighting (unique index on fort_sightings.fort_id and REPLACE instead of INSERT). i'll see if adding an index or two can help.

PS: it looks like it can be rewritten without a subquery. i'm at it

PS2: this one executes immidiately and should do the same:
SELECT fs.fort_id, fs.id, fs.team, fs.prestige, fs.guard_pokemon_id, MAX(fs.last_modified), f.lat, f.lon FROM fort_sightings fs JOIN forts f ON f.id=fs.fort_id GROUP BY fs.fort_id;

the status bar screen has many important statistics that are not visible when running with --no-status-bar.

i'm running it like this and barely get any useful info:
nohup python3.6 scan.py --no-status-bar --log-level INFO > /tmp/pm2 &

it would be great if the stats would appear every few minutes. i'm mostly interested in number of captchas, number of active accounts, skipped spawnpoints, average hash_key usage, known/unknown spanws.

@nickmhc on Discord
pasted Bossland hash key straight into the config.py so it's guaranteed accurate, but scan.py is not using it

Also changed map_start and map_end and those changes aren't being recognized

Not really an issue, but this seems to be the easiest way to communicate with you.
I think there are many devs (including me) that would like to contribute, but there isn't a clear vision or path. What are some features or annoyances that you'd like to be addressed? I'd like to get some experience under my belt while contributing to this project

any encounter_id which belongs to a mystery found in nearby can be used to update the last_seconds

currently accounts are rotated by replacing the account with the lowest sighting rate.
If you now have a scanner running over a city + rural area this means that the account further out get replaced regularly but the inner ones don't => the will be captchad

It would be grate if someone would change this.
Maybe replace the lowest + longest.
Additionally one may add a ROTATE_INTERVAL option, as you may like to rotate more often if you are using more workers.

should be currently 3 minutes

the radiuses of the workers are overlaping a lot in scans. like here:
http://upload.festzeit.ch/upload/pn.jpg

here the live map:
http://94.16.174.3:81/

that area could be easily covered with only a third of the accounts. my ideas:

• one could look for nearby workers and check if they are going to cover the point we are interested in in the next few minutes (configurable). if yes, we can go do other stuff.
• arrange possible worker-locations on a grid, like speedscan in pokemongo-map. with more overlap then needed, to allow "natural" movement and jitter. then you can have a configurable variable that says that you can visit each gridpoint at most every 5 minutes or so. i dont care if i have 25 or 30 minutes to get e pokemon.
• have a second configurable grid (next to the bootstraping one) where the workers always have to stay inside. i have the feeling many workers are traveling towards a few hotspots and leave other areas behind. maybe assign the number of workers per cell by number of spawnpoints inside.
• limit the density of workers. allow only X workers to be in a radius Y of another worker. something like that.

Hello

I have a 74km2 polygon to scan with 280 worker (14x20 gird).

But my Problem is, that many worker are scanning on the same place. Not all on the same but 3-4 groups with many workers and some area with no worker. Is ther a problem? Did i need more workers?

see all of my workers are just scanning the same area. And its been like this for a long time

And the Pokemon Found count is increasing for no reason. It increases by a constant amount every 20 seconds almost.

And workers are insanely slow even though the speed i've set is 22.5

After running monocle for a few days, I've found that I can run about 3 accounts per public ip address without getting captchas.

If I configure a bunch of socks proxies in the config file, will it spread the workers out evenly among them?

F:\pokeminer>python -i
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 25 2016, 22:01:18) [MSC v.1900 32 bit (In
tel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import db
Traceback (most recent call last):
File "", line 1, in
File "F:\pokeminer\db.py", line 12, in
import utils
File "F:\pokeminer\utils.py", line 4, in
import polyline
ImportError: No module named 'polyline'
db.Base.metadata.create_all(db.get_engine())
Traceback (most recent call last):
File "", line 1, in
NameError: name 'db' is not defined
db.Base.metadata.create_all(db.get_engine())
Traceback (most recent call last):
File "", line 1, in
NameError: name 'db' is not defined

what happen with me ?