tumi8 / hloc Goto Github PK

Hi,
I was following through the steps in README (on branch tma17) and got the .found file generated after the "Find" step. I'm not sure how I should interpret the information from the .found file. Specifically how should I generate the geolocation results in terms of Lat Lon from the result.
Thank you very much.

I'm trying to get the current version of HLOC up and running. I've followed the steps for acquiring the required location codes files. I looked at the example-initial-db-setup file for how to continue and came up with the following command (some of the file names are slightly different, but they all exist and should be correct, I believe).

python3 -m hloc.scripts.codes_parser --database-name hloc -ao location-data/pages_offline/ -le "location-data/locodePart{}.csv" -c location-data/clli.txt -g location-data/cities1000.txt -e location-data/iata_metropolitan.txt -m 100 -p 100000 -l ./codes-parsing -ll DEBUG -d

However, this command fails with the following error (I've truncated down to the relevant part):

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "states" does not exist
LINE 1: INSERT INTO states (name, iso3166code) VALUES ('china', 'cn'...

[SQL: INSERT INTO states (name, iso3166code) VALUES (%(name)s, %(iso3166code)s) RETURNING states.id]
[parameters: {'name': 'china', 'iso3166code': 'cn'}]

I've created the hloc database, using the suggested username and password, but it looks like the tables also need to be created as well. I tried searching/grep'ing through the repository to look for any sort of schema but as far as I could tell I couldn't find any for this table (tried searching the repo for "CREATE TABLE" and ".Table(" for sqlalchemy). I'm wondering if there might be other tables that need to be created, as well?

Hi, thank you very much for your open code!

I try to use it, but i get an error that relation "airport_infos" does not exist. I have collected all data and renamed it correctly.
This error happened when i executed script example-inital-db-setup.sh.

Here is the screenshot.

I think i have missed some steps about database but i dont know how can i do now.
Thank you in advance.

When running the example-initial-db-setup script:

Error
File "/Users/colin/dev/ixmaps/hloc/hloc/scripts/codes_parser.py", line 406, in get_geo_names
with open(file_path, encoding='utf-8') as geoname_file:
FileNotFoundError: [Errno 2] No such file or directory: 'location-data/cities1000.txt'

Method description from get_geo_names
"""Get the geo names from file ./collectedData/cities1000.txt"""

I assume the method description is wrong. Where can I obtain cities1000.txt? Is that what you are referring to here:
"We produce a daily export with all location hints for all domains with the minimal RTT measurement to the corresponding IP address. The file contains the following columns (the csv column title is in brackets):"

Hi and thanks for this great project.

I have been trying for the use the code in the past few weeks. So far, I managed to generate the SQL database with the all the location code entries. I have also managed to put the IP addresses and dns domains in the SQL database. (I added a few screenshots). However whenever I run validate.py with the following arguments :
python3 -m hloc.scripts.validate --number-processes 10 --ripe-request-limit 30 --ripe-request-burst-limit 50 --measurement-limit 100 --allowed-measurement-age 15552000 --buffer-time 0 --measurement-strategy aggressive --api-key 398a1e61-b3db-4b6e-b3b3-664ba5259025 --include-ip-encoded --use-efficient-probes --probes-per-measurement 1 --measurement-packets 1 --database-name test --endless-measurements --random-domains -l logs/validate-multi.log -ll DEBUG -dpf
The code appears to be scheduling measurements without ever getting matches. An example of domain which is part of my DB would be :
16 | ae-2-2.bar4.Warsaw1.Level3.net | 4.69.159.46 | | valid

and in the log file I see:
[20.05 15:47:58][check][DEBUG][domain_checking_6][Thread-24] validate.py:583 validating domain ae-2-2.bar4.Warsaw1.Level3.net
[20.05 15:47:58][check][DEBUG][domain_checking_7][Thread-21] validate.py:623 0 matches before filter
[20.05 15:47:58][check][DEBUG][domain_checking_8][Thread-26] validate.py:583 validating domain 197-80-5-57.jhb.mweb.co.za
[20.05 15:47:58][check][DEBUG][domain_checking_9][Thread-22] validate.py:593 number of saved results 0

I was wondering why it is not finding anything for no one of my domains

Thanks in advance for your kind answer.

Hello,

I haven't been able to read all the source code for your project yet, I'm still on your pdf (tma2017_paper8.pdf).

While reading your readme I saw that you mentioned "Unfortunately we could not find a public available CLLI list.".

Looking at wikipedia (https://en.wikipedia.org/wiki/CLLI_code)

I grab some examples and typing on google:
"MALTON22CG1 DLLSTXRNDS1 OTWAON080MD SWASONXTSG1 filetype: txt"

Google return: http://wedophones.com/Manuals/TelcoData/clli-lat-lon.txt

Is this what you are looking for?
There are also other different formats here: http://wedophones.com/Manuals/TelcoData/

Best regards,

Hi, Hope you find well can you please guide me to install this software in my pc, i have ubuntu version 16 i run the following command (pip install -r requirements.txt) your guidance is required in this matter.
Thank you.