Converts database form gni to gnindex
To connect to gni database it needs the following environment variables:
DB_USER
: mysql user for gni database
DB_PASSWORD
: mysql user password
DB_HOST
: host where database is located
DB_PORT
: database port (usually 3306)
DB_DATABASE
: database name (usually gni)
WORKERS_NUMBER
: Number of workers running concurrently
PARSER_URL
: A URL to a gnparser http service
DB_USER=root
DB_PASSWORD=secret
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=gni
WORKERS_NUMBER=4
PARSER_URL="http://parser.globalnames.org/api"
For exporting data to gnindex postgres database you need the following env variables:
GNINDEX_HOST
: Postgres host
GNINDEX_PORT
: Postgres port
GNINDEX_USERNAME
: Postgres user
GNINDEX_PASSWORD
: Postgres password
CANONICAL_DIR
: Directory where to put text files with canonical names. It should end with '/'
- Compile app
go buildto compile with build time and git hash use
go build -ldflags "-X main.buildstamp=`date -u '+%Y-%m-%d_%I:%M:%S%p'` \
-X main.githash=`git rev-parse HEAD | cut -c1-7`"
- Move binary file to
/usr/local/bin
sudo mv gnidump /usr/local/binGo to scripts directory and run
./dump
./restore
To see version run gnidump version
gnidump's People
Contributors
Watchers
gnidump's Issues
Add canonical with rank
Add parsing_quality field to name_strings
the field will allow to add parser quality to the score
As a User I want to know when a data source was updated
Before making a dump of data_sources table update updated_at field to reflect a timestamp of when
this data_source was imported last time
As a User I want to know when the program was built and what is the git hash
this can be done using -X flag for compiler that allows to substitute variables in the code.
Ask if restore happens for the database on the right computer
It is easy now to make a mistake and start upgrading a wrong database. We need to ask if user agrees to change this particular db.
change parser to gnparser written in Go
Improve JSON parsing
Use types to make JSON parsing better
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.