Comments (7)
It's likely to little memory. Imposm makes random access to the diff and cache files and each diff operation will likely hit the disk if these files do not fit in memory (cached by the OS). How large are these files for a Europe import?
You should see high IO load during the diff import (watch wa% in top).
from imposm3.
The dif files for europe are about 30 MB.
The database in postgis containing the information is (configurated by my mapping file):
osm_roads is about 16 GB;
osm_industrial 0,1 GB;
osm_nature 5 GB;
osm_water 1.5 GB;
osm_residential 0.8 GB;
Should I have then > 24 GB ram for faster processing ? Or should I use a SSD ?
from imposm3.
I was talking about the cache files in /home/xxx/cachedir and /home/xxx/diffdir, but yes you will need IO (or free RAM for caching) for the database as well.
8GB is definitely to small for Europe. With DB and caches you will have more then 32 GB. SSD will help a lot if you don't have enough RAM.
from imposm3.
ok. Its about 24 GB in the cache files en diffdir.
from imposm3.
Or use an other machine with 32GB for running imposm3 en keep the one with 8GB with the db on it ?
from imposm3.
Is postgis database still optimized after installing diffs?
from imposm3.
@tartakynov That's not really related to this issue, but the clustering degrades over time.
from imposm3.
Related Issues (20)
- Hangs on startup on arm64 Mac HOT 1
- Dockerfile example HOT 2
- Country border The Netherlands not loaded HOT 3
- Coordinates in Imposm dataset does not match those in original data HOT 5
- support for optional jsonb_tags instead of hstore HOT 2
- support for optional jsonb_tags instead of hstore
- support for optional jsonb_tags instead of hstore
- How to append data to PostgreSQL?
- Adding specific `AND` filters on tags HOT 1
- Writting osm data to postgresql sometimes get error HOT 1
- Question - adding static data
- Error while reading osm.pbf to cache in ubuntu 20.04 in WSL HOT 1
- Undesired behaviour when SQL error is experienced updating generalized tables HOT 1
- Error message when importing diff HOT 3
- hstore_tags include by mask HOT 2
- Does this module support cross compiling?
- Bug: PG 15 - pq: current transaction is aborted, commands ignored until end of transaction block HOT 12
- other_tags in JSON?
- Cannot generate expire tiles list when using undocumented option IMPOSM3_SINGLE_DIFF
- Generalized relation_member tables are loosing records during update
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from imposm3.