World-wide support in MapOSMatic, issues

By far the most important feature request we got, and the one we also consider the most important is to get world-wide support in MapOSMatic, and not only support for France. A large number of people volunteered to help us in translating and adapting the necessary parts of MapOSMatic. However, at the moment, we haven’t reached the point where the issue is on country-specific adaptations and translations.

Two weeks ago, David Decotigny started the import of planet.osm.bz2 into a new PostgreSQL database, so that we could start experimenting world-wide support. The import took around 4-5 days, and completed with a 75 GB PostgreSQL database.

Then, David worked on getting daily diffs applied to the database, so that we can keep it up-to-date with the contributions made on OpenStreetMap. Unfortunately, applying each daily diff takes around 9 hours on our server, which means that 9 hours per day, our server would be busy updating the database, something we don’t consider sustainable. Moreover, we tried to use MapOSMatic concurrently with the daily diff update and it did not work. See David’s email on the MapOSMatic list for further details.

At the moment, we are considering two solutions :

  • Having two servers, used alternatively. But that means paying two dedicated servers all year long, which cost quite a lot for a non-profit project like MapOSMatic, and is quite annoying to maintain.
  • Understand why the daily diff application process with osm2pgsql is so slow, and see if optimizations are possible. These optimizations would benefit to the entire OpenStreetMap community by allowing more people to play with OpenStreetMap data. Anyone having some PostgreSQL optimizations skills and some free time would definitely help in this area. I think we can provide an access to the server and database for testing purposes to motivated people.

Until this issue has been solved, we unfortunately won’t really be able to start with adding country selection support in MapOSMatic. Some people suggested to restrict ourselves to Europe, but Europe is only a third (in size) of the whole world, and no diffs are provided for Europe (even though they could probably be generated using osmosis). And getting to the whole world is more exciting !

Comments are closed.