Extract latest planet file

General
1

Hello, we have been maintaining weekly updated RDF extracts for OHM data on https://osm2rdf.cs.uni-freiburg.de/ for many years now.

For the conversion process, we use the latest OHM planet file.

Until a few weeks ago, the latest planet file URL was stored in https://s3.amazonaws.com/planet.openhistoricalmap.org/planet/state.txt.

But this file now contains links to the latest full history file.

Is this intended? How can we reliably get the link to the latest planet dump?

1 Like
2

This is our current workaround to get the latest file URL:

curl -Ls "https://s3.amazonaws.com/planet.openhistoricalmap.org?prefix=planet/&marker=planet%2Fplanet-251127_0301.osm.pbf" | sed "s/.osm.pbf/.osm.pbf\n/g" | sed "s/.*<Key>/https:\/\/s3.amazonaws.com\/planet.openhistoricalmap.org\//g" | grep -v "<" | sort | tail -n1
3

@Rub21 any idea if this change is intentional? If so, can we set up an endpoint for the latest normal dump without history?

4

Note: I just checked this again, and https://s3.amazonaws.com/planet.openhistoricalmap.org/planet/state.txt now contains the link to the normal dump again. Either this has been fixed, or this file is erroneously used to write the latest URL of both the latest normal dump, and the latest full history dump.

5

Looks like it’s supposed to be two different files:

1 Like
6

@Patrick_Brosi - thank you for stopping by and letting us know about this funkiness. Vielen dank to you and the rest of the team at Freiburg for all you do. Please be sure to let us know if there’s anything else we can be doing to help you or your users solve their problems! :folded_hands:

7

Update: at this moment, the file contains the link to fulll history file again.

curl https://s3.amazonaws.com/planet.openhistoricalmap.org/planet/state.txt
http://planet.openhistoricalmap.org.s3.amazonaws.com/planet/full-history/planet-history-251204_0501.osm.pbf

So I still very much suspect that at some point, the content that should be written into https://s3.amazonaws.com/planet.openhistoricalmap.org/planet/full-history/state.txt is erroneously written to /planet/state.txt

8

I think this correlates to a spate of errors we saw both last night and the last time about replication failures:

[OSM Replication] 2025-12-04_05:29:55 - :police_car_light: production: osmdbt-get-log failed
[OSM Replication] 2025-12-04_05:30:08 - :police_car_light: production: ERROR: Failed to enable osmdbt replication. Check PostgreSQL configuration (wal_level=logical, max_replication_slots >= 1, user with REPLICATION attribute).
[OSM Replication] 2025-12-04_05:30:40 - :police_car_light: production: osmdbt-get-log failed
[OSM Replication] 2025-12-04_05:31:21 - :police_car_light: production: ERROR: Failed to enable osmdbt replication. Check PostgreSQL configuration (wal_level=logical, max_replication_slots >= 1, user with REPLICATION attribute).
[OSM Replication] 2025-12-04_05:32:54 - :police_car_light: production: ERROR: Failed to enable osmdbt replication. Check PostgreSQL configuration (wal_level=logical, max_replication_slots >= 1, user with REPLICATION attribute).
[OSM Replication] 2025-12-04_05:35:42 - :police_car_light: production: ERROR: Failed to enable osmdbt replication. Check PostgreSQL configuration (wal_level=logical, max_replication_slots >= 1, user with REPLICATION attribute).

9

Hey @Patrick_Brosi , Thanks for reporting this, it was an issue with my latest refactors, i just fixed the issue and the state files show the right links to the planet files

1 Like
10

Great, thank you very much!