I have been adding natural features (mountains, water,…) to some areas I have been working. These can be significant historically and important locators on the map. In the last few days, I have been working the area around Cripple Creek, Colorado and imported a small set of NHDPlus data for Oil Creek and some tributaries. Of course it’s important to follow OSM/OHM guidelines (National Hydrography Dataset - OpenStreetMap Wiki) as much as possible and take care not to override preexisting data. I’ll welcome any comments on this as well as this example tagging scheme:
NHD:FCode = 46006
NHDPlusID = 21000900000933.0
intermittent = no
name = Oil Creek
source = NHDPlus_HR_import_HU4_20180813 (HR: high resolution, HU4 is the dataset, then the creation date)
waterway = stream
Not sure if FCode is really necessary as [intermittent] stream/river tags may make it redundant, but previous OSM imports included it and some other tags. Thought I would ask for feedback before going much further with this process. Thank you.
Dave - first, thanks for taking the lead on this - it’s fantastic!
My quick take is that this looks good overall.
My comments - all minor:
FCode - I wouldn’t bother importing this, unless you can imagine some way it will help future mappers. Essentially, anything that is duplicative of an OSM/OHM conversion tag can be omitted. The unique ID is what’s important. That said…
Maybe consider NHDPlus:id=* instead of NHDPlusID=*? That way, there’s a datasource prefix that could be used for other NHDPlus fields and all import-related metadata would share the same prefix? I’ve talked with some others about this and added a section in the source tag wiki page that describes this, but everything around this is new & open for discussion.
NHD & WGS84 considerations. When I compare the object you shared to Bing imagery, the data looks very accurate. But, you may want to be on guard for any issues with NDS and WGS84 outlined here.
Prior data. Please don’t hesitate to delete any natural features from osmosis_user_0 - that was the original import for OHM that is over 10 years old and contains data from even older sources.
Again, this level of detail is awesome and super interesting, given our disappearing waterways. Thank you!
This wiki page contains a comprehensive mapping from NHD fields to OSM tags, but I recall that there have been issues with that mapping in the past, and NHD itself has gone through several schema changes since the initial imports. I don’t know if the table has been updated in response. It might be worth checking in with OSM’s U.S. community to see if there are any other quality considerations that haven’t been documented there.
Looking it up (in the NHDPlus HR User Guide) the data for the contiguous US, Hawaii, PR, and VI is on NAD83(1986), Alaska is on NAD83(2011), and American Samoa, Guam, and the Northern Mariana Islands are on WGS84. It’s somewhat odd it’s not all on the same datum, but… US Government.
So, “theoretically”, transforming everything but Alaska to NAD83(2011), and then to WGS84, would show an improvement in alignment.
Actually looking at Changeset: 105676 | OpenHistoricalMap though, I don’t have the impression that it’s a major concern. The precision of the data seems low enough (varying from “right on” Bing to as much as 50-60 feet off) that an improvement in “accuracy” of generally less than 10 feet from the transform isn’t going to be noticeable… that’s in Colorado, however. In areas where the data is more precise (near coastlines, presumably) it might be worth revisiting.
The areas to be particularly concerned about would be coastal California and the Caribbean where (as stated by the NGS) “stations referenced to one plate were located on a different plate”. These areas specifically required modeling of tectonic motion and earthquakes in the NAD83(2011) adjustment, and data on the original NAD83 isn’t going to reflect that.
Good work, I did something similar for NE lower Peninsula of Michigan- in particular I emphasized waterbodies with a GNIS:feature_id, skipped over streams and focused on fixing up rivers
I changed the NHDPlusID key to NHDPlus:id on these streams. The locations seem to be at least as good as the OSM edits in this area; and they seem to match the default topo map in JOSM a little better than Bing imagery (I’m OK with either one). For this test, I just used the NAD83 to WGS 84 (16) USA, Colorado transformation in QGIS 3.34.2. I will certainly try this in a few other areas I’m interested in and may experiment some more with projections there.
Also, although some are tagged as intermittent=yes, they appear as solid lines on the time-slider map.
You have to be careful with NAD83 to WGS84 tranformations in QGIS (hence the long instructions on the wiki page). QGIS is quite happy to simply change the label on a layer while doing a “no operation” transformation, just increasing the “error bars” on the data without actually doing anything.
The real point of doing it the way described is actually to use the NADCON 5.0 grids to update the data to NAD83(2011) first… the newer realization is a far better approximation of a “mathematically perfect” NAD83 than the old one.
A position on NAD83(1986) is only “defined” in reference to the nearby published station positions, and there are known systemic biases the NADCON grids account for. The formal accuracy of the NAD83(2011) to WGS84 shift is much better, even though its technically the same math.
@Revent, I tried the series of transformations as specified on the Wiki page in this area. I detected no visible differences in the resulting geometry (excluding a few subsequent manual edits). Either way, the alignments look pretty good to me. There are differences with the imagery, but they mostly seem to be with local precision (frequent curves in stream channels missing from NHD, etc.). I tried the documented procedure on a small area in Massachusetts and I think it came out fine there as well, so I’ll probably continue with that; but, because of the thick ground cover in that area, it’s harder to detect differences with the imagery.
Fair enough. The scale of the difference it makes is very much dependent on what part of the country you are in, and how close you are to the edge of one of the “grids” in the original NAD83. The NGS gives heatmaps for the scale of the shifts in the NADCON 5.0 technical report, but unfortunately they are only for “each step” (USSD → NAD27 → NAD83 → HARN → FBN → NSRS2007 → 2011) and so don’t give the greatest indication of where it’s “needed” and where it’s “trivial”.
For example, they detected a net “residual plate rotation” between NSRS2007 and NAD83(2011), clockwise around a point west of central Mexico, but the effect of that is actually opposite in sign of the “grid” corrections for much of the country.
It’s my opinion that doing the grid shift is really only a big deal when it’s some dataset (like TIGER) where there’s not really any “visible” indication of the correct alignment. I only really popped in here because it was mentioned.
The one thing to be really careful about when finding “no change” though, is that QGIS will lie to you. It’s really important to make sure it’s actually using the NADCON grids… there are “equivalent” transformations that QGIS prefers to use that literally do nothing but change the label.
It looks like there is a very small difference in some locations where I had been sloppy with QGIS CRS conversions (I confess). The original is shown in red, and following the OSM Wiki documented step-by-step provides slightly different results from the proper transform (gray lines). In this area all differences are less than 10 feet, much smaller than differences between the NHD dataset and imagery or topographic maps. I don’t think it’s worth redoing any of these previous imports but will ensure the NADCON 5 grids are used going forward.
Jeff, I assume you’re referring to the difference between the imagery and the import data. A big part of the difference here may be because of meandering streams. In some places, it looks like the NHD import lines may be following old stream channels. In general the NHD data match the topographic maps better than the imagery, and that may be because of date differences. The publish date on the NHD data is 2018 but I suspect some of that might be much older–not sure about Bing or the topo maps in JOSM.
Thanks for the links, The historical areal photos might show changes in waterway geometry, but that’s one 174 gb zip file. That’s too much for my internet connection (or patience). Someday I may look for smaller sets of those.
If this import is “good enough” for OHM, I was thinking of moving on to missing links in important rivers (frustrating when trying to locate oneself in OHM) and leaving these small intermittent streams behind for now.
I suspect that the NHD (since you said it seems to be based on topographic maps) is generally following the center of the “channel” (i.e. the area between the defined stream banks) rather than the actual path of the low water flow. Since most satellite imagery is going to have been taken during periods of low flow, you’re going to see a lot of meandering that wouldn’t exist when the stream is full, and those gravel bars are going to move during every high water event.
There has to be some level at which tracking every meander that changes after every big storm becomes pointless. “Accuracy” (being in the right place) is IMO more important than “precision” (showing every meander) for streams, with “precision” being more relevant for actual bodies of water that have a more constant water level.
I’ll freely admit that the datum shift thing is a pet peeve of mine… I have been annoyed for years that the original TIGER import to OSM not only used mediocre data (blame the Census Bureau) but also didn’t transform the data to the then-current NSRS2007 (blame the NGS for making it obscure and telling everyone NAD83=WGS84 for decades after it wasn’t really true). This resulted in lots of “on the wrong side of the road” errors.
The followup to that is that its actual US Government policy that datasets be on NAD83(2011) but most agencies don’t seem to care. It’s really only a big deal when the data is either precise (from actual surveys) or not easily verifiable (boundaries). At the same time, it’s IMO something that people doing imports should be aware of, and the process of doing it correctly is fairly involved. Actually figuring out how to get QGIS to properly use the NADCON grids was a huge pain in the butt… fortunately, part of 5.0 was reworking the whole process so that future updates (while will happen in a year or so) will just be one more step.
