Importing the Newberry Library's Atlas of Historic County Boundaries

I’ve finished patching up the lakeward boundaries in all the Great Lakes states. However, I mostly left Michigan’s counties alone. I was able to track down the historical boundary claims of Menominee County, Wisconsin, in Green Bay, but otherwise I went with whatever @leonne and @matteditmsts mapped over water, corroborating it with the Census Bureau’s Boundary and Annexation Survey.

Speaking of which, the Boundary and Annexation Survey publishes a file called “Geographic Boundary Change Notes” listing changes to any kind of boundary in the country through January 1, 2013. There were 720 changes between 2001 and 2013 categorized as “County Boundary Change” or “County Boundary Correction”.

The BAS also publishes a listing of “substantial” county boundary changes that continues to be updated. Since 2001, the significant changes include the addition of four county-equivalents, the elimination of two others to become non–county equivalents, two renames (done), one annexation of non-county area, and two land swaps. Additionally, the Census Bureau has replaced Connecticut’s counties with council of government planning regions.

We can cross-reference this information with the Census Bureau’s cartographic boundary files from 2000 through 2018 and 2019 to present. In some cases, the changes hew to more local boundaries such as municipalities and CCDs that we can map based on a single snapshot in time. For example, some of the changes are the result of a city becoming an independent city or ceasing to be one, so no geometries actually change. For manual mapping, TIGERweb publishes WMSes that are suitable for use as background layers.

I’ve taken care of all the substantial county-equivalent boundary changes from 2001 to present. I tried my best to reconcile the imported Newberry boundaries with USGS topographic maps (either the built-in USGS topo layer or USGS topoView), TIGER boundaries, and in some cases current aerial imagery where property lines could be inferred from it. Any new boundaries extend into surrounding waters, but I didn’t go back in time and adjust previous stages of a county equivalent to also extend into the water.

Along the way, I learned a new trick for copying boundary relations in iD.

I need help with these changes. There are only 195 changes classified as real-world changes, a much more manageable starting point for us. Some states have many more changes than others; it ranges from one in Ohio (which turned out to be a correction rather than a real-world change) to a whopping 43 in Virginia (mostly changes to independent city boundaries).

Pick a state you’re interested in, filter the CSV to just that state (Type of Change: County Boundary Change, State: the FIPS state code), and compare the imported boundary to USGS and TIGER. You’ll tend to see lots of small discrepancies because of the coordinate transformation issues discussed earlier and general crudeness, but also because TIGER boundaries are often snapped to the nearest road or waterway centerline for statistical convenience. Thus, the USGS layer can serve as a baseline to distinguish these discrepancies from actual real-world changes.

If there has been a real-world change, remember to copy the boundary relation, add it to the chronology relation, and add an end_date to the existing boundary relation. Don’t simply modify the existing boundary relation’s geometry; that would cause an anachronism when viewing the map before the change took place.

This to-do list tracks lots of other cleanup tasks to do, big and small, that you can do in either iD or JOSM with enough patience. Please holler if you’re interested in helping out but need help getting started.

FYI, I have Snyder converted into GIS/database form already, in case you want it.

1 Like

Hello,

Sorry I missed all the activity on this, but congratulations on getting this data imported!

Can I change the links pointing to my SQL script and the outputs in the wiki to the following, respectively? I believe you used v1.1.0 (September 2023) for the upload based on the EDGE_ID in your sample having the FIPS:

Have the county chronologies been created? I wanted to add some boundary changes for Philadelphia that are missing from the Newberry dataset, but was debating if it would be better to hold off until this step was done first.

Separately, I would mention that the modern Geographic Boundary Change Notes should be taken with very large grains of salt. In Pennsylvania, the only way to change a county boundary (and not just settle a dispute) from the mid-1970s through 2022 was to circulate petitions to get signatures of 5% of voters from the last Governor’s race, hold a referendum in all counties affected, and get a majority of the vote in each county affected. Some of the county boundary changes listed were settled disputes that didn’t meaningfully move the line (Allegheny-Westmoreland, Carbon-Monroe), but I suspect the others were just GIS adjustments, as I haven’t seen any evidence for them in DCED or State Archives Land Office Map files. Should we be keeping track of which notes are resulting in OHM changes and which ones should be skipped?

1 Like

Sure, feel free to edit the wiki with that clarification.

Yes, as far as I can tell, all the county chronologies have been created. I’ve created a few myself, such as Relation: ‪Broomfield‬ (‪2798510‬) | OpenHistoricalMap. You can certainly create any that are missing. Once you do, please add an OpenHistoricalMap relation ID (P8424) statement to the corresponding Wikidata item for discoverability.

I’m also reaching for the salt. The change notes ostensibly distinguish between actual real-world changes (County Boundary Change) and mundane GIS corrections (County Boundary Correction), but at a glance, some have definitely been miscategorized. Still, the it’s still valuable as an index of leads to follow up on, since the Newberry dataset frequently used the change notes as a source. Also, I have a strong suspicion that the geometries were based in part on an earlier vintage of TIGER boundaries. The one County Boundary Change in Ohio smoothed out a jog in an imported boundary that was always defined to be straight in the statutes.

As you encounter false positives, you could leave a note on the wiki page or tag the relation in OHM with a note=*, or we could copy the CSV into Google Sheets and track the status there.

Hi @markconnellypro :wave:t2: - the products of your work are being put to great use! And, yes, please update the wiki for the project as appropriate whenever you like! Thanks for the changes you’ve already made.

Minor update: after thinking I had lost the lookups for the state boundaries, I have found them & have begun moving the state boundary relations to their component county segments instead of the older, longer segments. Whew.

I’m doing this semi-manually, to make sure I’m not missing any adjustments that have been made to the original long state boundary segments.

In cases where users have made changes, my general approach is to assume that the changes are correct, as there are plenty of small-scale errors in the source data. In some cases, however, there is no source cited for the changes. In these cases, I am reaching out to the authors to double-check why the changes were made.

I’m also creating state-level chronologies as I go.

1 Like

Just an update to let everyone know this work is continuing and I hope to have all of the state borders aligned with the county boundary segments in the next week or two. The discussion in the territorial waters post will inform how the coastal states are handled, so that’s a minor holdup. After that, there are a variety of cleanup tasks to complete, but we’ll keep hammering away at them until they’re complete.

Another reason not to take the Newberry dataset as being 100% definitive… or at least, another reason to question it. : )

https://www.in.gov/history/state-historical-markers/find-a-marker/indiana-territory-boundary-line/

image

The northern border of the Indiana Territory does not quite match the northern border depicted in the Indiana Territory (1809-16) relation:

Either the plaque’s wrong or we’re wrong…

Indiana Territory used to cover all of Illinois and Wisconsin as well. When Illinois Territory was split off in 1809, it left this gore in Indiana rather than transferring it to Illinois or Michigan territory:

…all that part of the Indiana territory which lies west of the Wabash river, and a direct line drawn from the said Wabash river and Post Vincennes, due north to the territorial line between the United States and Canada, shall, for the purpose of temporary government, constitute a separate territory, and be called Illinois.

The dashed line on the plaque represents a survey line, not the precise northern territorial boundary. Maps traditionally extrapolated survey lines because various boundaries would depend on the hypothetical intersection of two survey lines. The historical society probably considered the gore to be irrelevant to a plaque all the way in La Porte.

(I’m really appreciating all the source:citation tags you left in from the Newberry dataset. It’s a boon for further research.)

1 Like

So, you’re saying I forgot the third case… both the plaque & the map are wrong! :disguised_face:

Ok… very exciting news (for me, anyway!). With a few exceptions (see below), I believe I have replaced all of the old Newberry state boundaries (long segments) with their generally overlapping county boundary segments. This will enable mappers to update / correct the old county boundaries and have it automatically update old state boundaries, as well.

Exceptions to this claim:

  • New York and New Jersey historic relations appear to be under the good care of user:BMACS001. @markconnellypro - is this you by any chance? I think I noticed some references to the Local Geohistory Project in some of the BMACS001 changesets, but I cannot seem to find them now. If so, could I interest you in replacing the long state boundaries with their county segments? :slight_smile:
  • Maine has never really been part of the Newberry state imports for a variety of reasons, but its inland boundary long segments have been replaced with shorter county segments where possible.

Separately, I’ve experimented a little with maritime boundaries as part of the state boundaries for Maine, using a handy NOAA 3nm line for Maine from the Maine Geolibrary. This reduced the number of members in the Maine relations from over 4,000 (lots of islands!) to 44.

1 Like

@BMACS001 is our resident New Jersey boundary conoisseur. The parallel maritime boundary discussion has some background context.

1 Like

And… a peril of using JOSM and only downloading what you think you need is that sometimes you miss geometry that’s already there. In this case, the Eastern Time Zone boundary already included 3nm line ways, which I’ve substituted for the previous / duplicative Maine Geolibrary 3NM lines in the Maine relations.

By the way, there’s a lot of nuance and history in the U.S. maritime boundaries:


As the diagram shows, there’s a third dimension to the data, and a fourth.

1 Like

In JOSM, remember to resolve any “Incomplete relation modified” warnings before uploading; otherwise, you’re likely to break any relation you didn’t think to download ahead of time. You can also avoid some of these warnings proactively by zooming into the node where you’re about to split a way and going to File ‣ Download Current View before splitting. (This kind of breakage is less likely to happen in iD, which downloads the affected relations automatically.)

Consulting individual states’ open data portals seems difficult, given that some states don’t place their work in the public domain. Have you considered downloading the data directly from BOEM/NOAA’s MarineCadastre.gov, specifically these FeatureServers? There are some historical layers in there too.

1 Like

Mea culpa! I made a mistaken assumption that JOSM would intelligently deal with way splitting in relations (i.e., keep both parts). Hopefully, I didn’t delete any ways in the relations. I was trying to be diligent about downloading parent ways / relations before deleting anything.

Out of curiosity - how did you know these were broken? I’m glad you did know, but is there an alert or alarm of some sort?

I looked at MarineCadastre.gov and others, but found the NOAA data on the state portal. I hadn’t seen that feature server, but will go there next.

I happened to notice that the boundaries looked all wonky in Overpass Ultra. The closest thing to a monitoring system I know of is that this QLever query returns all the broken boundary relations (and can easily return broken multipolygons as well). But QLever is currently a few months out of date.

1 Like

QLever has been updated, and it looks like we might have a path to getting daily updates on there, which will make this a useful tool in the toolchest. That way we won’t have to fork and maintain a more specialized OSM QA tool like KeepRight or Osmose for this kind of error at least.

Edit: QLever now updates daily.

Ok… recognizing that improving quality of the historical county boundary relations of the United States will be an ongoing project, and… recognizing there are likely errors here and there in this effort that will need to get cleaned up after the fact, and… recognizing that nothing on this project is ever really “done”… I’m going to take the bold step of saying…

the import of the Newberry Library’s Atlas of Historic County Boundaries is done. : )

Time to put a bow on it :gift:.

Newberry is done!
Long live Newberry!

Wait… what?!?

Well, like @Minh_Nguyen pointed out above, everything still on our to-do list, still needs to get done to make sure we have the most up to date and robust dataset of historical county boundaries on the web. It’s really just the first phase that’s “done.”

Long live Newberry!

1 Like