OpenStreetMap Integration Guide

OpenStreetMap (OSM) provides community-maintained bridge height data that can supplement official government sources. This guide covers validation, cross-referencing, and importing OSM data.

Overview

What OSM Offers:

Community-maintained maxheight tags on bridges
617+ bridges with height data in Melbourne alone
10,000+ potential bridges nationwide
Regular updates from local contributors
Covers local roads not in state datasets

Limitations:

Community-maintained (varying accuracy)
May be outdated (no inspection dates)
Should be flagged as lower confidence than official sources
API rate limits for bulk queries

Use Cases

1. Fill Data Gaps

VIC's 45 missing rail overpasses
Local roads not covered by state datasets
Supplementary coverage in rural areas

2. Data Validation

Cross-reference official government data
Flag discrepancies for investigation
Verify suspicious measurements

3. Quality Assurance

Identify outdated official data
Detect measurement errors
Community verification

Implementation Phases

Phase 1: Manual Lookup Helper ✅ Complete

For targeted data collection of specific structures (e.g., VIC's 45 rail overpasses).

Script: scripts/phase1-osm-manual-lookup.ts

What It Does:

Fetches specific bridge records from database
Generates OSM search URLs for each structure
Creates tracking files for manual lookup
Auto-generates import script template

Generates:

data/vic-rail-overpasses-osm-lookup.csv - Excel-ready tracking file
data/vic-rail-overpasses-checklist.md - Progress tracker with OSM URLs
scripts/phase1-osm-import-from-csv.ts - Auto-generated import script

Usage:

# Generate lookup helper files
npm run osm:phase1-generate

Manual Process:

Open the generated checklist markdown file
Click OSM URL for each structure
Search for the bridge on OpenStreetMap
Record maxheight tag if found
Update CSV file with findings
Run generated import script

Example CSV Entry:

structure_id,osm_id,maxheight,verified
SN7095,,,"❌ Not Started"

After Manual Lookup:

structure_id,osm_id,maxheight,verified
SN7095,way/123456789,3.8,"✅ Verified"

Phase 2: Automated Import ✅ Created (⚠️ Rate Limited)

For automated bulk import from geographic regions.

Script: scripts/phase2-osm-tourist-areas.ts

What It Does:

Queries OSM Overpass API for bridges in defined areas
Parses maxheight tags (handles multiple formats)
Deduplicates against existing database records
Imports new bridges with proper attribution

Target Areas:

Great Ocean Road, VIC
Blue Mountains, NSW
Gold Coast, QLD
NSW South Coast
Margaret River, WA
Hobart, TAS
Adelaide Hills, SA
Grampians, VIC
Phillip Island, VIC
Ningaloo Coast, WA

Test Results:

Found 22 bridges in dry-run mode
- Blue Mountains: 12 bridges
- Gold Coast: 7 bridges
- NSW South Coast: 3 bridges

Usage:

# Dry-run (no database changes)
npm run osm:phase2-dry-run

# Live import
npm run osm:phase2-import

# With increased delay (if rate limited)
npm run osm:phase2-import --delay=10000

Rate Limiting Issue:

OSM Overpass API has strict rate limits
Querying large areas triggers HTTP 429 errors
Solution: Use longer delays between requests (10-30 seconds)

Example Query:

const query = `
  [out:json][timeout:60];
  (
    way["bridge"]["maxheight"](${minLat},${minLon},${maxLat},${maxLon});
    node["bridge"]["maxheight"](${minLat},${minLon},${maxLat},${maxLon});
  );
  out center;
`;

OSM Data Quality

Height Tag Formats

OSM uses various formats for maxheight:

Metric: "3.8" → 3.8m
Imperial: "12'6\"" → 3.81m
Mixed: "4.5m" → 4.5m
Feet only: "12'" → 3.66m

Parsing Logic:

function parseOSMHeight(maxheight: string): number | null {
  // Metric (most common)
  if (maxheight.match(/^\d+(\.\d+)?$/)) {
    return parseFloat(maxheight);
  }

  // Metric with unit
  if (maxheight.endsWith('m')) {
    return parseFloat(maxheight.replace('m', ''));
  }

  // Feet and inches: 12'6"
  const feetInches = maxheight.match(/^(\d+)'(\d+)"$/);
  if (feetInches) {
    const feet = parseInt(feetInches[1]);
    const inches = parseInt(feetInches[2]);
    return (feet * 0.3048) + (inches * 0.0254);
  }

  return null;
}

Data Confidence Levels

When importing OSM data, assign confidence levels:

Source	Confidence	Use Case
Official Government	High	Primary data, preferred for state highways
OSM (Verified)	Medium	Supplementary, good for local roads
OSM (Unverified)	Low	Flag for manual verification

Database Schema:

ALTER TABLE road_restrictions
  ADD COLUMN osm_id BIGINT,
  ADD COLUMN data_confidence TEXT DEFAULT 'high';

Confidence Assignment:

const confidence =
  data_source === 'openstreetmap' ? 'medium' :
  data_source === 'official_government' ? 'high' :
  'low';

Discrepancy Handling

When OSM and official data conflict:

1. Minor Differences (`<0.2m`)

Action: Accept both, flag for monitoring
Reason: Measurement precision, rounding differences
Example: Official: 3.8m, OSM: 3.7m → No action needed

2. Moderate Differences (0.2m - 0.5m)

Action: Prefer official source, log discrepancy
Reason: May indicate measurement method differences
Example: Official: 3.8m, OSM: 3.3m → Use 3.8m, investigate

3. Major Differences (`>0.5m`)

Action: Flag for manual verification
Reason: Likely indicates error or structural change
Example: Official: 3.8m, OSM: 2.8m → Requires investigation

4. OSM Only (No Official Data)

Action: Import with data_confidence: 'medium'
Reason: Better than no data
Example: Local road bridge → Use OSM data

Best Practices

1. Prioritize Official Sources

Always prefer government datasets for state highways
Use OSM to fill gaps in official data
Mark OSM-sourced records clearly

2. Attribution

Include OSM attribution in data_source field
Store OSM ID for traceability
Link back to OSM for updates

Example Record:

{
  data_source: 'openstreetmap',
  osm_id: 123456789,
  data_confidence: 'medium',
  source_url: 'https://www.openstreetmap.org/way/123456789',
  max_height_meters: 3.8,
  verified_date: '2025-10-15'
}

3. Regular Updates

Schedule monthly OSM sync for active areas
Update stale data (>6 months old)
Monitor OSM changesets for bridge edits

4. Community Contribution

Report errors back to OSM
Contribute verified measurements
Improve data quality for everyone

API Reference

Overpass API Endpoint

https://overpass-api.de/api/interpreter

Query Examples

Find all bridges with heights in bounding box:

const query = `
  [out:json][timeout:60];
  (
    way["bridge"]["maxheight"](${minLat},${minLon},${maxLat},${maxLon});
    node["bridge"]["maxheight"](${minLat},${minLon},${maxLat},${maxLon});
  );
  out center;
`;

Find specific bridge by ID:

const query = `
  [out:json];
  way(${osmId});
  out center;
`;

Rate Limiting

Overpass API Limits:

Requests: ~2-3 per second recommended
Timeout: 60-180 seconds per query
Large queries: May trigger HTTP 429

Implementation:

// Wait between requests
await new Promise(resolve => setTimeout(resolve, 2000));

// Increase timeout for large areas
const query = `[out:json][timeout:180];...`;

// Handle rate limit errors
if (response.status === 429) {
  console.log('Rate limited, waiting 60 seconds...');
  await new Promise(resolve => setTimeout(resolve, 60000));
}

Scripts Reference

Phase 1: Manual Lookup

# Generate OSM lookup helper files
npm run osm:phase1-generate

# Import from manually-filled CSV
npm run osm:phase1-import

Phase 2: Automated Import

# Test mode (dry-run)
npm run osm:phase2-dry-run

# Live import with default delay (2s)
npm run osm:phase2-import

# Live import with custom delay (10s)
npm run osm:phase2-import --delay=10000

Validation

# Cross-reference with OSM
npm run db:validate-osm

Future Enhancements

1. Automated Sync

Weekly/monthly scheduled OSM imports
Update changed records automatically
Flag deleted OSM features

2. Conflict Resolution

Automated discrepancy detection
User-reported corrections
Voting system for disputed measurements

3. Mobile App Integration

Display OSM attribution
Allow users to verify/update OSM data
Contribute measurements back to OSM

4. Data Quality Dashboard

Show coverage by source
Track verification status
Monitor update frequency

Overview - Complete implementation status
Data Quality Improvements - OSM validation methodology
VIC Rail Overpasses - Manual lookup use case

Overview​

Use Cases​

1. Fill Data Gaps​

2. Data Validation​

3. Quality Assurance​

Implementation Phases​

Phase 1: Manual Lookup Helper ✅ Complete​

Phase 2: Automated Import ✅ Created (⚠️ Rate Limited)​

OSM Data Quality​

Height Tag Formats​

Data Confidence Levels​

Discrepancy Handling​

1. Minor Differences (<0.2m)​

2. Moderate Differences (0.2m - 0.5m)​

3. Major Differences (>0.5m)​

4. OSM Only (No Official Data)​

Best Practices​

1. Prioritize Official Sources​

2. Attribution​

3. Regular Updates​

4. Community Contribution​

API Reference​

Overpass API Endpoint​

Query Examples​

Rate Limiting​

Scripts Reference​

Phase 1: Manual Lookup​

Phase 2: Automated Import​

Validation​

Future Enhancements​

1. Automated Sync​

2. Conflict Resolution​

3. Mobile App Integration​

4. Data Quality Dashboard​

Related Documents​