Skip to content

fix: use geodetic latitudes in haversine distance formula#14351

Open
VibhorGautam wants to merge 2 commits intoTheAlgorithms:masterfrom
VibhorGautam:fix/haversine-reduced-latitude
Open

fix: use geodetic latitudes in haversine distance formula#14351
VibhorGautam wants to merge 2 commits intoTheAlgorithms:masterfrom
VibhorGautam:fix/haversine-reduced-latitude

Conversation

@VibhorGautam
Copy link

@VibhorGautam VibhorGautam commented Mar 8, 2026

Describe your change:

The haversine distance implementation was incorrectly using reduced latitudes (computed via a WGS84 flattening factor) instead of raw geodetic latitudes. Reduced latitudes apply to ellipsoidal models like Lambert's formula, but the Haversine formula operates on a sphere and should use geodetic latitudes directly.

What was wrong:

flattening = (AXIS_A - AXIS_B) / AXIS_A
phi_1 = atan((1 - flattening) * tan(radians(lat1)))
phi_2 = atan((1 - flattening) * tan(radians(lat2)))

This computes reduced latitudes, which account for Earth's ellipsoidal shape. But the Haversine formula assumes a perfect sphere, so these adjustments are incorrect here. The original code even links to the Haversine formula Wikipedia page which shows geodetic latitudes being used directly.

What this PR fixes:

  • Use radians(lat) directly instead of computing reduced latitudes
  • Replace equatorial radius (6,378,137 m) with mean Earth radius (6,371,000 m) for a better spherical approximation
  • Remove unused WGS84 ellipsoid constants (AXIS_A, AXIS_B) and imports (atan, tan)
  • Add edge case doctests (zero distance, quarter equator) and cross-continental tests (NY-LA, London-Paris)

References:

Fixes #11308

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • An existing algorithm implementation is incorrect
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms include at least one URL that points to Wikipedia or another similar explanation.
  • If this pull request is for a pre-existing algorithm, I have linked to the issue.

The implementation was incorrectly using reduced latitudes (via a
flattening factor from WGS84 ellipsoid constants) instead of raw
geodetic latitudes. Reduced latitudes are appropriate for ellipsoidal
models like Lambert's formula, but the Haversine formula operates on
a sphere and should use geodetic latitudes directly.

Changes:
- Use radians(lat) directly instead of computing reduced latitudes
  with atan((1 - flattening) * tan(radians(lat)))
- Replace equatorial radius (6378137m) with mean Earth radius
  (6371000m) for better spherical approximation
- Remove unused WGS84 ellipsoid constants (AXIS_A, AXIS_B)
- Remove unused imports (atan, tan)
- Add edge case and cross-continental doctests

Fixes TheAlgorithms#11308
@algorithms-keeper algorithms-keeper bot added enhancement This PR modified some existing files awaiting reviews This PR is ready to be reviewed tests are failing Do not merge until tests pass labels Mar 8, 2026
…ngle

Lambert's ellipsoidal distance computes the central angle sigma by
dividing the haversine distance by a radius. Previously both functions
used the same equatorial radius (6378137m), so the values cancelled
out. After correcting haversine to use the mean Earth radius (6371000m),
Lambert's must divide by the same radius to recover the correct angle.

Also update the expected doctest values to match the corrected
haversine output.

Fixes TheAlgorithms#11308
@algorithms-keeper algorithms-keeper bot removed the tests are failing Do not merge until tests pass label Mar 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting reviews This PR is ready to be reviewed enhancement This PR modified some existing files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Haversine distance may be using the wrong phi1 and phi2 angle

2 participants