Skip to content

Improve error rate for observation events #112

@rhannequin

Description

@rhannequin

Introduction

I tried something to figure out if the library was buggy, so I tried to compute the Moon's observation events everyday for all positive coordinates on Earth (with precision of 1°). Positive coordinates mean I'm restricted to only a quarter of all the possible coordinates, but I assume this should still be representative and it reduces the execution time by four.

Data producer

tries = 0
interpolation_errors = []
math_errors = []
argument_errors = []
estimated_tries = 365 * 90 * 180
(0..364).to_a.each do |i|
  (0..89).to_a.each do |latitude|
    (0..179).to_a.each do |longitude|
      begin
        tries += 1
        pp "Try #{tries}/#{estimated_tries}"
        date = Date.new(2024, 1, 1).next_day(i)
        observer = Astronoby::Observer.new(
          latitude: Astronoby::Angle.from_degrees(latitude),
          longitude: Astronoby::Angle.from_degrees(longitude)
        )
        moon = Astronoby::Moon.new(time: date)
        observation_events = moon.observation_events(observer: observer)
      rescue Astronoby::IncompatibleArgumentsError
      interpolation_errors << [i, latitude, longitude]
      rescue Math::DomainError
        math_errors << [i, latitude, longitude]
      rescue ArgumentError
        argument_errors << [i, latitude, longitude]
      end
    end
  end
end

Data

argument_errors.csv
interpolation_errors.csv
math_errors.csv

Results

  • 5,913,000 instances tested
  • 320,153 errors occurred (5.41%)
    • Astronoby::IncompatibleArgumentsError (265,808 times, ~83%)
    • Math::DomainError (54,326 times, ~16%)
    • ArgumentError (19 times, ~1%)

Side note: I made a first attempt with all possible coordinates on Earth, close to 24m instances, but it took forever and I made a mistake which made me lose all the data. I just remember that the error rate was between 3% and 4%, with interpolation errors being something like 65% of them.

Errors

Astronoby::IncompatibleArgumentsError

Astronoby::IncompatibleArgumentsError is the error that happened the most, so I analyzed it first.

I checked the coordinates that were involved with this error the most frequently and sorted by the number of occurrences. Turns out this error happens more frequently with extreme longitudes. Over the 500 coordinates that were implicated the most frequently in interpolation issues, 481 had longitudes between 170° and 179°. This needs to be investigated.
My first guess is that there is a problem with the interpolation logic when values are close to the range limit. This doesn't seem to affect the latitude, which is a range between -90° and 90°.

Math::DomainError

For Math::DomainError, it is not very clean but it looks like the latitude has an impact for some reason. Over the 500 coordinates that were implicated the most frequently in Math errors, all had latitudes between 62° and 87°. But for the moment, this doesn't ring any bell.

ArgumentError

There was nothing to extract from the 19 argument errors. I guess the goal here would be to just reproduce them one by one and try to debug the problem individually.

Conclusion

A 5% error rate is not acceptable, especially when my intuition says this is just an interpolation range bug. My goal is to drop this number to less than 1% error rate. This can be achieve if most or all of the interpolation errors are fixed.

Technically, most of Math::DomainError errors should be fixable as a well as they are the result of incoherent number used for methods like Math.acos.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions