GPS data comparison after smoothing

Question

I'm trying to compare multiple algorithms that are used to smooth GPS data. I'm wondering what should be the standard way to compare the results to see which one provides better smoothing.

I was thinking on a machine learning approach. To crate a car model based on a classifier and check on which tracks provides better behaviour.

For the guys who have more experience on this stuff, is this a good approach? Are there other ways to do this?

Answer 1

Machine Learning is not an well suited approach for that task, you would have to define what is good smoothing...

Principially your task cannot be solved by an algorithm that gives an general answer because every smoothing destroy the original data by some amount and adds invented positions, and different systems/humans that use the smoothed data react differently on that changed data.

The question is: What do you want to achieve with smoothing? Why do you need smoothing? (have you forgotten to implement or enable a stand still filter that eliminates movement while the vehicle is standing still, which in GPS introduces jumping location during stand still?)

The GPS chip has already built in a (best possible?) real time smoothing using a Kalman filter, having on the one side more information than a post processed smotthing algo, on the other side it has less. So next you have to ask yourself: do you compare post processing smoothing algos or real time algos? (probably post processing) Comparing a real time smoothing algorithm with a post process smoothing algorithm is not fair.

Again: What do you expect from smoothed data: That they look somewhat fine, but unrealistic like photoshopped models for tv-advertisments?

What is good smoothing? near to real vehicle postion which nobody ever knows, or a curve whith low acceleration?

I would prefer an smoothing algorithm that produces the curve most near to the real (usually unknown) vehicle trajectory.

Or you might just think it should somehow look beautifull: In that case overlay the curves with different colors, display it on a satelitte image map, and let a team of humans (experts at least owning and driving an own car) decide what looks good and realistic. We humans have the best multi purpose pattern matching algorithm built in.

Again why smooth?: for display in a map to please humans that look at that map? or to use the smoothed tracks to feed other algorithms that have problems with the original data?
To please humans I have given an answer above.
To please other algorithms:
What they need? nearer positions? or better course value / direction between points. What attributes do you want to smooth: only the latitude, longitude coordinates, or also the speed value, and course value?

I have much professional experience with GPS tracks, and recommend, to just remove every location under 7km/h and keep the rest as it is. In most cases there is no need for further smoothing.

Otherwise it gets expensive:
A possible solution:

1. You arrange a 2000€ Reference GPS receiver delivered with a magnetic vehicle roof antenna (E.g Company hemisphere 2000 GPS receiver) and use that as reference
2. You use a comnsumer GPS usually used for your task (smartphone, etc.)

Both mounted inside the car: drive some test tracks, in good conditions (highways) but more tracks at very bad: strong curves combined with big houses left and right. And through tunnel, a struight and a curved one, if you have one.

3. apply the smoothing algoritms to the consumer GPS tracks
4. compare the smoothed to the reference track, by matching two positions and finally calulate the (RMSE Root mean squared error)

Difficulties matching two positions: Hopefully the time can be exactly matched which is usually not the case (0,5s offset possible). Think what do you do when having an GPS outage.

Consider first to display a raw track and identify what kind of unsmoothed data is not suitable/ nice looking. (Probably later posting the pics here)

Answer 2

Generally, there is no universally valid way for comparing two datasets, since it completely depends on the applied/required quality criterion.

For your appoach

I was thinking on a machine learning approach. To crate a car model based on a classifier and check on which tracks provides better behaviour.

this means that you will need to define your term "better behavior" mathematically.

One possible quality criterion for your application is as follows (it consists of two parts that express opposing quality aspects):

First part (deviation from raw data): Compute the RMSE (root mean squared error) between the smoothed data and the raw data. This gives you a measure for the deviation of your smoothed track from the given raw coordinates. This means, that the error (RMSE) increases, if you are smoothing more. And it decreases if you are smoothing less.

Second part (track smoothness): Compute the mean absolute lateral acceleration that the car will experience along the track (second deviation). This will decrease if you are smoothing more, and it will increase if you are smoothing less. I.e., it behaves in contrary to the RMSE.

Result evaluation:

(1) Find a sequence of your data where you know that the underlying GPS track is a straight line or where the tracked object is not moving. Note, that for those tracks, the (lateral) acceleration is zero by definition(!). For these, compute RMSE and mean absolute lateral acceleration. The RMSE of appoaches that have (almost) zero acceleration results from measurement inaccuracies!

(2) Plot the results in a coordinate system with the RMSE on the x axis and the mean acceleration on the y axis.

(3) Pick all approaches that have an RMSE similar to what you found in step (1).

(4) From those approaches, pick the one(s) with the smallest acceleration. Those give you the smoothest track with an error explained through measurement inaccuracies!

(5) You're done :)

Answer 3

Take a look at this paper that discusses comparing machine learning algorithms:

"Choosing between two learning algorithms based on calibrated tests" available at: http://www.cs.waikato.ac.nz/ml/publications/2003/bouckaert-calibrated-tests.pdf

Also check out this paper:

"Bayesian Comparison of Machine Learning Algorithms on Single and Multiple Datasets" available at: http://www.jmlr.org/proceedings/papers/v22/lacoste12/lacoste12.pdf

Note: It is noted from the question that you are looking into the best way to compare the results for machine learning algorithms and are not looking for additional machine learning algorithms that may implement this feature.

Answer 4

what about using the good old Kalman Filter!

Answer 5

i think you easily match the coordinates after the address conversion. because address have street,area and city. so you can easily match the different radius.

let try this link

Answer 6

I have no experience on this topic but I have few things in mind that may help you.

You know it is a car. You know that the data is generated from a car so you can define a set of properties of a car. For example if a car is moving with speed above 50km than the angle of the corner should be at least 110 degrees. I am absolutely guessing with the values but if you do a little research i am sure you will be able to define such properties. Next thing you can do is to test how each approximation fits the car properties and choose the best one.

Raw data. I assume you are testing all methods on a part of given road. You can generate a "raw gps track" - a track that best fits the movement of a car. Google maps may help you to generate such track os some gps devise with higher accuracy. Than you measure the distance between each approximation and your generated track - the one with the min distance wins.