Trips represented by straight lines #1095
Comments
|
Here's what I see in the logs: Previous trip ended at 8:21 New trip started at 13:50, including a VISIT_ENDED Received some more (null) transitions and restarts Until we received a VISIT_STARTED and ended the trip We then started another trip almost immediately and saw the same pattern (14:11 to 14:40)Multiple (null) and restart transitions Until we got another VISIT started and end the trip We then get another geofence exit almost immediately and see the same pattern from 14:41 to 14:52and then a VISIT transition and multiple RESTARTS until we get a trip end Similar pattern from 14:57 to 15:03And one more from 15:22 to 15:57The next exit was a substantial time away so is not part of this fragmented trip. |
|
There are 5 clusters of points in the map, and 5 trip segments in the logs. So the first order of business is to look at why we are ending the trips so many times. @JGreenlee for visibility |
|
We are ending trips because of the VISIT transitions Looking at the logs around that We do check for invalid visit started transitions here: However, in this case, neither check was successful. Because of the restarts, there were multiple transitions. And the last two transitions were So we didn't even check for the deltaTs. And even if we had found the correct transitions and checked for the deltaTs, it would have been more than a few minutes. |
|
The visit API is a black box, so we are not able to look into it or understand why it is generating these spurious visit transitions. Since the transition here was several minutes after the trip start, the only possible check would be that the user was still moving - i.e. that we were receiving location updates that represented movement consistently. If so, we can ignore the VISIT_STARTED notifications. |
|
Looking at the location logs, it looks like this won't really work. The last log statement before the Verified using uploaded data We get data around 14:07 and then nothing else until 14:11 |
|
Let's see what is happening between 14:07 and 14:11 in the logs. We get the last location input that we see, and then we don't see any other logs until we initialize the plugins as part of restarting the app, just before we receive the VISIT transition. With these logs, and the iOS distance filter, it is not 100% clear whether:
Given that we see the But it would be good to get some indication from the user whether they actually did stop moving at that time. Because, we do try to detect when the app has been terminated, and I don't see that log either. |
|
Poking around a bit more at the termination, note (from https://developer.apple.com/documentation/corelocation/cllocationmanager/startupdatinglocation())
We do have the But while the background modes protect the app from being terminated because it went off the foreground, it clearly does not prevent it from being terminated due to resource constraints. Looking at documentation around termination: We now require a minimum of iOS 13+ so that is definitely an option. |
|
Checked the organizer - it does not have enough metrics. Which seems to argue against the reason being termination if this is consistent behavior for Nathan and Joe. Note also that, although we noticed that there was a restart around the time we had no locations, we also had multiple restarts that were not correlated with lack of locations. Focusing on the transitions: Checking the location logs around those, we see that there seem to be pauses for ~ 5 minutes, followed by reinitializing, and then seeing locations again |
|
Next steps:
|
|
One other observation from the logs before I forget. I noticed that there were several UI logs that were overlapping with the start of this trip e.g. but then The UI load seems to have started at around and if we look at the periods where the app appeared to restart, we were still loading data when the app appeared to be terminated. @JGreenlee I wonder if this is due to the fact that we load data on every resume now. If we are getting terminated due to resource constraints, the problem may be:
I would guess that this doesn't trigger for the test phone because it is not running anything else (so the resources are not constrained) and there aren't a lot of recent trips anyway since I don't carry the test phone everywhere. It seems like a general optimization would be to not load the diary if the app has been launched in the background since nobody is watching anyway. Need to see if the suspend/resume/launch notifications let us determine whether the launch was in the foreground or the background. |
|
I am surprised that the webview is even able to run while the app is not in the foreground. (I am guessing this iOS-specific because I don't think it would be possible on Android.)
We certainly do not want to be loading the diary when in the background – especially because when the app is resumed from the background, it will all be reloaded again anyway.
Yes it is possible via React Native's AppState API https://reactnative.dev/docs/0.71/appstate |
Yes, on iOS, the webview keeps running even when the app is in the background. For the trip-end notifications, this was actually helpful, here it is annoying 😄 |
|
wrt (1) from #1095 (comment) PSU also reported this trip (to a concert at around 5pm, returning at around 9:15pm). This was a freeway car trip, but still has clusters of location points. This seems to argue for the app termination as the cause of the issue. The relevant transitions, clustered by trip start and end are:Checking for app initializations and the resulting gaps... Interleaving them, we get
Two main takeaways from that deep dive:
|
|
Testing out the change, I see The full history (
|
|
Pushed it out to production after some delays. However, PSU reported that they were still seeing gaps in the reported locations. Checking the logs, we see that the app is no longer crashing, but there are arbitrary 5 minute gaps in the data collection. In some cases, the speed is low But in a lot of cases it is not. The app is clearly not restarting (no Let us see if we see similar behavior on the test phone. It would help a lot if we are able to reproduce; we may need to look at the phone logs instead of only the app logs. |
|
Alas, this is not reproducible on my test iphone. I plotted the trajectories from one of my trips, and we can even see the difference between two separate trips along the same major street. 
|
|
Given that the OS throttling the points that we get, the most likely culprit is a different version of the OS Checking the logs, it looks like my test phone is on
I think we have two potential next steps:
|
|
Just to make doubly sure, we do set everything that is needed for the location code. We do have And we listen for |
|
While waiting to figure out the versions of the test phones that we have, I pulled some data from a program that had predominantly iOS users. I pulled data for two days The first two had perfect trajectories[1] so I was feeling fairly optimistic, and then the third one had spotty trajectories!!! [1] I also noticed that even for the good trajectories, the points were clustered together more closely off the freeway than on (aka slower speeds = closer points). Since iOS has a distance filter, this should not be the case, but I guess the phone was going fast enough that it overwhelmed the distance filter. |
|
The three trajectories, plotted with
|
|
I thought this was an almost exclusively iOS dataset, but apparently not (at least for the dates that we were testing). Let me clean up the analysis and work with a larger range of dates so that we can get a better sense of the iOS data |
|
Cleaning up the analysis and working with the full range of dates, we get a lot more iOS users It seems pretty clear that the vast majority of users have good trajectories but there are a couple (at least two) that do significantly worse. I don't see any correlation with OS version at least at this point. Some of this may be due to the geofence exit delay, so let's retry after dropping the first point of every trip. Note also that
|
|
It took me essentially all of yesterday to strip out the first point because I (potentially foolishly) decided to not rely on the existing segmentation output and go directly to the transitions. But I finally have it working, and the results are fairly similar. There is one phone who seems to have non-fine-grained trips, but the majority of other phones are doing fairly well. There is more variability in the time between points, as might be expected from an OS with a distance filter - the time delta will depend on the speed.
Digging a little deeper into that phone... |
|
Digging into phone
Description of the median inter-point distance across trips Again, it seems like there are some trips where the median distance between points is close to what we expect, but some where it is much larger. Let's plot a few and see if any of them show the distinctive stutter of points that PSU is seeing. And then I will have to move on to other work. |
|
Plotting some of the trajectories as basic lines, we see that some of them do have gaps but I don't see the stutter (lots of points and then a gap) that PSU is seeing. The 90th percentile median one in particular is clearly with evenly spaced points. I do think that the points are coming in at more than 1 meter, but they seem to be fairly consistent at that distance. Trajectories of trips whose medians are at various quantiles for the overall trip medians (all distances in meters)
|
|
For the record, here's are the same metrics for the PSU phone
To summarize, working from a larger dataset of iPhones, we see that:
We can see this in the time boxplots - the median on the PSU phone is low, pulled down by the "bunches of points", while the median on the sample phone is high, representing a constant stream of widely spaced points.
|
|
Summary of the problem:
Given this, I suggest the following next steps:
|
For a moment we were considering sending my iPhoneSE (2nd gen) so I moved to a new opcode to verify tracking, thinking the fact I hadn't seen trips in a while was due to the pipeline being stuck, but when I got a notification yesterday that "trip tracking may not work".
I'll try uninstalling and reinstalling the app to see if that gets tracking working - it's worth noting that I tracked a hike with Strava this weekend and had no issues, though I have no idea how OpenPATH tracking compares. |
For the first time in a while I have a trip record, but it is only the last 1/2 mile of my 3 mile commute home. Happy to provide logs/dump/screenshots on internal repo. |
Report from PSU, who has been testing data collection
They took this trip (from roughly 205pm to 330pm)
and it was recorded as
Public issue created with permission (email dated Oct 24, with title "OpenPath questions about gaps in trips")
The text was updated successfully, but these errors were encountered: