Enhance _get_and_store_range Function to Include Trip Statistics and Last API Call Tracking

[TeachMeTW]

Overview

Enhances _get_and_store_range by adding total_trips, labeled_trips, and last_call fields to the user profile, enabling better insights into user activities and API usage.

Changes

1. Extended User Profile Updates:

   • total_trips: Total number of trips recorded for the user.
   • labeled_trips: Number of labeled/annotated trips.

2. Integrated Last API Call Tracking:

   • Accesses stats/server_api_time to retrieve timestamps of the last GET and PUT API calls.
   • Determines the latest call and stores it as last_call in the user's profile in UNIX format.

Testing

1. Picked a user, cleared pipeline, re-ran pipeline, verified user profile was updated.

Please review and provide feedback or approval.

[TeachMeTW]

Now I only have one concern: the current iteration does not use UUID in the api_call query, so I’m not too sure if the last call is accurate. If I do include the UUID, the last call is NULL for that UUID.

docs_cursor = edb.get_timeseries_db().find({
    "metadata.key": "stats/server_api_time",
    "user_id" : user_id
})

Thoughts?

[TeachMeTW]

@JGreenlee Comments addressed, I split the function into a different file as suggested -- though it may be overkill. Seeking thoughts

[JGreenlee]

Good refactor! I agree that the smaller functions are overkill.
Sometimes it's just a matter of finding the balance between tidiness and complexity.

[JGreenlee]

good

[TeachMeTW]

@JGreenlee Comments addressed

[TeachMeTW]

@JGreenlee Addressed and rebased