fix(lib-classifier): record classification times between the first and last annotation #6449

eatyourgreens · 2024-11-07T09:27:14Z

Add tests for classification started_at and finished_at timestamps, to catch Classification started_at is off by hours (or days) for continuous sessions in the same tab. #6447.
fix the failing test for classification start times by recording started_at when a volunteer first interacts with a task eg. making a choice selection for a survey, choosing an answer to a question etc. If they go straight to Done, without updating an annotation, started_at will fall back to the old value.

Please request review from @zooniverse/frontend team or an individual member of that team.

Package

lib-classifier

Linked Issue and/or Talk Post

fixes MST error in the Classification model tests #6448.
adds tests that reproduce Classification started_at is off by hours (or days) for continuous sessions in the same tab. #6447.
fixes Classification started_at is off by hours (or days) for continuous sessions in the same tab. #6447.
Incorrect timing on the stats page #6441 has a chart from my personal stats page, where 10 minutes worth of classification effort was recorded as 5 hours, because of this bug.
Talk comment from Peter Mason about unreliable classification durations in project data that he works with: https://www.zooniverse.org/talk/17/3490049?comment=5741903&page=1

How to Review

classification.metadata.started_at should now record the time that you started a classification, proxied as the time that you first annotate a subject. You should be able to test by waiting between downloading a subject and starting a classification.

It should fix cases, mentioned by Peter Mason on Talk, where project teams are finding classifications that seem to have taken hours or days, but really only took minutes.

I have seen differences in the start time (started classifying by the volunteer) and finished times (completed classification received by zooniverse) of many hours, even days.

The new tests for classification start time and duration should fail when run with the old code for classification.metadata.started_at.
https://github.com/zooniverse/front-end-monorepo/actions/runs/11720164440/job/32644906423#step:9:5027

Logs from a failing test for classification start time, where the expected time is one hour different from the recorded time.

I've got a staging project with a variety of workflows and tasks, including drawing tasks, which now record started_at when you first draw on the subject:
https://localhost:8080/?project=eatyourgreens/-project-testing-ground&workflow=3370

Checklist

PR Creator - Please cater the checklist to fit the review needed for your code changes.
PR Reviewer - Use the checklist during your review. Each point should be checkmarked or discussed before PR approval.

General

Tests are passing locally and on Github
Documentation is up to date and changelog has been updated if appropriate
You can yarn panic && yarn bootstrap or docker-compose up --build and FEM works as expected
FEM works in all major desktop browsers: Firefox, Chrome, Edge, Safari (Use Browserstack account as needed)
FEM works in a mobile browser

General UX

Example Staging Project: i-fancy-cats

All pages of a FEM project load: Home Page, Classify Page, and About Pages
Can submit a classification
Can sign-in and sign-out
The component is accessible
- Can be used with a screen reader BBC guide to VoiceOver
- Can be used from the keyboard WebAIM guide to keyboard testing
- It is passing accessibility checks in the storybook

Bug Fix

The PR creator has listed user actions to use when testing if bug is fixed
The bug is fixed
Unit tests are added or updated

eatyourgreens · 2024-11-07T09:46:33Z

packages/lib-classifier/src/store/Classification/ClassificationMetadata.js

+        const userLanguage = getRoot(self)?.locale
+        if (userLanguage) {
+          self.update({ userLanguage })
+        }


This isn't related to the timestamps bug but fixes the store error in #6448.

coveralls · 2024-11-07T09:51:24Z

coverage: 77.854% (+0.02%) from 77.838%
when pulling 8bed133 on eatyourgreens:test-classification-timestamps
into 0e8655e on zooniverse:master.

eatyourgreens · 2024-11-07T12:38:49Z

Tests are passing but started_at is still wrong when I check this with I Fancy Cats. Setting it back to draft while I look at that.

eatyourgreens · 2024-11-07T13:14:12Z

Figured it out: A new, empty annotation is created when the subject first loads, so the code needs to wait until annotation._inProgress is true before setting the classification start time. That happens when someone first interacts with a task eg. selecting an answer to a question.

EDIT: for a survey task, annotation starts when annotation._choiceInProgress is true. All other task annotations start when annotation._inProgress is true.

eatyourgreens · 2024-11-07T13:44:33Z

packages/lib-classifier/src/store/Classification/Classification.js

+    function _onAnnotationsChange () {
+      // set started at when inProgress changes from false to true
+      if (self.inProgress) {
+        self.setStartedAt()


If you comment this out, the tests should fail.

eatyourgreens · 2024-11-07T15:35:31Z

packages/lib-classifier/src/store/Classification/Classification.js

+    get inProgress() {
+      let inProgress = false
+      self.annotations.forEach(annotation => {
+        inProgress ||= annotation._inProgress || annotation._choiceInProgress


_choiceInProgress is used by the survey task. All other tasks use _inProgress.

- [x] Add tests for classification `started_at` and `finished_at` timestamps. - [ ] fix the failing test for classification start times.

A new, empty annotation is created when the subject first loads, so wait until `annotation._inProgress` is true before setting the classification start time.

eatyourgreens commented Nov 7, 2024

View reviewed changes

eatyourgreens marked this pull request as ready for review November 7, 2024 09:48

eatyourgreens force-pushed the test-classification-timestamps branch from 9d9cc4a to 616ece1 Compare November 7, 2024 09:56

eatyourgreens changed the title ~~test(lib-classifier): test classification timestamps~~ fix(lib-classifier): record classification times between the first and last annotation Nov 7, 2024

eatyourgreens force-pushed the test-classification-timestamps branch from c46b663 to d6fbdbe Compare November 7, 2024 11:03

eatyourgreens mentioned this pull request Nov 7, 2024

Incorrect timing on the stats page #6441

Closed

eatyourgreens marked this pull request as draft November 7, 2024 12:37

eatyourgreens marked this pull request as ready for review November 7, 2024 13:21

eatyourgreens commented Nov 7, 2024

View reviewed changes

eatyourgreens force-pushed the test-classification-timestamps branch from 837a0bd to 9cf45c4 Compare November 7, 2024 13:47

eatyourgreens commented Nov 7, 2024

View reviewed changes

This was referenced Nov 8, 2024

Classification started_at can be off by hours or days zooniverse/Panoptes-Front-End#7215

Open

fix(classifier): set started_at timestamp when annotation starts zooniverse/Panoptes-Front-End#7216

Open

eatyourgreens force-pushed the test-classification-timestamps branch from 0d754ca to de95e8e Compare November 11, 2024 09:04

goplayoutside3 self-assigned this Nov 12, 2024

eatyourgreens added 5 commits November 18, 2024 18:40

test(lib-classifier): test classification timestamps

f41911f

- [x] Add tests for classification `started_at` and `finished_at` timestamps. - [ ] fix the failing test for classification start times.

Update startedAt when the first annotation is made

09e4229

test the length of the mock classification

d4562d3

Set started_at when annotation._inProgress changes

e1cf963

A new, empty annotation is created when the subject first loads, so wait until `annotation._inProgress` is true before setting the classification start time.

support the survey task's _choiceInProgress flag

8bed133

eatyourgreens force-pushed the test-classification-timestamps branch from de95e8e to 8bed133 Compare November 18, 2024 18:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(lib-classifier): record classification times between the first and last annotation #6449

fix(lib-classifier): record classification times between the first and last annotation #6449

eatyourgreens commented Nov 7, 2024 •

edited

Loading

eatyourgreens Nov 7, 2024 •

edited

Loading

coveralls commented Nov 7, 2024 •

edited

Loading

eatyourgreens commented Nov 7, 2024

eatyourgreens commented Nov 7, 2024 •

edited

Loading

eatyourgreens Nov 7, 2024

eatyourgreens Nov 7, 2024

fix(lib-classifier): record classification times between the first and last annotation #6449

Are you sure you want to change the base?

fix(lib-classifier): record classification times between the first and last annotation #6449

Conversation

eatyourgreens commented Nov 7, 2024 • edited Loading

Package

Linked Issue and/or Talk Post

How to Review

Checklist

General

General UX

Bug Fix

eatyourgreens Nov 7, 2024 • edited Loading

Choose a reason for hiding this comment

coveralls commented Nov 7, 2024 • edited Loading

eatyourgreens commented Nov 7, 2024

eatyourgreens commented Nov 7, 2024 • edited Loading

eatyourgreens Nov 7, 2024

Choose a reason for hiding this comment

eatyourgreens Nov 7, 2024

Choose a reason for hiding this comment

eatyourgreens commented Nov 7, 2024 •

edited

Loading

eatyourgreens Nov 7, 2024 •

edited

Loading

coveralls commented Nov 7, 2024 •

edited

Loading

eatyourgreens commented Nov 7, 2024 •

edited

Loading