Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suppression state propagation #328

Merged
merged 15 commits into from
Jan 12, 2024
Merged

Conversation

SandyRogers
Copy link
Member

@SandyRogers SandyRogers commented Sep 20, 2023

This PR:

  • adds a "dependency tree" to propagate suppression states between models.
    • e.g. when a study is suppressed, the samples, runs, assemblies, analyses that belong to that study are also suppressed
    • the complexity of this is really so that it can be done in bulk operations. It could be simplified if we were happy to have slow jobs that update objects one by one.
  • adds a suppression state of "Ancestor Suppressed" to keep track of these events.
    • e.g. so that a descendant (say, a sample) can remain suppressed even if its ancestor (say, a study) is unsuppressed – only descendants with the "Ancestor Suppressed" state should be unsuppressed.
  • adds a select_related_only queryset method, to reduce the amount of data selected by complex queries like the AnalysisJob.
    • This lets us keep necessary joins (like getting the study.accession for an analysisjob), without querying the many other fields on sample that are never picked by the serializer.

Together these simplify the querying of some models, because suppression states of related objects do not have to be checked at query time and a slightly smaller query is used. This has been implemented on the Analysis endpoint currently, and seems to give a speedup of around 20%... to get more improvement, we need to do something similar with the is_private states. These are more complicated because we need to add submission_account info on other models, and check this depending on the request user. I have not done that in this PR, but it might be worth a go in future...

@SandyRogers SandyRogers marked this pull request as ready for review September 21, 2023 15:22
Copy link
Member

@mberacochea mberacochea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great stuff @SandyRogers . I've added 2 minor comments. I think @MGS-sails should give this PR a look before merging.

Co-authored-by: Martín Beracochea <[email protected]>
Copy link
Contributor

@MGS-sails MGS-sails left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple of comments.
Thanks

self.is_suppressed = True
self.suppressed_at = timezone.now()
self.suppression_reason = suppression_reason
if save:
self.save()
if propagate:
Copy link
Contributor

@MGS-sails MGS-sails Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SandyRogers might it be better to return here, if not propagate ?
This might help to reduce nesting inside the if block and the for loops inside it.

self.is_suppressed = False
self.suppressed_at = None
self.suppression_reason = None
if save:
self.save()
if propagate:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SandyRogers this might be overkill, and please ignore if it's going to cause an uneccessary delay,
but I think in the long term, it will be better to refactor the suppress and unsuppress methods.
It seems like they are doing the same thing, with only slight variations.
So it might be better if they both call a generalised method, but with different arguments.

@mberacochea
Copy link
Member

@SandyRogers should we merge this one?

@SandyRogers SandyRogers merged commit dd827d9 into develop Jan 12, 2024
4 checks passed
@SandyRogers SandyRogers deleted the suppression-state-propagation branch January 12, 2024 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants