Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PM-10560] Create notification database storage #4688

Merged
merged 28 commits into from
Sep 9, 2024
Merged

Conversation

Thomas-Avery
Copy link
Contributor

🎟️ Tracking

https://bitwarden.atlassian.net/browse/PM-10560

📔 Objective

The purpose of this PR is to add the database and repository layer for the notification center work.

⏰ Reminders before review

  • Contributor guidelines followed
  • All formatters and local linters executed and passed
  • Written new unit and / or integration tests where applicable
  • Protected functional changes with optionality (feature flags)
  • Used internationalization (i18n) for all UI strings
  • CI builds passed
  • Communicated to DevOps any deployment requirements
  • Updated any necessary documentation (Confluence, contributing docs) or informed the documentation team

🦮 Reviewer guidelines

  • 👍 (:+1:) or similar for great changes
  • 📝 (:memo:) or ℹ️ (:information_source:) for notes or general info
  • ❓ (:question:) for questions
  • 🤔 (:thinking:) or 💭 (:thought_balloon:) for more open inquiry that's not quite a confirmed issue and could potentially benefit from discussion
  • 🎨 (:art:) for suggestions / improvements
  • ❌ (:x:) or ⚠️ (:warning:) for more significant problems or concerns needing attention
  • 🌱 (:seedling:) or ♻️ (:recycle:) for future improvements or indications of technical debt
  • ⛏ (:pick:) for minor or nitpick changes

Copy link

codecov bot commented Aug 21, 2024

Codecov Report

Attention: Patch coverage is 20.39801% with 160 lines in your changes missing coverage. Please review.

Project coverage is 41.76%. Comparing base (b103e8f) to head (4341f10).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...ationCenter/Repositories/NotificationRepository.cs 0.00% 61 Missing ⚠️
...enter/Repositories/NotificationStatusRepository.cs 0.00% 36 Missing ⚠️
...enter/Repositories/NotificationStatusRepository.cs 0.00% 24 Missing ⚠️
...ationCenter/Repositories/NotificationRepository.cs 0.00% 14 Missing ⚠️
...c/Core/NotificationCenter/Entities/Notification.cs 0.00% 13 Missing ⚠️
.../NotificationCenter/Entities/NotificationStatus.cs 0.00% 4 Missing ⚠️
...onCenter/Models/Filter/NotificationStatusFilter.cs 0.00% 2 Missing ⚠️
...ucture.Dapper/DapperServiceCollectionExtensions.cs 0.00% 2 Missing ⚠️
...ramework/NotificationCenter/Models/Notification.cs 75.00% 2 Missing ⚠️
...rk/NotificationCenter/Models/NotificationStatus.cs 75.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4688      +/-   ##
==========================================
- Coverage   41.83%   41.76%   -0.07%     
==========================================
  Files        1294     1305      +11     
  Lines       61740    61941     +201     
  Branches     5695     5703       +8     
==========================================
+ Hits        25827    25868      +41     
- Misses      34722    34882     +160     
  Partials     1191     1191              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

github-actions bot commented Aug 21, 2024

Logo
Checkmarx One – Scan Summary & Details35fb76a5-58d4-49fb-9b02-22d1d99ef64c

New Issues

Severity Issue Source File / Package Checkmarx Insight
MEDIUM Privacy_Violation /src/Api/Auth/Controllers/TwoFactorController.cs: 488 Attack Vector
MEDIUM Privacy_Violation /src/Api/Vault/Models/Request/CipherRequestModel.cs: 169 Attack Vector
MEDIUM Privacy_Violation /src/Api/Controllers/DevicesController.cs: 133 Attack Vector
MEDIUM Privacy_Violation /src/Api/Auth/Controllers/AccountsController.cs: 549 Attack Vector
MEDIUM Privacy_Violation /src/Api/Auth/Controllers/AccountsController.cs: 829 Attack Vector
MEDIUM Privacy_Violation /src/Api/Auth/Controllers/AccountsController.cs: 412 Attack Vector
MEDIUM Privacy_Violation /src/Api/Vault/Controllers/CiphersController.cs: 906 Attack Vector
MEDIUM Privacy_Violation /src/Api/Auth/Controllers/AccountsController.cs: 847 Attack Vector
MEDIUM Privacy_Violation /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 259 Attack Vector
MEDIUM Privacy_Violation /src/Api/Controllers/DevicesController.cs: 159 Attack Vector
MEDIUM Privacy_Violation /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 403 Attack Vector
MEDIUM Privacy_Violation /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 350 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164 Attack Vector
LOW Log_Forging /src/Api/Vault/Controllers/CiphersController.cs: 217 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 445 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 118 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 165 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 174 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 191 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 200 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 231 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 249 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 288 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 297 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 306 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 323 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 332 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 340 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 393 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 404 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 412 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 423 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/TwoFactorController.cs: 103 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 821 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 839 Attack Vector
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 541 Attack Vector
LOW Log_Forging /src/Api/Vault/Controllers/CiphersController.cs: 898 Attack Vector
LOW Log_Forging /src/Api/Controllers/DevicesController.cs: 124 Attack Vector
LOW Log_Forging /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 239 Attack Vector
LOW Log_Forging /src/Api/Controllers/DevicesController.cs: 150 Attack Vector
LOW Log_Forging /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 315 Attack Vector
LOW Log_Forging /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 378 Attack Vector

Fixed Issues

Severity Issue Source File / Package
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 121
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 107
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 74
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 61
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/MembersController.cs: 98
MEDIUM CSRF /src/Api/Auth/Controllers/TwoFactorController.cs: 118
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/MembersController.cs: 98
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 229
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 478
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 283
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 478
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 176
MEDIUM CSRF /src/Api/Billing/Public/Controllers/OrganizationController.cs: 47
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 551
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 503
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 565
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationsController.cs: 445
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 357
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 337
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 74
MEDIUM CSRF /src/Api/Controllers/DevicesController.cs: 61
MEDIUM CSRF /src/Api/AdminConsole/Controllers/OrganizationUsersController.cs: 255
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 929
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 107
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 121
MEDIUM CSRF /src/Api/AdminConsole/Public/Controllers/OrganizationController.cs: 43
MEDIUM CSRF /src/Api/Vault/Controllers/CiphersController.cs: 676
MEDIUM Privacy_Violation /src/Api/AdminConsole/Controllers/OrganizationAuthRequestsController.cs: 59
MEDIUM Privacy_Violation /src/Core/Auth/UserFeatures/TdeOffboardingPassword/TdeOffboardingPasswordCommand.cs: 81
MEDIUM Privacy_Violation /src/Core/Auth/UserFeatures/TdeOffboardingPassword/TdeOffboardingPasswordCommand.cs: 81
MEDIUM Privacy_Violation /src/Core/Auth/UserFeatures/UserMasterPassword/SetInitialMasterPasswordCommand.cs: 59
MEDIUM Privacy_Violation /src/Core/Auth/UserFeatures/UserMasterPassword/SetInitialMasterPasswordCommand.cs: 59
MEDIUM Privacy_Violation /src/Api/AdminConsole/Controllers/OrganizationAuthRequestsController.cs: 59
LOW Log_Forging /src/Api/AdminConsole/Controllers/ProvidersController.cs: 72
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 882
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164
LOW Log_Forging /src/Billing/Controllers/StripeController.cs: 164
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 246
LOW Log_Forging /src/Api/Auth/Controllers/AccountsController.cs: 366

@mzieniukbw mzieniukbw marked this pull request as ready for review September 3, 2024 09:46
@mzieniukbw mzieniukbw requested a review from a team as a code owner September 3, 2024 09:46
Copy link
Contributor

@withinfocus withinfocus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few things, plus I believe you're still working with @rkac-bw on the DB tuning right?

@mzieniukbw
Copy link
Contributor

@withinfocus On DB optimisations I have provided the new schema, queries and testing data to @rkac-bw for further investigation. We have few ideas, but nothing seems to be set in stone yet.
I suggest we do the DB optimisations at later stage, as part of separate Jira ticket, so we can continue with service and api layers in the meantime.

Copy link
Contributor

@withinfocus withinfocus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few small things but very close!

}

var notifications = await notificationQuery
.OrderByDescending(n => n.Priority)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ I am anticipating that we'll tweak this in the future.

withinfocus
withinfocus previously approved these changes Sep 9, 2024
Copy link
Contributor

@withinfocus withinfocus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing work here!

@mzieniukbw
Copy link
Contributor

mzieniukbw commented Sep 9, 2024

@withinfocus i fixed the SP that i broke in previous commits, see ddbacac.
In essence by removing the Notification_ReadByUserId and only leaving Notification_ReadByUserIdAndStatus i forgot to update it, to also work when no status filter is provided.

withinfocus
withinfocus previously approved these changes Sep 9, 2024
@mzieniukbw
Copy link
Contributor

@withinfocus i have pushed one more change to that SP with bec9dec, to closer meet the requirement of status filtering:

Filter not provided = no check on notification status
Filter provided = only notifications with status

  • is read
    • null / not provided: no check on that column
    • true: read date is populated
    • false: read date is null
  • is deleted
    • same thing

Copy link
Contributor

@withinfocus withinfocus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a chat with @rkac-bw and it's good to go, with a potential follow-up for tuning on performance.

@withinfocus
Copy link
Contributor

I've noticed a couple database testing issues with locks or the instance not being up in time which is strange. I wonder if we need to adjust the delays.

@mzieniukbw mzieniukbw merged commit 4c0f8d5 into main Sep 9, 2024
54 checks passed
@mzieniukbw mzieniukbw deleted the km/pm-10560 branch September 9, 2024 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants