-
Notifications
You must be signed in to change notification settings - Fork 922
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Load and store embedded dataset #5370
base: feature/cris/malicious-site-protection/intercept-requests
Are you sure you want to change the base?
Load and store embedded dataset #5370
Conversation
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
e99a589
to
41dbcee
Compare
3932394
to
13cfb3a
Compare
41dbcee
to
45d1aaa
Compare
13cfb3a
to
8d0f3b0
Compare
45d1aaa
to
ca1717c
Compare
8d0f3b0
to
0b7f33d
Compare
212e3b7
to
7c5e5c2
Compare
615f371
to
7fd8100
Compare
7c5e5c2
to
54f233d
Compare
7fd8100
to
c24dff3
Compare
54f233d
to
0f7697a
Compare
0f7697a
to
7c8c545
Compare
3fb96e8
to
601db6b
Compare
d585fa1
to
e310dad
Compare
2c3627c
to
cfc8cea
Compare
1ea8bd3
to
768449f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some questions about the insert, if that's the expected behavior I'm good here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NIT: you can extract some logic here to a generic method since every method has a similar structure.
private inline fun loadAndParseEmbeddedData(@rawres file: Int): T? {
return try {
val data = loadEmbeddedData(file)
val adapter = moshi.adapter(T::class.java)
adapter.fromJson(String(data, Charsets.UTF_8))
} catch (e: Exception) {
Timber.e(e, "Failed to fetch embedded data for resource: $file")
null
}
}
malwareFiltersRevision = max(lastRevision?.malwareFiltersRevision ?: 0, malwareFilterSetRevision ?: 0), | ||
), | ||
) | ||
insertHashPrefixes(phishingHashPrefixes.map { HashPrefixEntity(hashPrefix = it, type = "phishing") }) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
QQ about inserts here: This will insert but not replace right? so it's additive and will replace only if there's a conflict. But if no conflict, old data will still be present. Example: if we have A, B, C, D, and receive B, D, E ...DB state will be A, B, C, D, E. Shouldn't be the new state B, D, E?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cmonfortep, you're right. In a previous implementation revisions were a FK, so by removing them, we would remove filters and hash prefixes, but that's not the case anymore. I'll fix it
db4d9bc
to
bad45a7
Compare
768449f
to
d6c864d
Compare
Task/Issue URL: https://app.asana.com/0/1205008441501016/1208870183150417/f
Description
Load and store embedded dataset for malicious site protection
Steps to test this PR
UI changes
no UI changes