-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add remaining GET endpoints to parity check #1847
Commits on Oct 23, 2024
-
Optimise the
ResponseComparison
deletion on re-runDeleting the existing parity check results can be slow as they can be very large records in the database. We don't care about callbacks or validation here, so we can optimise the deletion with `delete_all`.
Configuration menu - View commit details
-
Copy full SHA for 1919ad1 - Browse repository at this point
Copy the full SHA 1919ad1View commit details -
Support checking an individual resource
We need to be able to request an individual resource as part of the parity check. The difficulty with this is that we need a valid ID for each lead provider. Instead of specifying these manually, I've opted to allow dynamic substitution by evaluating code from the YAML file. We can now specify an ID attribute in the path with `:id` and a corresponding option in the YAML file that, when evaluated (in the context of the `Client`) should return a valid id for the lead provider.
Configuration menu - View commit details
-
Copy full SHA for 61891c9 - Browse repository at this point
Copy the full SHA 61891c9View commit details
Commits on Oct 24, 2024
-
Add remaining GET endpoints to YAML file
Add all the listing and individual GET resource endpoints to the YAML file.
Configuration menu - View commit details
-
Copy full SHA for 60b95bd - Browse repository at this point
Copy the full SHA 60b95bdView commit details -
Convert CSV responses to hexdigest
The CSV responses are very large and cause issues when trying to insert into Postgres. To be able to include them for now we are converting the response CSV to a hexdigest that can be stored and compared. If we want more details we will have to make the requests manually to inspect the responses.
Configuration menu - View commit details
-
Copy full SHA for 1dbfa65 - Browse repository at this point
Copy the full SHA 1dbfa65View commit details -
Stop paginating on first mismatch
In order to finish the parity check as soon as possible, we stop paginating responses as soon as we encounter a discrepency. A mismatch in a earlier page is likely to effect later pages anyway, so this is all we will be interested in (and other issues would be surfaced on future runs when its fixed).
Configuration menu - View commit details
-
Copy full SHA for 4086426 - Browse repository at this point
Copy the full SHA 4086426View commit details -
These cause issues with timeouts on the database; I'm not sure why as we hash the large CSV response so it shouldn't cause issues. Remove CSV endpoints; if we want to include them we can add another ticket to make them work.
Configuration menu - View commit details
-
Copy full SHA for c752d79 - Browse repository at this point
Copy the full SHA c752d79View commit details -
Persist formatted path to ResponseComparison
As the id we grab for the lead provider is random, we need to persist it in the `ResponseComparison` to be able to replicate the request. Add `id` to `request_path` before saving (avoids adding a new attribute to `ResponseComparison`).
Configuration menu - View commit details
-
Copy full SHA for 0d74ceb - Browse repository at this point
Copy the full SHA 0d74cebView commit details -
When we output the JSON diff we want to pretty format it or we end up with a single line which is hard to compare.
Configuration menu - View commit details
-
Copy full SHA for 6ebc3f5 - Browse repository at this point
Copy the full SHA 6ebc3f5View commit details -
If the response is JSON we deep sort the resulting hash so that we can compare the contents irrespective of order (as the new serializers in NPQ reg may serialize the fields in a different order).
Configuration menu - View commit details
-
Copy full SHA for e57502d - Browse repository at this point
Copy the full SHA e57502dView commit details -
Configuration menu - View commit details
-
Copy full SHA for ffb3de2 - Browse repository at this point
Copy the full SHA ffb3de2View commit details -
Replace use of eval with safer solution
Instead of evaluating the YAML contents directly, we can specify a method in the YAML file and call in the `Client`. This also has the benefit of being able to more easily test the possible `id` options.
Configuration menu - View commit details
-
Copy full SHA for abba51f - Browse repository at this point
Copy the full SHA abba51fView commit details