Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sanity check data before start of processing #33

Open
tiadams opened this issue Mar 8, 2024 · 0 comments
Open

Sanity check data before start of processing #33

tiadams opened this issue Mar 8, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@tiadams
Copy link
Member

tiadams commented Mar 8, 2024

Check data for possible issues like:

  • Wrong/ non aligning column names
  • Malformed/non-symetric structure of tables
  • Too many data points (that may potentially break the frontpage due to timeouts)

Return appropriate warnings when the data is not in a salvageable format and do not start the processing if triggered. Also think about subsampling data if the provided tables are to big (e.g. only process 1000 random rows)

@tiadams tiadams added the enhancement New feature or request label Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant