Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unittests tests.test_bigbio_hub bioasq_task_b are failing #924

Closed
mart1nro opened this issue Jun 28, 2024 · 1 comment
Closed

Unittests tests.test_bigbio_hub bioasq_task_b are failing #924

mart1nro opened this issue Jun 28, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@mart1nro
Copy link
Contributor

Describe the bug

"runTest (main.TestDataLoader) [Check multiple choice]" is failing. This is because "choices" is always set to [] for each record in _generate_examples, even for yesno questions:

biomedical/bigbio/hub/hub_repos/bioasq_task_b/bioasq_task_b.py

Line 791 in 5c9e606

"choices": [],

Tests for bioasq10b are failing with datasets.exceptions.DatasetGenerationError because there is no "BioASQ-training10b" folder in "BioASQ-training10b.zip" and the path "BioASQ-training10b/training10b.json" is incorrect:

biomedical/bigbio/hub/hub_repos/bioasq_task_b/bioasq_task_b.py

Line 698 in 5c9e606

"bioasq_10b": "BioASQ-training10b/training10b.json",

The correct path should be "training10b.json", similar to the bioasq_8b example.

Steps to reproduce the bug

Download dataset zips of task b from http://participants-area.bioasq.org/datasets/ and put them into a data_dir

python -m tests.test_bigbio_hub bioasq_task_b --data_dir /home/robert/Desktop/bioasq --test_local

Expected results

All tests pass.

Actual results

See attached test_output.txt
test_output.txt

Environment info

  • datasets version: 2.20.0
  • Platform: Linux-6.9.5-arch1-1-x86_64-with-glibc2.34
  • Python version: 3.8.19
  • huggingface_hub version: 0.23.4
  • PyArrow version: 16.1.0
  • Pandas version: 2.0.3
  • fsspec version: 2024.5.0
@mart1nro mart1nro added the bug Something isn't working label Jun 28, 2024
@mart1nro mart1nro mentioned this issue Jun 28, 2024
Merged
8 tasks
phlobo pushed a commit that referenced this issue Jul 24, 2024
@mart1nro
Added 11b version of BioASQ to existing bioasq_task_b.py. Closes #925 …
03d2d96 
…and #924 (#926)

* implemented bioasq 11b

* Normalized yesno answers and added choices to satisfy

runTest (__main__.TestDataLoader) [Check multiple choice]

Note: This test used to fail also for previous iterations of the dataset but is now satisfied.

* fixed path of training10b.json
@phlobo
Copy link
Collaborator

phlobo commented Jul 24, 2024

closed by #926

@phlobo phlobo closed this as completed Jul 24, 2024
phlobo pushed a commit to davidkartchner/biomedical that referenced this issue Oct 21, 2024
@mart1nro @phlobo
Added 11b version of BioASQ to existing bioasq_task_b.py. Closes bigs…
a41cb50 
…cience-workshop#925 and bigscience-workshop#924 (bigscience-workshop#926)

* implemented bioasq 11b

* Normalized yesno answers and added choices to satisfy

runTest (__main__.TestDataLoader) [Check multiple choice]

Note: This test used to fail also for previous iterations of the dataset but is now satisfied.

* fixed path of training10b.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@phlobo @mart1nro