New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Confirm loss and eval_fn() #7

Open

plpxsk opened this issue Oct 21, 2024 · 1 comment

Assignees

Owner

plpxsk commented Oct 21, 2024 •

edited

Loading

Confirm valid implementation

References:

Loss? Loss is:
Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.

https://huggingface.co/transformers/v4.9.2/model_doc/bert.html?highlight=bertforquestionanswering
https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/models/bert/modeling_bert.py#L1996

google:
https://github.com/google-research/bert/blob/master/run_squad.py#L646

plpxsk self-assigned this

plpxsk mentioned this issue

Refactor repo #6

Merged

6 tasks

Owner Author

plpxsk commented Oct 25, 2024

Losses are dropping, and more training means better answers, so seems to be ok for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment