Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental results on ARC-C subset for challeging reasoning? #47

Open
tongyx361 opened this issue Jul 18, 2024 · 0 comments
Open

Experimental results on ARC-C subset for challeging reasoning? #47

tongyx361 opened this issue Jul 18, 2024 · 0 comments

Comments

@tongyx361
Copy link

“Notably, except for ORPO, almost all approaches lead to consistent drops in one or more settings.”
But ARC shows that commonsense reasoning is improved.
However, the contradiction between reasoning tasks is counter-intuitive.
I wonder if this is caused by the difficulty difference.
So I am curious about the results on the ARC-C subset.
It would be so nice of you if you could curate related data and report the numbers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant