Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrates ACL Sentence Piece Model for calculating affinity scores #72

Open
wants to merge 180 commits into
base: master
Choose a base branch
from

Conversation

purujitgoyal
Copy link
Contributor

@purujitgoyal purujitgoyal commented Sep 16, 2021

Only files in the expertise/models/acl_scorer need to be reviewed. Everything else is formatting changes.

@purujitgoyal
Copy link
Contributor Author

@neubig this branch is stable, however, we are parallely working on another project to expose expertise as an API. We plan to release both of them together, most probably next week. While we finish working on that, it'd be great if you can use this branch to test whether the model's working correctly or not.

@purujitgoyal purujitgoyal marked this pull request as ready for review October 20, 2021 01:22
Base automatically changed from feature/api-dev to master October 28, 2021 19:20
@neubig
Copy link

neubig commented Nov 9, 2021

Hi! We've been looking at this, but seeing some unexpected behavior (namely the overall scores are lower than we expect). I'd like to dig into this a bit more, but this commit has so many files changed it's a bit hard for me to tell which parts I should be looking at. I wonder if you could rebase this branch so only the changes that actually are relevant to the ACL scorer would be visible?

@neubig
Copy link

neubig commented Nov 10, 2021

Also, to clarify what the issue is here: when running the ACL reviewer matching code we have previously gotten relatively high matching scores of 0.7 or so, but when running the code here we get lowish scores of 0.3. We're not sure exactly where the difference is coming from, but it'd be nice to be able to read the changed parts and try to debug.

@melisabok
Copy link
Member

Hi @neubig , we are working on this, it seems @purujitgoyal fixed formatting issues in the same PR. We are creating a new branch with only the changes needed for ACL model.

@neubig
Copy link

neubig commented Nov 10, 2021

Thanks a lot!

@melisabok
Copy link
Member

This is the draft PR with changes needed to run the ACL scores: #80

Feel free to make code suggestions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate Standard ACL Paper Matching Scorer
4 participants