You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello, thank you for the great work! I'm checking your prompts, especially your recently released prompt for GQA (prompts/benchmark/gqa.yaml). I see two types of example programs: some for demonstrating API usage, and some as in-context demonstration. Could you clarify how you curate these examples? Are they automatically generated or manually curated, and do you have specific criteria when selecting these examples?
Also, I see that some of the example questions are from GQA training set, and a few are even from the test set (for example, "Is that blanket to the right of a pillow"). Is this a bug? Does it potentially leak the test set?
The text was updated successfully, but these errors were encountered:
Hello, thank you for the great work! I'm checking your prompts, especially your recently released prompt for GQA (
prompts/benchmark/gqa.yaml
). I see two types of example programs: some for demonstrating API usage, and some as in-context demonstration. Could you clarify how you curate these examples? Are they automatically generated or manually curated, and do you have specific criteria when selecting these examples?Also, I see that some of the example questions are from GQA training set, and a few are even from the test set (for example, "Is that blanket to the right of a pillow"). Is this a bug? Does it potentially leak the test set?
The text was updated successfully, but these errors were encountered: