Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added extra experiments - mainly around macro chunking #16

Merged
merged 32 commits into from
Oct 2, 2024
Merged

Added extra experiments - mainly around macro chunking #16

merged 32 commits into from
Oct 2, 2024

Commits on Sep 26, 2024

  1. initial experiment setup

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7ce00ae View commit details
    Browse the repository at this point in the history

  2. changed len tokens

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    ffed829 View commit details
    Browse the repository at this point in the history

  3. soft boundary correctly overlaps for later batches

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b467575 View commit details
    Browse the repository at this point in the history

  4. removed incorrect soft

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    d0ef88a View commit details
    Browse the repository at this point in the history

  5. added loop over overlap sizes

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7985c66 View commit details
    Browse the repository at this point in the history

  6. added results

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    56a6a41 View commit details
    Browse the repository at this point in the history

  7. experiment for WikimQA

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    f3eb17d View commit details
    Browse the repository at this point in the history

  8. added narrativeQA task

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    50b2b00 View commit details
    Browse the repository at this point in the history

  9. added remainder of longembed datasets

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    3f226a5 View commit details
    Browse the repository at this point in the history

  10. typo

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    19ab752 View commit details
    Browse the repository at this point in the history

  11. more tasks for soft/hard

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    57cc8ba View commit details
    Browse the repository at this point in the history

  12. experiments

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    9043a3f View commit details
    Browse the repository at this point in the history

  13. fix merge

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e1d66e1 View commit details
    Browse the repository at this point in the history

  14. added macro chunk experiment file

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0a4a27e View commit details
    Browse the repository at this point in the history

  15. for merge

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    043eec6 View commit details
    Browse the repository at this point in the history

  16. merge def values

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    f960b15 View commit details
    Browse the repository at this point in the history

  17. chunk size results

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8e0c80d View commit details
    Browse the repository at this point in the history

  18. added benchmark files for macro chunks

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    753b603 View commit details
    Browse the repository at this point in the history

  19. removed raw results

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    6b52c8c View commit details
    Browse the repository at this point in the history

  20. added plotting files for results, requires running them first

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    9fc7d52 View commit details
    Browse the repository at this point in the history

  21. renamed file

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7cff570 View commit details
    Browse the repository at this point in the history

  22. renamed file

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b831989 View commit details
    Browse the repository at this point in the history

  23. added plt.show()

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    1eccfe5 View commit details
    Browse the repository at this point in the history

  24. renamed to macro chunking

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4219ca6 View commit details
    Browse the repository at this point in the history

  25. moved file

    @dannyjameswilliams
    dannyjameswilliams committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    ad9f37c View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. Update chunked_pooling/chunking.py according to comment 

    Co-authored-by: Michael Günther <[email protected]>
    @dannyjameswilliams @guenthermi
    dannyjameswilliams and guenthermi authored Oct 1, 2024
    Configuration menu
    Copy the full SHA
    ada2a91 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. updated main experiment file with long late chunking

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    d4f99ce View commit details
    Browse the repository at this point in the history

  2. remove redundant macro chunking file

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    936ae51 View commit details
    Browse the repository at this point in the history

  3. updated default to truncation (8192)

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    1ed7fb6 View commit details
    Browse the repository at this point in the history

  4. updated error message/print statement

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    abab7fa View commit details
    Browse the repository at this point in the history

  5. changed how local llm is loaded

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    b46d469 View commit details
    Browse the repository at this point in the history

  6. removed comment on pip and update default model to phi

    @dannyjameswilliams
    dannyjameswilliams committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    b64c2a6 View commit details
    Browse the repository at this point in the history