Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How we train saes replication #123

Merged
merged 39 commits into from
May 7, 2024
Merged

How we train saes replication #123

merged 39 commits into from
May 7, 2024

Commits on May 6, 2024

  1. rebase

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    cee824a View commit details
    Browse the repository at this point in the history

  2. l1 scheduler, clip grad norm

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    3223b3b View commit details
    Browse the repository at this point in the history

  3. rebase

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    d6a670a View commit details
    Browse the repository at this point in the history

  4. add provisional ability to normalize activations

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    f3e81ad View commit details
    Browse the repository at this point in the history

  5. notebook

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    960affa View commit details
    Browse the repository at this point in the history

  6. change heuristic norm init to constant, report b_e and W_dec norms (f… 

    …ix tests later)
    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    cdde456 View commit details
    Browse the repository at this point in the history

  7. fix mse calculation

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    51d8281 View commit details
    Browse the repository at this point in the history

  8. add benchmark test

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    336f0dc View commit details
    Browse the repository at this point in the history

  9. profiling the code

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    5eb9cdc View commit details
    Browse the repository at this point in the history

  10. update heuristic init to 0.1

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    e882f5e View commit details
    Browse the repository at this point in the history

  11. make tests pass device issue

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    cde300c View commit details
    Browse the repository at this point in the history

  12. hopefully handle resuming conflict correctly

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    85d2102 View commit details
    Browse the repository at this point in the history

  13. continue rebase

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    078c6e6 View commit details
    Browse the repository at this point in the history

  14. use better args in benchmark

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    72b7509 View commit details
    Browse the repository at this point in the history

  15. remove stack in get activations

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    b9ec8ea View commit details
    Browse the repository at this point in the history

  16. broken! improve CA runner

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    3e9d725 View commit details
    Browse the repository at this point in the history

  17. fix zeros bug

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    1bf5ddb View commit details
    Browse the repository at this point in the history

  18. fix zeros bug

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    7456b44 View commit details
    Browse the repository at this point in the history

  19. get cache activation runner working and add some tests

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    46e569e View commit details
    Browse the repository at this point in the history

  20. WIP

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    81b8a0f View commit details
    Browse the repository at this point in the history

  21. current scripts

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    300d07f View commit details
    Browse the repository at this point in the history

  22. string

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    cb7d4ff View commit details
    Browse the repository at this point in the history

  23. make configs easier to access, gitignore cached activations

    @hijohnnylin @jbloomAus
    hijohnnylin authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    f48c5e4 View commit details
    Browse the repository at this point in the history

  24. use correct path

    @hijohnnylin @jbloomAus
    hijohnnylin authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    57d86fa View commit details
    Browse the repository at this point in the history

  25. add training steps to path

    @hijohnnylin @jbloomAus
    hijohnnylin authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    ce74a79 View commit details
    Browse the repository at this point in the history

  26. avoid ghost grad tensor casting

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    8988ec3 View commit details
    Browse the repository at this point in the history

  27. enable download of full dataset if desired

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    168a066 View commit details
    Browse the repository at this point in the history

  28. current scripts

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    db91215 View commit details
    Browse the repository at this point in the history

  29. ignore fixture in benchmark

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    4e1b33b View commit details
    Browse the repository at this point in the history

  30. add benchmark for cache activation runner

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    548c34d View commit details
    Browse the repository at this point in the history

  31. ignore prof

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    01794f4 View commit details
    Browse the repository at this point in the history

  32. minor fixes

    @jbloomAus
    jbloom-md authored and jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    b6816d6 View commit details
    Browse the repository at this point in the history

  33. current script

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    7c9f32a View commit details
    Browse the repository at this point in the history

  34. control

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    1407d50 View commit details
    Browse the repository at this point in the history

  35. reformat

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    d27f9ff View commit details
    Browse the repository at this point in the history

  36. try to get tests passing

    @jbloomAus
    jbloomAus committed May 6, 2024
    Configuration menu
    Copy the full SHA
    75eb8c6 View commit details
    Browse the repository at this point in the history

  37. make tests pass

    jbloom-md committed May 6, 2024
    Configuration menu
    Copy the full SHA
    e46d1d6 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. add updated tutorial

    jbloom-md committed May 7, 2024
    Configuration menu
    Copy the full SHA
    8272475 View commit details
    Browse the repository at this point in the history

  2. format

    jbloom-md committed May 7, 2024
    Configuration menu
    Copy the full SHA
    eb35e67 View commit details
    Browse the repository at this point in the history