Skip to content

Issues: pytorch/torchtitan

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

57 Open 97 Closed
57 Open 97 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

HSDP IB Issues
#633 opened Oct 18, 2024 by karan-dalal
add H100 in CI better_engineering Repo code quality improvements integration test Adding integration tests
#632 opened Oct 18, 2024 by tianyu-l torchtitan release 1.0
create a note on torchtitan official release documentation Improvements or additions to documentation release_blocking Issues that are blocking the milestone / release completion
#631 opened Oct 18, 2024 by tianyu-l torchtitan release 1.0
Non-DP runs default to float32 precision enhancement New feature or request
#630 opened Oct 18, 2024 by carmocca
1
add Llama 3.2 support
#625 opened Oct 18, 2024 by tianyu-l torchtitan release 1.0
@fduwjj
[Triton] Implement Liger Kernels enhancement New feature or request
#623 opened Oct 17, 2024 by casper-hansen
Is there way to offload training memory to DRAM (using FSDP2?) for training Llama3-8B with torchtitan? question Further information is requested
#620 opened Oct 15, 2024 by 0781532
1
9
Question about torch.compile has better throughput with 128-GPUs than 8-GPUs question Further information is requested
#619 opened Oct 15, 2024 by dz1iang
@tianyu-l
3
redundant checks in checkpoint.py better_engineering Repo code quality improvements good first issue Good for newcomers
#617 opened Oct 14, 2024 by eminorhan torchtitan release 1.0
Ability to train based on epoch enhancement New feature or request good first issue Good for newcomers
#613 opened Oct 13, 2024 by abatilo
1
[Compile] Understand why FSDP2 saves both SDPA out and wo in for bwd question Further information is requested
#610 opened Oct 11, 2024 by awgu
why is xformers not used for attention computation? question Further information is requested
#608 opened Oct 9, 2024 by jason718
4
Granular layer selection during Pipeline Parallelism question Further information is requested
#598 opened Oct 3, 2024 by bhuvan777
2
Gradient norm clipping with pipeline parallelism (PP) bug Something isn't working release_blocking Issues that are blocking the milestone / release completion
#596 opened Oct 1, 2024 by zijian-hu torchtitan release 1.0
5
Support Gemma2 in torchtitan enhancement New feature or request
#594 opened Oct 1, 2024 by pansershrek
3
reproducable numerics for loss, weights and gradients for single node (8 GPUs) enhancement New feature or request
#593 opened Oct 1, 2024 by weifengpy
1
2
Inference with the checkpoint enhancement New feature or request
#586 opened Sep 23, 2024 by mathmax12
1
Support INT8 mixed-precision training from torchao? enhancement New feature or request
#578 opened Sep 14, 2024 by gau-nernst
8
Wrong train_state.step when resuming from checkpoint for the second time bug Something isn't working
#571 opened Sep 8, 2024 by LeoXinhaoLee
@wz337
1
Pipeline Parallelism + FSDP question Further information is requested
#562 opened Aug 29, 2024 by jeromeku
1
Fail-safe and partial redundancy for HSDP on unreliable compute enhancement New feature or request
#561 opened Aug 27, 2024 by evkogs
@wconstab
5
Adjust MFU to account for FP8
#560 opened Aug 23, 2024 by lessw2020
1
PP UX/training confusion re: loss = -1. (need to better document or add auto logging of last rank loss?)
#550 opened Aug 21, 2024 by lessw2020
2D whole model compile fails at embedding layer bug Something isn't working
#534 opened Aug 20, 2024 by tianyu-l
@bdhirsh @tianyu-l
2
[rfc] getting rid of seed-checkpoint for Pipeline Parallelism enhancement New feature or request release_blocking Issues that are blocking the milestone / release completion
#514 opened Aug 10, 2024 by wconstab torchtitan release 1.0
4
ProTip! Mix and match filters to narrow down what you’re looking for.