Skip to content

Commit

Permalink
Update README.md (#1036)
Browse files Browse the repository at this point in the history
  • Loading branch information
jerryzh168 authored Oct 9, 2024
1 parent b67407a commit 900f9ac
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion torchao/prototype/awq/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ Evaluation perplexity numbers were calculated using the script in awq/example.py


The following tests were performed using LM eval and groupsize = 128
| Model | Quantization | Perplexity | Truthful QA MC2 | WinoGrande | ARC challenge |

| Model | Quantization | Perplexity | Truthful QA MC2 | WinoGrande | ARC challenge |
|--------------------|--------------|------------|-----------------|------------|---------------|
| Llama-3-8B-Instruct| bfloat16 | 10.936 | 0.540 | 0.783 | 0.567 |
| | awq-hqq-int4 | 11.383 | 0.522 | 0.772 | 0.543 |
| | awq-uint4 | 11.409 | 0.519 | 0.756 | 0.577 |
Expand Down

0 comments on commit 900f9ac

Please sign in to comment.