Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more information to quantized linear module and added some logs #782

Merged
merged 5 commits into from
Sep 4, 2024

Conversation

jerryzh168
Copy link
Contributor

@jerryzh168 jerryzh168 commented Aug 30, 2024

Summary:
Fixes #771

Test Plan:
python test/dtypes/test_affine_quantized_tensor.py -k test_print_quantized_module

Example output:

Linear(in_features=128, out_features=256, weight=AffineQuantizedTensor(shape=torch.Size([256, 128]), block_size=(1, 128), device=cuda:0, layout_type=PlainLayoutType(), layout_tensor_dtype=torch.int8, quant_min=None, quant_max=None))
.Linear(in_features=128, out_features=256, weight=LinearActivationQuantizedTensor(activation=<function _int8_asymm_per_token_quant at 0x7feb1d146820>, weight=AffineQuantizedTensor(shape=torch.Size([256, 128]), block_size=(1, 32), device=cuda:0, layout_type=PlainLayoutType(), layout_tensor_dtype=torch.int8, quant_min=-8, quant_max=7)))
.Linear(in_features=128, out_features=256, weight=LinearActivationQuantizedTensor(activation=<function _int8_symm_per_token_reduced_range_quant at 0x7feb1d146af0>, weight=AffineQuantizedTensor(shape=torch.Size([256, 128]), block_size=(1, 128), device=cuda:0, layout_type=PlainLayoutType(), layout_tensor_dtype=torch.int8, quant_min=None, quant_max=None)))
.Linear(in_features=128, out_features=256, weight=AffineQuantizedTensor(shape=torch.Size([256, 128]), block_size=(1, 32), device=cuda:0, layout_type=TensorCoreTiledLayoutType(inner_k_tiles=8), layout_tensor_dtype=torch.int32, quant_min=0, quant_max=15))
.Linear(in_features=128, out_features=256, weight=LinearActivationQuantizedTensor(activation=<function _int8_symm_per_token_reduced_range_quant at 0x7feb1d146af0>, weight=AffineQuantizedTensor(shape=torch.Size([256, 128]), block_size=(1, 128), device=cuda:0, layout_type=SemiSparseLayoutType(), layout_tensor_dtype=torch.int8, quant_min=None, quant_max=None)))

Reviewers:

Subscribers:

Tasks:

Tags:

Copy link

pytorch-bot bot commented Aug 30, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/782

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit f7e0bbd with merge base 6080986 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 30, 2024
def insert_subclass(lin):
lin.weight = torch.nn.Parameter(constructor(lin.weight), requires_grad=False)
lin.weight = torch.nn.Parameter(constructor(lin.weight, **kwargs), requires_grad=False)
lin.extra_repr = types.MethodType(_linear_extra_repr, lin)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: is there anything we can do to preserve any preexisting custom repr?

Copy link
Contributor Author

@jerryzh168 jerryzh168 Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually there will be a recursive call here, since we are overriding extra_repr()

Copy link
Contributor

@vkuzo vkuzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, thanks for working on this!

Summary:
Fixes pytorch#771

Test Plan:
python test/dtypes/test_affine_quantized_tensor.py -k test_print_quantized_module

Reviewers:

Subscribers:

Tasks:

Tags:
@jerryzh168 jerryzh168 merged commit 0987dd6 into pytorch:main Sep 4, 2024
17 checks passed
@jerryzh168 jerryzh168 deleted the print-quantized-module branch September 4, 2024 00:26
jerryzh168 added a commit to jerryzh168/ao that referenced this pull request Sep 4, 2024
…ytorch#782)

* Add more information to quantized linear module and added some logs

Summary:
Fixes pytorch#771

Test Plan:
python test/dtypes/test_affine_quantized_tensor.py -k test_print_quantized_module

Reviewers:

Subscribers:

Tasks:

Tags:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

printing a quantized model is not informative
3 participants