Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interpreter support for quantized type #2388

Merged
merged 8 commits into from
Jun 20, 2024

Conversation

sdasgup3
Copy link
Member

@sdasgup3 sdasgup3 commented Jun 8, 2024

fixes #2373

The PR is rebased on top on #2383 and cherry-pick changes from #2384.

Direction to reviewer

Please review the commit 4d7dc1a excluding the following files

  • docs/generated/stablehlo_passes.md
  • stablehlo/transforms/Passes.td
  • stablehlo/transforms/ShapeLegalizeToStablehlo.cpp

Copy link
Contributor

@mlevesquedion mlevesquedion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this approach is fine, but I thought the point of the reference interpreter was to show how the various operations work. Would it be much more onerous to implement support for uniform_quantize and uniform_dequantize directly?

stablehlo/reference/Api.cpp Show resolved Hide resolved
stablehlo/reference/Api.cpp Outdated Show resolved Hide resolved
stablehlo/reference/Api.cpp Outdated Show resolved Hide resolved
stablehlo/reference/Api.cpp Outdated Show resolved Hide resolved
@sdasgup3 sdasgup3 force-pushed the pass-pileline-for-quant-eval branch from 4d7dc1a to bd607c0 Compare June 10, 2024 23:22
stablehlo/reference/Api.cpp Outdated Show resolved Hide resolved
@sdasgup3 sdasgup3 force-pushed the pass-pileline-for-quant-eval branch 2 times, most recently from 94f16aa to 509f4a7 Compare June 11, 2024 00:26
@sdasgup3
Copy link
Member Author

I think this approach is fine, but I thought the point of the reference interpreter was to show how the various operations work. Would it be much more onerous to implement support for uniform_quantize and uniform_dequantize directly?

That is a very good point!
Even if we support uniform_{de}quantize operations we still need support for other quantized operations, like add with quantized type, in the interpreter.

The idea here is to handle all the quantized operation uniformly. That means, either

(A) Support the execution semantics of all the quantized operations (uniform_quantize, uniform_dequantize, and any other operations which support quantized types) natively in the interpreter, OR
(B) Apply the pass uniformly to lower any quantized operations.

(B) is the fastest path towards evaluating quantize program.

@sdasgup3 sdasgup3 force-pushed the pass-pileline-for-quant-eval branch from 509f4a7 to 5eea628 Compare June 18, 2024 23:56
@sdasgup3 sdasgup3 force-pushed the pass-pileline-for-quant-eval branch from 3ca19aa to cdf55ac Compare June 20, 2024 18:38
@sdasgup3 sdasgup3 merged commit 6802016 into openxla:main Jun 20, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Interpreter support for quantized type
3 participants