Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] update benchmark page #1234

Merged
merged 14 commits into from
Sep 8, 2023
Merged

Conversation

felixdittrich92
Copy link
Contributor

@felixdittrich92 felixdittrich92 commented Jun 29, 2023

This PR:

  • updates models/benchmark page
  • add benchmarks for new available models
  • set aspect ratio to False to avoid eval script changes atm
  • update onnx also in tf build which was already added for the dev build

Any feedback is welcome 🤗

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #1234 (2d7ed4d) into main (ccfc3e0) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1234   +/-   ##
=======================================
  Coverage   95.76%   95.76%           
=======================================
  Files         154      154           
  Lines        6902     6902           
=======================================
  Hits         6610     6610           
  Misses        292      292           
Flag Coverage Δ
unittests 95.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1 file with indirect coverage changes

📢 Have feedback on the report? Share it here.

@felixdittrich92 felixdittrich92 force-pushed the doc-updates branch 2 times, most recently from ebc3ab8 to 1e05e8c Compare July 11, 2023 12:27
@felixdittrich92 felixdittrich92 changed the title DRAFT: [docs] update docs export & benchmark pages DRAFT: [docs] update benchmark page Jul 24, 2023
@felixdittrich92 felixdittrich92 added this to the 0.7.0 milestone Jul 28, 2023
@felixdittrich92 felixdittrich92 self-assigned this Jul 28, 2023
@felixdittrich92 felixdittrich92 added topic: documentation Improvements or additions to documentation benchmark This issue reports a benchmark type: misc Miscellaneous ext: docs Related to docs folder labels Jul 28, 2023
@ArsalanYounus007
Copy link

Can you please add MS OCR to the comparison table as well?

@felixT2K
Copy link
Contributor

Can you please add MS OCR to the comparison table as well?

Hey 👋🏼 ,
you mean the Azure Vision API (OCR) ?
I will check this 👍🏼

@felixdittrich92
Copy link
Contributor Author

Can you please add MS OCR to the comparison table as well?

Added benchmark for Azure form recognizer (v3.2). I have tested the standard OCR API also but this performs really poor on document types (because it's made for scene text in the wild) so i decided to add only the form recognizer benchmark to be fair :)

@ArsalanYounus007
Copy link

Thank you for considering MS Azure Read API.
I am a little confused about "this performs really poor on document types" because we use this for many projects and it performs very well on scanned document images.

Just to confirm, You tried using Read API ?? I have shared the URL that we use.
https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-read?view=doc-intel-3.1.0

@felixdittrich92
Copy link
Contributor Author

Thank you for considering MS Azure Read API. I am a little confused about "this performs really poor on document types" because we use this for many projects and it performs very well on scanned document images.

Just to confirm, You tried using Read API ?? I have shared the URL that we use. https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-read?view=doc-intel-3.1.0

The added benchmark is from document intelligence and before i have tested the vision api ocr endpoint

@felixdittrich92 felixdittrich92 changed the title DRAFT: [docs] update benchmark page [docs] update benchmark page Sep 8, 2023
@felixdittrich92 felixdittrich92 marked this pull request as ready for review September 8, 2023 14:08
Copy link
Collaborator

@odulcy-mindee odulcy-mindee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @felixdittrich92, great work here

@felixdittrich92 felixdittrich92 merged commit 75bddfc into mindee:main Sep 8, 2023
58 checks passed
@felixdittrich92 felixdittrich92 deleted the doc-updates branch September 8, 2023 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmark This issue reports a benchmark ext: docs Related to docs folder topic: documentation Improvements or additions to documentation type: misc Miscellaneous
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants