feat(doris): add catalog support for Apache Doris

[liujiwen-up]

SUMMARY

add catalog for apache doris
In Apache Doris, in order to be compatible with different BI tools, we have information_schema for each catalog. When no catalog is specified, the default catalog is internal. This feature corresponds to this PR of Doris, apache/doris#28919

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[korbit-ai]

I've completed my review and didn't find any issues... but I did find this chicken.

  \\
   (o>
  <_ )
   ^^

Files scanned

​

File Path	Reviewed
superset/db_engine_specs/doris.py	✅

Explore our documentation to understand the languages and file types we support and the files we ignore.

Need a new review? Comment /korbit-review on this PR and I'll review your latest changes.

Korbit Guide: Usage and Customization

Interacting with Korbit

• You can manually ask Korbit to review your PR using the /korbit-review command in a comment at the root of your PR.
• You can ask Korbit to generate a new PR description using the /korbit-generate-pr-description command in any comment on your PR.
• Too many Korbit comments? I can resolve all my comment threads if you use the /korbit-resolve command in any comment on your PR.
• Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
• Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.

Customizing Korbit

• Check out our docs on how you can make Korbit work best for you and your team.
• Customize Korbit for your organization through the Korbit Console.

Current Korbit Configuration

General Settings

​

Setting	Value
Review Schedule	Automatic excluding drafts
Max Issue Count	10
Automatic PR Descriptions	❌

Issue Categories

​

Category	Enabled
Naming	✅
Database Operations	✅
Documentation	✅
Logging	✅
Error Handling	✅
Systems and Environment	✅
Objects and Data Structures	✅
Readability and Maintainability	✅
Asynchronous Processing	✅
Design Patterns	✅
Third-Party Libraries	✅
Performance	✅
Security	✅
Functionality	✅

Feedback and Support

• Tell us what you think of Korbit
• Schedule a call with our team
• Email us @ [email protected]

Note

Korbit Pro is free for open source projects 🎉

Looking to add Korbit to your team? Get started with a free 2 week trial here

[codecov]

Codecov Report

Attention: Patch coverage is 90.47619% with 2 lines in your changes missing coverage. Please review.

Project coverage is 83.77%. Comparing base (76d897e) to head (05957e4).
Report is 1240 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/db_engine_specs/doris.py	90.47%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #31580       +/-   ##
===========================================
+ Coverage   60.48%   83.77%   +23.28%     
===========================================
  Files        1931      538     -1393     
  Lines       76236    39142    -37094     
  Branches     8568        0     -8568     
===========================================
- Hits        46114    32790    -13324     
+ Misses      28017     6352    -21665     
+ Partials     2105        0     -2105     

Flag	Coverage Δ
hive	48.75% <33.33%> (-0.41%)	⬇️
javascript	?
mysql	76.47% <33.33%> (?)
postgres	76.56% <33.33%> (?)
presto	53.27% <33.33%> (-0.54%)	⬇️
python	83.77% <90.47%> (+20.28%)	⬆️
sqlite	76.02% <33.33%> (?)
unit	60.90% <90.47%> (+3.27%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[michael-s-molina]

@mistercrunch Any idea about the docker image error?

[rusackas]

@betodealmeida is definitely the subject matter expert on catalog support lately!

[villebro]

This PR is missing a proper description, making it difficult to understand key details in the featured changes. Also, a few general comments that caught my eye while reviewing.

[villebro]

Thanks for the improvements @liujiwen-up! One last comment, after that I feel this is good to go 👍

[villebro]

This may sound like a total nit, but I actually had some issues following what's going on here, especially the catalog or current_catalog or "" logic. As current_catalog is unnecessary if catalog is defined, I would have maybe just reused the latter variable for all these uses. Something like:

        if catalog:
            pass
        elif uri.database and "." in uri.database:
            catalog, _ = uri.database.split(".", 1) or ""  # notice how I also moved the `or ""` part here
        else:
            catalog = "internal"

Then later just

        adjusted_database = ".".join([catalog, "information_schema"])

Also, why is .rstrip(".") needed? I don't see how we can ever hit that, as adjusted_database will always end with .information_schema, right?

[liujiwen-up]

@villebro Thanks for your advice. After in-depth testing with Doris, we found that there is still a problem. The previous test only tested the case of linking data sources. When operating on SQL Lab, it will also go to this function and cannot use the information_schema library fixedly. When there is a schema value, the user-provided schema should be used for querying. This implementation is the correct behavior at present.

1. When linking data sources, the schema is empty and the information_schema library is used uniformly
2. When the schema has a value, the schema value provided by the user is used

[liujiwen-up]

@villebro Please help me push it forward. Thank you.
In addition, I would like to ask, after the PR is merged, in which superset release version will it be released? I need to update the official documentation of apache doris based on this information for reference by superset users.