Skip to content

Commit

Permalink
[SPARK-47373][SQL] Match FileSourceScanLike to get metadata instead o…
Browse files Browse the repository at this point in the history
…f FileSourceScanExec

### What changes were proposed in this pull request?

When get Spark Plan info, we should match basic trait `FileSourceScanLike` to get metadata instead of matching
subclass `FileSourceScanExec`.

So that user-define file scan operators(which extend `FileSourceScanLike`) can be matched.

### Why are the changes needed?

Match user-define file scan operators.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Exists Unit Test

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#45491 from zwangsheng/SPARK-47373.

Authored-by: zwangsheng <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
zwangsheng authored and dongjoon-hyun committed Mar 13, 2024
1 parent a5d0c37 commit 7d0f083
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ private[execution] object SparkPlanInfo {

// dump the file scan metadata (e.g file path) to event log
val metadata = plan match {
case fileScan: FileSourceScanExec => fileScan.metadata
case fileScan: FileSourceScanLike => fileScan.metadata
case _ => Map[String, String]()
}
new SparkPlanInfo(
Expand Down

0 comments on commit 7d0f083

Please sign in to comment.