Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-49200][SQL] Fix null type non-codegen ordering exception
### What changes were proposed in this pull request? Spark mark `NullType` as orderable, and we return 0 when gen comparing code for `NullType`. ``` object OrderUtils { def isOrderable(dataType: DataType): Boolean = dataType match { case NullType => true ``` This pr makes `NullType` ordering work for non-codegen path to avoid exception. ### Why are the changes needed? Fix exception: ```sql set spark.sql.codegen.factoryMode=NO_CODEGEN; set spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.EliminateSorts; select * from range(10) order by array(null); ``` ``` org.apache.spark.SparkIllegalArgumentException: Type PhysicalNullType does not support ordered operations. at org.apache.spark.sql.errors.QueryExecutionErrors$.orderedOperationUnsupportedByDataTypeError(QueryExecutionErrors.scala:352) at org.apache.spark.sql.catalyst.types.PhysicalNullType.ordering(PhysicalDataType.scala:246) at org.apache.spark.sql.catalyst.types.PhysicalNullType.ordering(PhysicalDataType.scala:243) at org.apache.spark.sql.catalyst.types.PhysicalArrayType$$anon$1.<init>(PhysicalDataType.scala:283) at org.apache.spark.sql.catalyst.types.PhysicalArrayType.interpretedOrdering$lzycompute(PhysicalDataType.scala:281) at org.apache.spark.sql.catalyst.types.PhysicalArrayType.interpretedOrdering(PhysicalDataType.scala:281) at org.apache.spark.sql.catalyst.types.PhysicalArrayType.ordering(PhysicalDataType.scala:277) at org.apache.spark.sql.catalyst.expressions.InterpretedOrdering.compare(ordering.scala:67) at org.apache.spark.sql.catalyst.expressions.InterpretedOrdering.compare(ordering.scala:40) at org.apache.spark.sql.execution.UnsafeExternalRowSorter$RowComparator.compare(UnsafeExternalRowSorter.java:254) at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorter$SortComparator.compare(UnsafeInMemorySorter.java:70) at org.apache.spark.util.collection.unsafe.sort.UnsafeInMemorySorter$SortComparator.compare(UnsafeInMemorySorter.java:44) ``` ### Does this PR introduce _any_ user-facing change? yes, bug fix ### How was this patch tested? add test ### Was this patch authored or co-authored using generative AI tooling? no Closes apache#47707 from ulysses-you/null-ordering. Authored-by: ulysses-you <[email protected]> Signed-off-by: youxiduo <[email protected]>
- Loading branch information