Skip to content

[VL] No reason for native validation failed with round function #9542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dcoliversun opened this issue May 7, 2025 · 1 comment · May be fixed by #9560 or #9576
Open

[VL] No reason for native validation failed with round function #9542

dcoliversun opened this issue May 7, 2025 · 1 comment · May be fixed by #9560 or #9576
Labels
bug Something isn't working triage

Comments

@dcoliversun
Copy link
Contributor

dcoliversun commented May 7, 2025

Backend

VL (Velox)

Bug description

Two problems:

  • Velox support round but Gluten couldn't push into native
  • No reason for fallback

Test case:

test("round") {
    withTempPath {
      path =>
        Seq[Integer](
          25,
          36
        )
          .toDF("value")
          .write
          .parquet(path.getCanonicalPath)

        spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("view")

        runQueryAndCompare("SELECT round(value, -1) from view") {
          checkGlutenOperatorMatch[ProjectExecTransformer]
        }
    }
  }

The log is as follow:

16:05:46.980 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

W20250507 16:05:56.172621 1131222 MemoryArbitrator.cpp:84] Query memory capacity[460.50MB] is set for NOOP arbitrator which has no capacity enforcement
E20250507 16:06:37.676007 1131695 Exceptions.h:66] Line: /opt/gluten/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2068, Function:terminate, Expression:  Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE

E20250507 16:06:46.796408 1131695 Exceptions.h:66] Line: /opt/gluten/ep/build-velox/build/velox_ep/velox/exec/Task.cpp:2068, Function:terminate, Expression:  Cancelled, Source: RUNTIME, ErrorCode: INVALID_STATE
16:06:53.176 WARN org.apache.spark.sql.execution.GlutenFallbackReporter: Validation failed for plan: Project, due to: 
 - Native validation failed: 
   |- 

16:06:54.658 WARN org.apache.spark.sql.execution.GlutenFallbackReporter: Validation failed for plan: Project[QueryId=13], due to: 
 - Native validation failed: 
   |- 



executedPlan.exists(((plan: org.apache.spark.sql.execution.SparkPlan) => tag.runtimeClass.isInstance(plan))) was false Expect ProjectExecTransformer exists in executedPlan:
 Project [round(value#144, -1) AS round(value, -1)#151]
+- VeloxColumnarToRow
   +- ^(3) BatchScanTransformer parquet file:/tmp/spark-c676d58b-b13a-43f2-82e6-cbdb3abcf21b[value#144] ParquetScan DataFilters: [], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/spark-c676d58b-b13a-43f2-82e6-cbdb3abcf21b], PartitionFilters: [], PushedAggregation: [], PushedFilters: [], PushedGroupBy: [], ReadSchema: struct<value:int> RuntimeFilters: [] NativeFilters: []

ScalaTestFailureLocation: org.apache.spark.sql.GlutenQueryTest at (GlutenQueryTest.scala:421)
org.scalatest.exceptions.TestFailedException: executedPlan.exists(((plan: org.apache.spark.sql.execution.SparkPlan) => tag.runtimeClass.isInstance(plan))) was false Expect ProjectExecTransformer exists in executedPlan:
 Project [round(value#144, -1) AS round(value, -1)#151]
+- VeloxColumnarToRow
   +- ^(3) BatchScanTransformer parquet file:/tmp/spark-c676d58b-b13a-43f2-82e6-cbdb3abcf21b[value#144] ParquetScan DataFilters: [], Format: parquet, Location: InMemoryFileIndex(1 paths)[file:/tmp/spark-c676d58b-b13a-43f2-82e6-cbdb3abcf21b], PartitionFilters: [], PushedAggregation: [], PushedFilters: [], PushedGroupBy: [], ReadSchema: struct<value:int> RuntimeFilters: [] NativeFilters: []

Gluten version

Gluten-1.4

Spark version

Spark-3.5.x

Spark configurations

No response

System information

No response

Relevant logs

@dcoliversun dcoliversun added bug Something isn't working triage labels May 7, 2025
@jinchengchenghh
Copy link
Contributor

The scale in velox is uint8_t, so it does not support the negative scale, but I don't know why the reason cannot be returned

@FelixYBW FelixYBW changed the title No reason for native validation failed with round function [VL] No reason for native validation failed with round function May 9, 2025
@lgbo-ustc lgbo-ustc linked a pull request May 9, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
2 participants