Skip to content

Commit 8f14c72

Browse files
authored
add_intellij_debug_statements (#1760)
1 parent 7d9e3b2 commit 8f14c72

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

docs/source/contributor-guide/img.png

190 KB
Loading

docs/source/contributor-guide/spark-sql-tests.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ ENABLE_COMET=true build/sbt "hive/testOnly * -- -l org.apache.spark.tags.Extende
6565
ENABLE_COMET=true build/sbt "hive/testOnly * -- -n org.apache.spark.tags.ExtendedHiveTest"
6666
ENABLE_COMET=true build/sbt "hive/testOnly * -- -n org.apache.spark.tags.SlowHiveTest"
6767
```
68-
#### Steps to run individual test suites
68+
#### Steps to run individual test suites through SBT
6969
1. Open SBT with Comet enabled
7070
```sbt
7171
ENABLE_COMET=true sbt -Dspark.test.includeSlowTests=true
@@ -74,6 +74,14 @@ ENABLE_COMET=true sbt -Dspark.test.includeSlowTests=true
7474
```sbt
7575
sql/testOnly org.apache.spark.sql.DynamicPartitionPruningV1SuiteAEOn -- -z "SPARK-35568"
7676
```
77+
#### Steps to run individual test suites in IntelliJ IDE
78+
1. Add below configuration in VM Options for your test case (apache-spark repository)
79+
```sbt
80+
-Dspark.comet.enabled=true -Dspark.comet.debug.enabled=true -Dspark.plugins=org.apache.spark.CometPlugin -DXmx4096m -Dspark.executor.heartbeatInterval=20000 -Dspark.network.timeout=10000 --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED
81+
```
82+
2. Set `ENABLE_COMET=true` in environment variables
83+
![img.png](img.png)
84+
3. After the above tests are configured, spark tests can be run with debugging enabled on spark/comet code. Note that Comet is added as a dependency and the classes are readonly while debugging from Spark. Any new changes to Comet are to be built and deployed locally through the command (`PROFILES="-Pspark-3.4" make release`)
7785

7886
## Creating a diff file for a new Spark version
7987

0 commit comments

Comments
 (0)