Skip to content

Commit 68199c2

Browse files
authored
docs: Update supported Spark versions (#1580)
## Which issue does this PR close? Part of #1579 Follows on from #1573
1 parent c4c998c commit 68199c2

File tree

1 file changed

+15
-5
lines changed

1 file changed

+15
-5
lines changed

docs/source/user-guide/installation.md

+15-5
Original file line numberDiff line numberDiff line change
@@ -30,22 +30,32 @@ Make sure the following requirements are met and software installed on your mach
3030

3131
### Supported Spark Versions
3232

33-
Comet currently supports the following versions of Apache Spark:
33+
Comet currently supports the following versions of Apache Spark. Note that this documentation is for the main
34+
branch. Please refer to the version of this documentation in the appropriate GitHub release branch for released
35+
Comet versions.
3436

35-
- 3.4.x (Java 8/11/17, Scala 2.12/2.13)
36-
- 3.5.x (Java 8/11/17, Scala 2.12/2.13)
37+
We recommend only using Comet with Spark versions where we currently have both Comet and Spark tests enabled in CI.
38+
Other versions may work well enough for development and evaluation purposes.
39+
40+
| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
41+
| ------------- | ------------ | ------------- | ----------------- | ----------------- |
42+
| 3.4.3 | 8/11/17 | 2.12/2.13 | Yes | Yes |
43+
| 3.5.4 | 8/11/17 | 2.12/2.13 | No | Yes |
44+
| 3.5.5 | 8/11/17 | 2.12/2.13 | Yes | Yes |
3745

3846
Experimental support is provided for the following versions of Apache Spark and is intended for development/testing
3947
use only and should not be used in production yet.
4048

41-
- 4.0.0-preview1 (Java 17/21, Scala 2.13)
49+
| Spark Version | Java Version | Scala Version | Comet Tests in CI | Spark Tests in CI |
50+
| -------------- | ------------ | ------------- | ----------------- | ----------------- |
51+
| 4.0.0-preview1 | 17 | 2.13 | Yes | Yes |
4252

4353
Note that Comet may not fully work with proprietary forks of Apache Spark such as the Spark versions offered by
4454
Cloud Service Providers.
4555

4656
## Using a Published JAR File
4757

48-
Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it
58+
Comet jar files are available in [Maven Central](https://central.sonatype.com/namespace/org.apache.datafusion) for amd64 and arm64 architectures for Linux. For Apple OSX, it
4959
is currently necessary to build from source.
5060

5161
Here are the direct links for downloading the Comet 0.7.0 jar file.

0 commit comments

Comments
 (0)