feat: Set/cancel with job tag and make max broadcast table size configurable

[wForget]

Which issue does this PR close?

Closes #1692, #1691.

Rationale for this change

Improve CometBroadcastExchangeExec:

• Make max broadcast table size configurable
• Set/cancel with job tag on spark 3.5+

What changes are included in this PR?

How are these changes tested?

added maxBroadcastTableSize test

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 58.72%. Comparing base (f09f8af) to head (b1f6ff7).
Report is 173 commits behind head on main.

Files with missing lines	Patch %	Lines
...e/spark/sql/comet/CometBroadcastExchangeExec.scala	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1693      +/-   ##
============================================
+ Coverage     56.12%   58.72%   +2.59%     
- Complexity      976     1083     +107     
============================================
  Files           119      126       +7     
  Lines         11743    12565     +822     
  Branches       2251     2361     +110     
============================================
+ Hits           6591     7379     +788     
- Misses         4012     4014       +2     
- Partials       1140     1172      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[parthchandra]

This conf is only available since Spark 4.1, right?

[wForget]

This conf is only available since Spark 4.1, right?

Yes, but we can port it to any version. For the native engine, we may configure a smaller heap memory, which makes broadcast more likely to cause OOM. I think we can use this configuration to limit the maximum bytes of broadcast.

[parthchandra]

I don't see it being back-ported to the 3.x branch, but as long as this builds against older releases and doesn't throw an exception that the conf could not be found, we are ok.