Adapt remaining SQL plugin callers to Java helpers#15034
Conversation
b7a2605 to
f6e6ac4
Compare
e895b00 to
f061f44
Compare
f6e6ac4 to
34804ee
Compare
f061f44 to
6f4f246
Compare
19f0ac5 to
bd7c6fb
Compare
6f4f246 to
5f9363a
Compare
bd7c6fb to
86e001e
Compare
1457c07 to
64a238d
Compare
505e2e5 to
7cc1e13
Compare
1f90438 to
6011c26
Compare
7cc1e13 to
0ae48f8
Compare
6011c26 to
fea9cb1
Compare
0ae48f8 to
4356413
Compare
fea9cb1 to
7160eb5
Compare
179e067 to
81b157f
Compare
7160eb5 to
3c8769b
Compare
31fb4c7 to
a9e05a6
Compare
7bbdcc0 to
d35c3a9
Compare
a9e05a6 to
2db5bfe
Compare
d35c3a9 to
cf518aa
Compare
2db5bfe to
084c315
Compare
cf518aa to
f714012
Compare
7440c9d to
32c9b07
Compare
63a0477 to
4f0fb18
Compare
3987773 to
0202d1d
Compare
e69ce28 to
bc1ad84
Compare
cd6b7b9 to
d158664
Compare
Signed-off-by: Gera Shegalov <gshegalov@nvidia.com>
3848638 to
e505feb
Compare
d158664 to
11f74bf
Compare
Greptile SummaryThis PR is one layer of the unshim stack: it migrates the remaining SQL plugin callers off Spark's private
Confidence Score: 4/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
subgraph Before["Before (Spark private Logging + case class)"]
A["extends Logging\n(Spark internal)"] --> B["case class ProfileInitMsg\ncase class LoreRDDMeta\ncase class AggAndReplace\ncase class BoundGpuWindowFunction\n..."]
B --> C["Pattern match via\nunapply / case class deconstruct"]
end
subgraph After["After (slf4j + Java-friendly class)"]
D["RapidsLocalLog trait\nor inline slf4j.LoggerFactory"] --> E["class ProfileInitMsg (Java)\nclass LoreRDDMeta\nclass AggAndReplace\nclass BoundGpuWindowFunction\n..."]
E --> F["Pattern match via\ncase msg: Foo => msg.field"]
end
subgraph Missing["Forward dependency (HEAD)"]
G["GpuLiteral extends GpuLiteralShim"] -.->|"GpuLiteralShim not yet defined\n(expected in PR #15052)"| H["⚠ Compilation gap"]
end
Before --> After
Reviews (1): Last reviewed commit: "Update Spark 4.1 generated support metad..." | Re-trigger Greptile |
| * In order to do type conversion and checking, use GpuLiteral.create() instead of constructor. | ||
| */ | ||
| case class GpuLiteral (value: Any, dataType: DataType) extends GpuLeafExpression { | ||
| case class GpuLiteral (value: Any, dataType: DataType) extends GpuLiteralShim { |
There was a problem hiding this comment.
Forward reference to undefined
GpuLiteralShim
GpuLiteralShim is imported from com.nvidia.spark.rapids.shims and GpuLiteral now extends it, but a search across the entire repository (all Scala and Java sources, all shim version directories) finds no definition of this trait anywhere in the current HEAD. Building this layer in isolation — or any branch that cherry-picks or rebases onto it before PR #15052 lands — will fail with an unresolved symbol error. If the definition is intentionally deferred to the "common shim-helper cleanup" next layer, a // TODO defined in #15052 comment or a temporary stub would make the dependency explicit.
| val ONE_SECOND_MICROSECONDS = 1000000 | ||
|
|
||
| private def currentTimestampMicros: Long = { | ||
| val instant = Instant.now() | ||
| instant.getEpochSecond * ONE_SECOND_MICROSECONDS + instant.getNano / 1000 |
There was a problem hiding this comment.
Plain arithmetic instead of overflow-safe multiply
Spark's own instantToMicros uses Math.multiplyExact and Math.addExact to guard against overflow. The replacement here uses plain * and +. For timestamps within a practical range the result is identical, but it silently returns a wrong (wrapped) value rather than throwing if a pathological timestamp is ever passed, while the shim it replaces would have thrown. Worth noting since this replaces a shim that was previously calling through to Spark's overflow-checked path.
Related to #14834.
Description
This PR is one reviewable layer in the unshim stack introduced by #15025. It updates the remaining SQL plugin callers to use the moved Java helpers. This closes out the caller migration before the common shim-helper cleanup begins.
Stack context
Testing and validation notes
Checklists
Documentation
Testing
(Covered by the validation notes in the PR description.)
Performance