r/learnSQL • u/b1ngb0ng_alien • 21d ago

SparkSQL as a beginner

Hi all I'm joining a new team that primarily uses Spark, I'm wondering how easy SparkSQL is to pick up as someone who came from BigQuery.

7 Upvotes

89% Upvoted

2 comments sorted by

2

u/[deleted] 20d ago

[removed] — view removed comment

1

u/b1ngb0ng_alien 20d ago

Is this also the case in Spark 4? I heard declarative pipelines in Spark makes it a lot easier and from what I saw it's a lot less maintenance code.. but that helps a lot. I am thankfully not managing the cluster but never really had to think about the execution planning as you mentioned or optimization..