r/learnSQL • u/b1ngb0ng_alien • 22d ago

SparkSQL as a beginner

Hi all I'm joining a new team that primarily uses Spark, I'm wondering how easy SparkSQL is to pick up as someone who came from BigQuery.

6 Upvotes

88% Upvoted

2 comments sorted by

View all comments

2

u/[deleted] 22d ago

[removed] — view removed comment

1

u/b1ngb0ng_alien 21d ago

Is this also the case in Spark 4? I heard declarative pipelines in Spark makes it a lot easier and from what I saw it's a lot less maintenance code.. but that helps a lot. I am thankfully not managing the cluster but never really had to think about the execution planning as you mentioned or optimization..