[FAQ] Using Global Temporary Views in Spark

[kkh1902]

Course

machine-learning-zoomcamp

Question

How do you share a DataFrame across multiple Spark sessions?

Answer

Spark provides Global Temporary Views to share DataFrames across different Spark sessions within the same Spark application.
Unlike regular temporary views, global temporary views are accessible from any session.

Step 1: Create a Global Temporary View

# Create a global temporary view
df.createOrReplaceGlobalTempView('trips_global')

This registers the DataFrame as a global temporary view named trips_global.

Step 2: Query the Global View from Any Session

spark.sql("SELECT * FROM global_temp.trips_global").show()

Global temporary views are stored in the global_temp database and must be referenced using the global_temp. prefix.

Checklist

• I have searched existing FAQs and this question is not already answered
• The answer provides accurate, helpful information
• I have included any relevant code examples or links

[github-actions]

✅ FAQ NEW proposal created in PR #99

Please review and approve the changes.