Apache Spark Sandbox

Apache Spark Sandbox with patterns often focusing on Microsoft Fabric

· Devcontainer Overview · Spark Devcontainer Source ·

Context

This repository contains POCs/POTs in Apache Spark.

Quickstart

1. Follow CONTRIBUTING to get a windows machine ready to start the devcontainer.

2. Install the Dev Containers plugin in your WSL VSCode (ms-vscode-remote.remote-containers):

   

3. Inside WSL, start the devcontainer with Ctrl + Shift + P > Dev Containers: Rebuild and Reopen in Container

4. Dig into:

   • spark-scala to run local Scala demonstrations.
   • spark-dbt to run local dbt demonstrations.

Repo Architecture