Skip to content
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d106545
first bit
spopelka-dsac Nov 17, 2025
48a4860
adding more code
spopelka-dsac Nov 17, 2025
599ba9d
got all the tables
spopelka-dsac Nov 18, 2025
9e52a52
more tweaks
spopelka-dsac Nov 18, 2025
2c54439
more tweaks
spopelka-dsac Nov 18, 2025
e09aa00
now just need to ensure the inserts work
spopelka-dsac Nov 19, 2025
2ad815a
renaming file
spopelka-dsac Nov 19, 2025
a7e7315
more changes
spopelka-dsac Nov 25, 2025
22bb4d1
more tweaks
spopelka-dsac Nov 26, 2025
774b468
removing playwright files
spopelka-dsac Nov 26, 2025
beb1d9e
tweaking credentials
spopelka-dsac Nov 26, 2025
7d845df
sjp/halloween-csvs
spopelka-dsac Nov 26, 2025
5db3c46
commenting provider_to_credential
spopelka-dsac Nov 28, 2025
310dd8e
Adding explanatory content
spopelka-dsac Dec 3, 2025
4499402
clear all outputs
spopelka-dsac Dec 3, 2025
cf05e41
Merge branch 'main' into sjp/halloween-csvs
spopelka-dsac Dec 3, 2025
aa0f676
sjp/halloween-csvs
spopelka-dsac Dec 3, 2025
4af193e
just in case
spopelka-dsac Dec 9, 2025
8012223
renaming V13 migration to V14
spopelka-dsac Dec 10, 2025
240e158
Merge branch 'main' into sjp/halloween-csvs
spopelka-dsac Dec 12, 2025
e1256a4
some tweaks for ids
spopelka-dsac Dec 15, 2025
a5127b0
Delete flyway/docker-compose.yml
spopelka-dsac Jan 29, 2026
e65f8bc
Merge branch 'main' into sjp/halloween-csvs
spopelka-dsac Jan 29, 2026
f77b121
Merge branch 'main' into sjp/halloween-csvs
spopelka-dsac Jan 29, 2026
84bbd8b
removing comments
spopelka-dsac Jan 5, 2026
3907db6
clear outputs
spopelka-dsac Jan 30, 2026
8f5286d
Merge branch 'main' into sjp/halloween-csvs
wbprice Jan 30, 2026
8591c1c
uncommenting one line
spopelka-dsac Jan 30, 2026
23fada6
Merge branch 'main' into sjp/halloween-csvs
spopelka-dsac Jan 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions etls/loadHalloween/dbHelpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import os
from dotenv import load_dotenv
from sqlalchemy import create_engine


def createEngine():
# Get database details and create engine
load_dotenv()
username = os.getenv('SOURCE_DB_USER')
password = os.getenv('SOURCE_DB_PASSWORD')
instance = os.getenv('SOURCE_DB_HOST')
db = os.getenv('SOURCE_DB_NAME')
port = os.getenv('SOURCE_DB_PORT')
engine = create_engine(
f"postgresql+psycopg2://{username}:{password}@{instance}:{port}/{db}")
return engine
Loading
Loading