Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

I generated the base features table with the mock_data specified in FeaturesTable class in utils.py, the table looks like below:

+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|account_id| contact_id|          timestamp|f_contact_click_count_over_30_days|f_contact_click_count_over_60_days|f_contact_open_count_over_30_days|f_contact_open_count_over_60_days|
+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|       100|second_user|2024-12-03 08:00:00|                                 0|                                 5|                               10|                               15|
|       101| first_user|2024-12-07 10:00:00|                                20|                                40|                               20|                               40|
|       100| first_user|2024-12-01 06:00:00|                                20|                                40|                               20|                               40|
|       102| third_user|2024-12-05 07:00:00|                               100|                               100|                              100|                              100|
+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+

I then performed a "Merge"/"Upsert" where we "update all cols" unless all columns (except timestamp) of the new potential entry matches with an entry in the existing table, then we only update the timestamp column.

Here's the table after updating the table with the batch_data specified in FeaturesTable class in utils.py

+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|account_id| contact_id|          timestamp|f_contact_click_count_over_30_days|f_contact_click_count_over_60_days|f_contact_open_count_over_30_days|f_contact_open_count_over_60_days|
+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|       100|second_user|2024-12-03 08:00:00|                                 0|                                 5|                               10|                               15|
|       101| first_user|2024-12-09 10:00:00|                                20|                                40|                               20|                               40| #Timestamp got updated as expected
|       100| first_user|2024-12-01 06:00:00|                                20|                                40|                               20|                               40|
|       102| third_user|2024-12-05 07:00:00|                               100|                               100|                              100|                              100|
|       100| first_user|2024-12-02 06:00:00|                                40|                                80|                               40|                               80|
|       100|fourth_user|2024-12-11 09:00:00|                               200|                               400|                              200|                              400|
+----------+-----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+

Now it's time to do the point in time joins (PIT Joins) where our left table is the observations tables and for each observation we bring the most recent feature computed:

+----------+----------+-------------------+----------+----------+----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|account_id|contact_id|          timestamp|event_type|account_id|contact_id|          timestamp|f_contact_click_count_over_30_days|f_contact_click_count_over_60_days|f_contact_open_count_over_30_days|f_contact_open_count_over_60_days|
+----------+----------+-------------------+----------+----------+----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+
|       100|first_user|2024-12-01 06:01:00|     click|       100|first_user|2024-12-01 06:00:00|                                20|                                40|                               20|                               40|
|       100|first_user|2024-12-02 12:00:00|     click|       100|first_user|2024-12-02 06:00:00|                                40|                                80|                               40|                               80|
|       101|first_user|2024-12-07 11:00:00|      open|      NULL|      NULL|               NULL|                              NULL|                              NULL|                             NULL|                             NULL|
|       102|third_user|2024-12-04 00:00:00|      open|      NULL|      NULL|               NULL|                              NULL|                              NULL|                             NULL|                             NULL|
+----------+----------+-------------------+----------+----------+----------+-------------------+----------------------------------+----------------------------------+---------------------------------+---------------------------------+

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages