BYOC_ID: 44535 Refactor A&E

[OluwatobiOni]

Opening a draft PR for refactoring ae. Will be updated when changes required for NSS to make are made.

[Jennit07]

Suggested change

	# denodo_connect_cup,
	# denodo_connect,

[LucyEmma22]

TO-DO:

read_extract_ae

1. Check assumption that arrival_financial_year == financial_year
2. Replace care_reference_number with case_reference_number
3. Remove data type corrections

process_extract_ae

1. Remove ae_cup_file read in from disk and uncomment ae_cup_file read in from denodo
2. Replace placeholder for data path in denodo - dbplyr::in_schema("sdl", "sdl_ae_cup_source_placeholder")
3. Replace placeholder for year - dplyr::filter(year == c_year_cup)

[LucyEmma22]

Comparison of data read in from disk (old) to data read in from Denodo (new):

Old Data: 36 columns; 1571185 rows
New Data: 36 columns; 1618925 rows

Data Types:

         col       new          old
1   keytime1 character hms difftime
2   keytime2 character hms difftime
3     gender character      numeric
4 ae_patflow character      numeric
5        age integer64      numeric
6   commhosp integer64    character

[LucyEmma22]

Might need to revert these changes if we test targets with a new script?

[LucyEmma22]

May need to keep changes for keytime1 and keytime2 if data type is not available in Denodo. Need to check this conversion is accurate.

[Jennit07]

Dependency missing: A&E CUP file

[OluwatobiOni]

ae cup file has now been made available but some UAT to-dos before this PR is ready for testing:

IN THE AE FILE:

• commhosp is currently integer64. Character is expected.
• patient_sex, patient_flow_code, and age_at_midpoint_of_financial_year are cast as integer but these do not affect the output because their values are whole numbers anyway.

IN THE AE CUP FILE:

• cup_marker is currently integer64. Should be cast as numeric
• ed_arrival_time is currently character. Should be cast as TIME (HH:MM:SS)
• ed_discharge_time is currently character. Should be cast as TIME (HH:MM:SS)