BYOC_ID: 45966 Refactor sc demographics by Jennit07 · Pull Request #1313 · Public-Health-Scotland/source-linkage-files

Jennit07 · 2026-04-28T16:38:42Z

PR not ready - WIP

data not ready in SDL views
dataset has dependencies.

Opening as a draft to review changes.

TODO:

This PR is linked with SC Care Homes. Please merge care home branch into demographics branch before testing in development.

Data not ready in SDL - On hold for testing locally/in BYOC

#1299

Jennit07 · 2026-04-28T16:41:43Z

-  if (!fs::file_exists(get_sandpit_extract_path(type = "demographics"))) {
-    sc_demog %>%
-      write_file(get_sandpit_extract_path(type = "demographics"),
-        group_id = 3206 # hscdiip owner
-      )
-
-    sc_demog %>%
-      process_tests_sc_sandpit(type = "demographics")
-  }
-


I've removed this intermediate saving of extracts. I think this would be redundant in the automation. However, if anyone has reasons to keep this, please let me know. I think this will also appear in other social care datasets!

Jennit07 · 2026-05-13T15:10:37Z

Is SC Demographics being added to dummy_targets and run_sdl or is this done in the SC Care Home refactoring?

Good spot - i think i still need to add this to dummy_targets and run_sdl

Jennit07 · 2026-05-13T15:28:18Z

Is SC Demographics being added to dummy_targets and run_sdl or is this done in the SC Care Home refactoring?

Good spot - i think i still need to add this to dummy_targets and run_sdl

Change made. Ready for another review/test once we have the data. Thanks @LucyEmma22 for all your suggestions

LucyEmma22 · 2026-06-02T11:51:32Z

    # Function
-    get_spd_path(),
+    get_spd_data(BYOC_MODE),
    format = "file"


Should this be removed as spd_data is now a tibble rather than a filepath?

LucyEmma22 · 2026-06-02T11:52:36Z

-    get_slf_ch_name_lookup_path(),
+    slf_ch_name_lookup_data,
+    get_slf_ch_name_lookup_data(BYOC_MODE),
    format = "file"


Should this be removed as slf_ch_name_lookup_data is now a tibble rather than a filepath?

LucyEmma22 · 2026-06-02T11:58:39Z

-    slf_ch_name_lookup_path,
-    get_slf_ch_name_lookup_path(),
+    slf_ch_name_lookup_data,
+    get_slf_ch_name_lookup_data(BYOC_MODE),


Should this be:

get_slf_ch_name_lookup_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:04:01Z

+    spd_data,
    # Function
-    get_spd_path(),
+    get_spd_data(BYOC_MODE),


Should this be:

get_spd_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:04:47Z

-      age = as.difftime(180, units = "days")
-    )
+    uk_postcode_data,
+    get_uk_postcode_data(BYOC_MODE)


Should this be:

get_uk_postcode_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:17:03Z

-    "combined_deaths",
-    "chi_deaths"
-  )
+  types = c("demog")


I think this should be types = c("sc_demog_lookup")?

Aye. It should be sc_demog_lookup to match the current changes.

LucyEmma22 · 2026-06-02T12:17:52Z

+    spd_data,
+    # Function
+    get_spd_data(BYOC_MODE),
+    format = "file"


Should this be removed as spd_data is now a tibble rather than a filepath?

LucyEmma22 · 2026-06-02T12:18:18Z

+  tar_target(
+    slf_ch_name_lookup_data,
+    get_slf_ch_name_lookup_data(BYOC_MODE),
+    format = "file"


Should this be removed as slf_ch_name_lookup_data is now a tibble rather than a filepath?

LucyEmma22 · 2026-06-02T12:19:42Z

+    # Target name
+    spd_data,
+    # Function
+    get_spd_data(BYOC_MODE),


Should this be:

get_spd_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:37:19Z

+  tar_target(
+    # Target name
+    uk_postcode_data,
+    get_uk_postcode_data(BYOC_MODE)


Should this be:

get_uk_postcode_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:37:47Z

+  # Care home name look up------
+  tar_target(
+    slf_ch_name_lookup_data,
+    get_slf_ch_name_lookup_data(BYOC_MODE),


Should this be:

get_slf_ch_name_lookup_data( denodo_connect = get_denodo_connection(BYOC_MODE = BYOC_MODE), BYOC_MODE = BYOC_MODE )

LucyEmma22 · 2026-06-02T12:49:53Z

I think the following targets also need to be added to this script?

source_gp_lookup
tests_source_gp_lookup
source_pc_lookup
tests_source_pc_lookup
all_care_home
tests_all_care_home

Jennit07 · 2026-06-16T11:12:45Z


  # postcode data
-  spd_file <- read_file(spd_path) %>%
+  spd_file <- get_spd_data(BYOC_MODE) %>%


This should be in the parameter now

Jennit07 · 2026-06-16T11:12:55Z

-  spd_file <- read_file(
-    path = spd_path,
-    col_select = c(
+  spd_file <- get_spd_data(BYOC_MODE) %>%


This should be in the parameter

Jennit07 · 2026-06-16T11:21:28Z

-  ch_name_lookup <- openxlsx::read.xlsx(ch_name_lookup_path,
-    detectDates = TRUE
-  ) %>%
+  ch_name_lookup <- get_slf_ch_name_lookup_data(BYOC_MODE) %>%


add this to the parameter

Jennit07 and others added 4 commits April 24, 2026 15:39

Refactor sc demographics

05923f2

Style code

3abc488

Add missing ,

2c842d8

Update documentation

76d00f1

Jennit07 added On hold Waiting for something / someone outside of our control dependencies Pull requests that update a dependency file labels Apr 28, 2026

Style code

1ec703c

Jennit07 commented Apr 28, 2026

View reviewed changes

Update fn sc_demog_lookup_path

5b871ec

Jennit07 commented Apr 29, 2026

View reviewed changes

Comment thread R/get_sc_lookup_paths.R

Jennit07 and others added 5 commits April 29, 2026 11:23

Style code

e699ba5

Update documentation

0db7b38

Update roxygen and add ...

7b450d8

Use get_denodo_connection

0e81a2e

Update targets

c1559d0

LucyEmma22 mentioned this pull request Apr 29, 2026

BYOC_ID: 46080 Refactor SC Client #1314

Open

Jennit07 and others added 14 commits April 29, 2026 15:55

Add function get_spd_data

9beb202

Use get_spd_data in demographics

3a70618

Style code

bb42e34

WIP - to come back to

69f196d

fix brackets and commas

f43a906

Add new function get_uk_postcode_data

14d8e99

Style code

1339f92

New fn get_slf_ch_name_lookup_data

2973872

Style code

f203d95

Refactor read_sc_all_care_home

85693d3

Style code

989892a

add BYOC_MODE

e82e964

Update documentation

e44c221

Update fill_ch_names

e8a7ea2

Jennit07 added 2 commits May 13, 2026 15:20

Update dummy targets for demog file

05aaadc

Update run_sdl.r

c6074d6

Jennit07 added ready for PHS testing PR complete and removed PR ready for review labels May 20, 2026

move get_sdl_helpers.R to get_sdl_helpers branch

95611af

LucyEmma22 mentioned this pull request Jun 2, 2026

Refactor simd locality #1334

Draft

LucyEmma22 reviewed Jun 2, 2026

View reviewed changes

Jennit07 commented Jun 16, 2026

View reviewed changes

Comment thread R/fill_ch_names.R Outdated

Jennit07 commented Jun 16, 2026

View reviewed changes

Comment thread R/fill_ch_names.R Outdated

Jennit07 commented Jun 16, 2026

View reviewed changes

Comment thread R/fill_ch_names.R Outdated

Jennit07 added 3 commits June 16, 2026 12:04

Apply suggestion from @Jennit07

f9bbe74

Apply suggestion from @Jennit07

64da751

Apply suggestion from @Jennit07

6b728a1

Jennit07 commented Jun 16, 2026

View reviewed changes

Conversation

Jennit07 commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Jennit07 commented May 13, 2026

Uh oh!

Jennit07 commented May 13, 2026

Uh oh!

LucyEmma22 Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LucyEmma22 Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Jennit07 commented Apr 28, 2026 •

edited

Loading

LucyEmma22 Jun 2, 2026 •

edited

Loading

LucyEmma22 Jun 2, 2026 •

edited

Loading