-
Notifications
You must be signed in to change notification settings - Fork 10
Expand file tree
/
Copy pathcln_000makefile.sas
More file actions
4235 lines (3531 loc) · 178 KB
/
cln_000makefile.sas
File metadata and controls
4235 lines (3531 loc) · 178 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This code should run 'lights out', day or night. All data is downloaded from GitHub
programatically, except the makefile.
Download the makefile into the Classic 1980s SAS editor and highlight and submit sections of code.
You should be able to run the entire script but I suggest you do a piece at a time.
After setup you could put together a short driver that just calls the macros.
But I think that is a little premature. Code is under development.
*****************************************************************************************************************;
* *;
*; %let pgm=cln_000makefile; *;
* *;
*; %let purpose=An example of CDISC end to end processing of a clinical traial; *;
* *;
*; %put &=pgm; *;
*; %put &=purpose; *;
* *;
* I suggest you run sections of this code by highlighting and submitting. *;
* The entire script should run but I am not sure about all the dependecies. *;
* *;
* This code s still under development. *;
* *;
* ==============================================================================================================*;
* *;
* PROJECT TOKEN = cln *;
* *;
* WIN 10 64bit SAS 9.4M6(64bit) (This code will not run in lockdown an probaly not in EG. *;
* Best with Classic SAS *;
* *;
* Documentation *;
* ============== *;
* *;
* Origin (I have heavily modified this repository- for better or worse) *;
* https://github.com/wyp1125/SAS-Clinical-Trials-Toolkit *;
* Many thanks to the author. *;
* *;
* This is by no means a full and perfect clinical trial. *;
* Not all codelists, domains and values are present. *;
* This is just an example code. Maybe it can be educational. *;
* *;
* All input data ia in my github repository *;
* https://raw.githubusercontent.com/rogerjdeangelis/utl-end-to-end-cdisc-SDTM-ADaM-processing *;
* _ _ _ _ *;
* _____ _| |_ ___ _ __ _ __ __ _| | (_)_ __ _ __ _ _| |_ ___ *;
* / _ \ \/ / __/ _ \ `__| `_ \ / _` | | | | `_ \| `_ \| | | | __/ __| *;
* | __/> <| || __/ | | | | | (_| | | | | | | | |_) | |_| | |_\__ \ *;
* \___/_/\_\\__\___|_| |_| |_|\__,_|_| |_|_| |_| .__/ \__,_|\__|___/ *;
* |_| *;
* *;
* KEY EXTERNAL INPUT SAS EXCEL WORKBOOKS WITH DETAIL META DATA *;
* THES WORKBOOKS DRIVE ALL THE ANALYSIS EVEN THE CREATION OF THE DEFINE XML *;
* ========================================================================= *;
* _ _ _ _ *;
* _____ _____ ___| | _ __ ___ ___| |_ __ _ __| | __ _| |_ __ _ *;
* / _ \ \/ / __/ _ \ | | `_ ` _ \ / _ \ __/ _` | / _` |/ _` | __/ _` | *;
* | __/> < (_| __/ | | | | | | | __/ || (_| | | (_| | (_| | || (_| | *;
* \___/_/\_\___\___|_| |_| |_| |_|\___|\__\__,_| \__,_|\__,_|\__\__,_| *;
* *;
* GitHub (Workbooks are downloaded programatically) *;
* *;
* cln_sdtmMetadata.xlsx SDTM *;
* ====================== *;
* *;
* https://tinyurl.com/y2daeq4v *;
* *;
* *;
* cln_adamMetadata.xlsx ADaM *;
* ====================== *;
* *;
* https://tinyurl.com/y4kpeosh *;
* *;
* *;
* *;
* KEY EXTERNAL INPUT *;
* ================== *;
* *;
* GROUND TRUTH META DATA(Meta data driven processing) *;
* Below are the key sheets and tabs to produce ADAM and SDTM SAS datasets *;
* We start with two excel workbooks then create the define.xml from the workbooks. *;
* *;
* We use th meta data in the excel workbooks, not the define XML. *;
* The define XML is just excel output directed to ODM xlm sections. The ODM sections are *;
* concatenated to produce the SDTM an ADaM define XML files. *;
* _ _ _ *;
* ___ __| | |_ _ __ ___ _ __ ___ ___| |_ __ _ *;
* / __|/ _` | __| `_ ` _ \ | `_ ` _ \ / _ \ __/ _` | *;
* \__ \ (_| | |_| | | | | | | | | | | | __/ || (_| | *;
* |___/\__,_|\__|_| |_| |_| |_| |_| |_|\___|\__\__,_| *;
* *;
* All the input data is programatically downloaded from GiTHub *;
* *;
* &gbl_root/cln/xls/cln_sdtmMetadata.xlsx (excel image of sdtm_define.xml -download from Github ) *;
* *;
* SHEETS *;
* Member *;
* Name Type Vars *;
* *;
* DEFINE_HEADER_METADATA$ DATA 9 *;
* TOC_METADATA$ DATA 10 *;
* VARIABLE_METADATA$ DATA 16 *;
* VALUELEVEL_METADATA$ DATA 15 *;
* COMPUTATION_METHOD$ DATA 2 *;
* CODELISTS$ DATA 11 *;
* *;
* *;
* SHEET DEFINE_HEADER_METADATA *;
* ============================ *;
* *;
* Middle Observation(1 ) of DEFINE_HEADER_METADATA - Total Obs 1 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* STUDYOID N8 123 *;
* *;
* FILEOID C6 XYZ123 *;
* STUDYNAME C6 XYZ123 *;
* STUDYDESCRIPTION C132 A PHASE IIB, DOU *;
* PROTOCOLNAME C6 XYZ123 *;
* STANDARD C4 SDTM *;
* VERSION C5 3.1.2 *;
* SCHEMALOCATION C48 http://www.cdisc *;
* STYLESHEET C15 define1-0-0.xsl *;
* CREATIONDATE C19 2021-01-23T11:48 *;
* TOTOBS C16 1 *;
* *;
* SHEET TOC_METADATA *;
* ================== *;
* *;
* Middle Observation(5 ) of TOC_METADATA - Total Obs 11 *;
* *;
* Variable Type/Length Middle Value *;
* *;
* DOMAIN C6 SUPPDM *;
* NAME C6 SUPPDM *;
* REPEATING C3 Yes *;
* ISREFERENCEDATA C3 No *;
* PURPOSE C12 Tabulation *;
* LABEL C34 DM - Supplementa *;
* STRUCTURE C51 Supplemental Qua *;
* DOMAINKEYS C48 STUDYID, RDOMAIN *;
* CLASS C22 Supplemental Qua *;
* ARCHIVELOCATIONID C8 ./suppdm *;
* TOTOBS C16 11 *;
* *;
* *;
* SHEET VARIABLE_METADATA *;
* ======================= *;
* *;
* Middle Observation(64 ) of VARIABLE_METADATA - Total Obs 128 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* DOMAIN C6 LB ** DOMAIN (eleven domains) *;
* VARIABLE C8 LBDTC *;
* VARNUM N8 21 *;
* LENGTH N8 16 *;
* TYPE C7 date *;
* LABEL C40 Date/Time of Spe *;
* SIGNIFICANTDIGITS C1 *;
* ORIGIN C18 Central laborato *;
* DISPLAYFORMAT C3 *;
* COMPUTATIONMETHODOID C14 *;
* CODELISTNAME C8 *;
* MANDATORY C3 No *;
* ROLE C18 Timing *;
* ROLECODELIST C8 ROLECODE *;
* VALUELISTOID C13 *;
* ITEMOID C40 LB.LBDTC *;
* TOTOBS C16 128 *;
* COMMENT C2000 *;
* *;
* *;
* SHEET VALUELEVEL_METADATA *;
* ========================= *;
* *;
* Middle Observation(5 ) of VALUELEVEL_METADATA - Total Obs 10 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* VALUELISTOID C13 LBTESTCDVALUE *;
* VALUENAME C6 BILDIR * LAB TEST *;
* TYPE C5 float *;
* LENGTH N8 8 *;
* LABEL C32 Direct Bilirubin *;
* SIGNIFICANTDIGITS C1 1 *;
* ORIGIN C18 Central laborato *;
* DISPLAYFORMAT C3 4.2 *;
* COMPUTATIONMETHODOID C1 *;
* CODELISTNAME C1 *;
* MANDATORY C2 No *;
* ROLE C1 *;
* ROLECODELIST C1 *;
* ITEMOID C200 LBTESTCDVALUE.BI *;
* TOTOBS C16 10 *;
* COMMENT C2000 *;
* *;
* *;
* SHEET COMPUTATION_METHOD *;
* ========================= *;
* *;
* Middle Observation(1 ) of COMPMETHOD - Total Obs 1 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* COMPUTATIONMETHODOID C14 AGECALCULATION *;
* COMPUTATIONMETHOD C42 integer value of (BRTHDT - RFSTDTC)/365.25 *;
* *;
* *;
* SHEET CODELISTS *;
* =============== *;
* *;
* Middle Observation(67 ) of CODELISTS - Total Obs 135 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* CODELISTNAME C8 SEX *;
* CODEDVALUE C36 F *;
* RANK N 8 3 *;
* TRANSLATED C46 FEMALE *;
* TYPE C4 TEXT *;
* CODELISTDICTIONARY C6 *;
* CODELISTVERSION C3 *;
* SOURCEDATASET C11 demographic *;
* SOURCEVARIABLE C8 gender *;
* SOURCEVALUE C11 . *;
* SOURCETYPE C9 number *;
* TOTOBS C16 135 *;
* *;
* _ _ *;
* __ _ __| | __ _ _ __ ___ _ __ ___ ___| |_ __ _ *;
* / _` |/ _` |/ _` | `_ ` _ \ | `_ ` _ \ / _ \ __/ _` | *;
* | (_| | (_| | (_| | | | | | | | | | | | | __/ || (_| | *;
* \__,_|\__,_|\__,_|_| |_| |_| |_| |_| |_|\___|\__\__,_| *;
* *;
* ADAM has the sheets above and two additional sheets (excel image of adam_define.xml-download from Github ) *;
* *;
* &gbl_root/cln/xls/cln_adamMetadata.xlsx *;
* *;
* *;
* SHEET ANALYSIS_RESULTS *;
* ====================== *;
* *;
* Middle Observation(2 ) of ANALYSISRESULTS - Total Obs 5 *;
* *;
* Variable Type/Length Sample Value *;
* *;
* ANALYSISDATASET C5 ADEF *;
* DISPLAYID C13 Table_14.2.1 *;
* DISPLAYNAME C47 Responder Analysis by Visit *;
/* PROGRAMMINGCODE C800 procfreq;data=adef;bycrit1avisitn;tablestrtpn*crit1fl;run */;
* DOCLEAFID C40 SAP_Section_9.2. *;
* RESULTID C43 Responder Analysis by Visit *;
* REASON C58 Efficacy endpoint pre-specified in SAP *;
* PARAMLIST C41 XPPAIN/Pain Scor *;
* ANALYSISVARIABLES C22 CRIT1FL *;
* SELECTIONCRITERIA C54 Where ITTFL='Y' and ABLFL ^= 'Y' and PARAMCD='XPPAIN' *;
* DOCUMENTATION C24 SAP_Section_9.2. *;
* PROGRAM *;
* *;
* SHEET EXTERNALLINKS *;
* =================== *;
* *;
* Middle Observation(3 ) of externallinks - Total Obs 6 *;
* *;
* LEAFID C17 SAP_Section_9.2. *;
* LEAFRELPATH C130 .\studyxyz123\studyxyz123_csr.pdf#nameddest=SAP_SEC_9.2.2 *;
* TITLE C17 SAP Section 9.2. *;
* *;
* ____ ____ _____ _ _ *;
* _ __ __ ___ __ / ___| _ \| ___| __| | __ _| |_ __ _ *;
* | `__/ _` \ \ /\ / / | | | |_) | |_ / _` |/ _` | __/ _` | *;
* | | | (_| |\ V V / | |___| _ <| _| | (_| | (_| | || (_| | *;
* |_| \__,_| \_/\_/ \____|_| \_\_| \__,_|\__,_|\__\__,_| *;
* *;
* *;
* RAW CRF SOURCE SAS DATASETS (DOWNLOAD) *;
* --------------------------------------- *;
* *;
* Base64 encoded SAS raw for programatic downloads *;
* Downloading binary data is problematic so I provide *;
* encoded text files and SAS tools to covert to decode into binary SAS datasets. *;
* *;
* CLN_RAWADVERSE.B64 *;
* CLN_RAWDEMOGRAPHIC.B64 *;
* CLN_RAWDOSING.B64 *;
* CLN_RAWLABS.B64 *;
* CLN_RAWPAIN.B64 *;
* *;
* I aslo provide SAS datasets however I do use these, You can download and eliminate the b64 decode/ *;
* *;
* SAS V9 datasets *;
* CLN_RAWADVERSE.SAS7BDAT *;
* CLN_RAWDEMOGRAPHIC.SAS7BDAT *;
* CLN_RAWDOSING.SAS7BDAT *;
* CLN_RAWLABS.SAS7BDAT *;
* CLN_RAWPAIN.SAS7BDAT *;
* *;
* ___ _ ____ _ _ *;
* / ___| / \ / ___| | |_ ___ ___ | |___ *;
* \___ \ / _ \ \___ \ | __/ _ \ / _ \| / __| *;
* ___) / ___ \ ___) | | || (_) | (_) | \__ \ *;
* |____/_/ \_\____/ \__\___/ \___/|_|___/ *;
* *;
* https://raw.githubusercontent.com/rogerjdeangelis/CostReports/master/cst_010.sas *;
* *;
* Package cst_010.sas *;
* *;
* Collection of abour 40 tools (sas macros) *;
* *;
* *;
* ==============================================================================================================*;
* *;
* INTERNAL MACROS *;
* =============== *;
* *;
* cln_100makedefine make define.xml *;
* *;
* cln_110mkesdtmdy cdisc tools *;
* cln_120mkesrtodr *;
* cln_130cfb *;
* cln_140mkeMtyTbl *;
* cln_150domainx *;
* cln_160dtc2dt *;
* cln_170mrgsup *;
* *;
* cln_500sdtmDm create stdm datasets *;
* cln_525sdtmAe *;
* cln_550sdtmEx *;
* cln_575sdtmLb *;
* cln_600sdtmXp *;
* *;
* cln_615adamAdsl create aam datasets *;
* cln_625adamAdae *;
* cln_650adamAdef *;
* cln_675adamAdTte *;
* *;
* *;
* EXTERNAL MACROS IN AUTOCALL LIBRARY *;
* ==================================== *;
* Utility and performance macros in *;
* *;
* cln_010.sas (package of 40+ utilties) *;
* *;
* *;
* PINNACLE (You can test the v5 xport files with pinnacle - interatively) *;
* ============================================================ *;
* Download Pinnacle 21 *
* *;
* https://www.c.com/projects/installing-opencdisc-community *;
* *;
* Create parent folder and subfolders *;
* *;
* global_root/pin (set up for Pinnacle ) *;
* *;
* Cfg * for pinnacle config files *;
* Txt * for pinnacle report file *;
* Xml * for pinnacle xml files *;
* *;
* SAS Proc CDISC *;
* ============== *;
* *;
* There is proc cdisc code you can run to check the SDTM DM sas Dataset *;
* *;
/* PROC CDISC MODEL = SDTM; */;
/* SDTM SDTMVersion = "3.1"; */;
/* DOMAINDATA data = cln.Dm */;
/* domain = DM */;
/* category = SPECIAL; */;
/* run;quit; */;
* *;
* _ __ _ __ ___ ___ ___ ___ ___ *;
* | `_ \| `__/ _ \ / __/ _ \/ __/ __| *;
* | |_) | | | (_) | (_| __/\__ \__ \ *;
* | .__/|_| \___/ \___\___||___/___/ *;
* |_| *;
* *;
* 1. define root directory *;
* *;
* 2. assign libname to root directory *;
* *;
* 3. create autocall subdirectory *;
* *;
* 4. create directory structure *;
* *;
* 5. copy sas tools from the GitHub into OTO(autocall) subdirectory *;
* *;
* 6. compile tools into work.sasmacr *;
* *;
* 7. create formats *;
* *;
* 8. copy excel workbooks from the GiHub (key step) *;
* *;
* 9. create stdmDefine.xml and adamDefine.xml *;
* *;
* 10. copy raw crf sas datasets from the GitHub *;
* *;
* 11. ceate sdtm datasets and associated V5 export files *;
* *;
* ae.sas7bdat cln_sdtmAe.xpt *;
* dm.sas7bdat cln_sdtmdm.xpt *;
* ex.sas7bdat cln_sdtmLb.xpt *;
* lb.sas7bdat cln_sdtmsEx.xpt *;
* suppdm.sas7bdat cln_sdtmsuppdm.xpt *;
* xp.sas7bdat cln_sdtmsXp.xpt *;
* *;
* 12. ceate adam datasets and associated V5 export files *;
* *;
* adae.sas7bdat cln_adamadae.xpt *;
* adef.sas7bdat cln_adamadef.xpt *;
* adsl.sas7bdat cln_adamadsl.xpt *;
* adtte.sas7bdat cln_adamadtte.xpt *;
* *;
* _ _ *;
* ___ _ _| |_ _ __ _ _| |_ *;
* / _ \| | | | __| `_ \| | | | __| *;
* | (_) | |_| | |_| |_) | |_| | |_ *;
* \___/ \__,_|\__| .__/ \__,_|\__| *;
* |_| *;
* *;
* THE PRIMARY OUTPUTS ARE THE ADaM and SDTM, DEFINE.XMLs, DATASETS, AND XPORT FILES *;
* *;
* FINAL STATE OF DIRECTORIES *;
* *;
* I-INPUT FROM GITHUB O= OUTPUT *;
* *;
* Directory of &gbl_root/cln *;
* *;
* O cln_fmt.sas7bcat * formats *;
* *;
* I cln_rawadverse.sas7bdat * raw data *;
* I cln_rawdemographic.sas7bdat *;
* I cln_rawdosing.sas7bdat *;
* I cln_rawlabs.sas7bdat *;
* I cln_rawpain.sas7bdat *;
* *;
* O adae.sas7bdat * adams; *;
* O adef.sas7bdat *;
* O adsl.sas7bdat *;
* O adtte.sas7bdat *;
* *;
* O ae.sas7bdat *sdtms; *;
* O dm.sas7bdat *;
* O ex.sas7bdat *;
* O lb.sas7bdat *;
* O suppdm.sas7bdat *;
* O xp.sas7bdat *;
* *;
* <DIR> b64 *;
* I cln_adamMetadata.b64 *base 64 of excel mata data for adam datasets; *;
* I cln_sdtmMetadata.b64 *base 64 of excel mata data for sdtm datasets; *;
* *;
* I cln_rawadverse.b64 * base 64 encoded raw sas datasets; *;
* I cln_rawdemographic.b64 *;
* I cln_rawdosing.b64 *;
* I cln_rawlabs.b64 *;
* I cln_rawpain.b64 *;
* <DIR> log *;
* <DIR> oto *;
* I cln_010.sas * many macro tools; *;
* *;
* I cln_100makedefine.sas * CDISC utilities ( extract meta data from excel tabs; *;
* I cln_110mkesdtmdy.sas *;
* I cln_120mkesrtodr.sas *;
* I cln_130cfb.sas *;
* I cln_140mkeMtyTbl.sas *;
* I cln_150domainx.sas *;
* I cln_160dtc2dt.sas *;
* I cln_170mrgsup.sas *;
* *;
* I cln_500sdtmDm.sas * create sdtm datasets DM; *;
* I cln_525sdtmAe.sas * AE *;
* I cln_550sdtmEx.sas * EX *;
* I cln_575sdtmLb.sas * LB *;
* I cln_600sdtmXp.sas * XP *;
* *;
* I cln_615adamAdsl.sas * create adam datasets ADSL *;
* I cln_625adamAdae.sas * ADSL *;
* I cln_650adamAdef.sas * ADAE *;
* I cln_675adamAdTte.sas * ADEF *;
* *;
* <DIR> pdf *;
* <DIR> pin *;
* <DIR> png *;
* <DIR> rtf *;
* <DIR> sas *;
* *;
* <DIR> xls *;
* O cln_adamMetadata.xlsx * excel mata data for adam datasets; *;
* O cln_sdtmMetadata.xlsx * excel mata data for sdtm datasets; *;
* *;
* <DIR> xml *;
* *;
* O cln_adamDefine.xml * concatenation of all odm sections below *;
* *;
* O cln_adamdefine_header.xml * excel odm sections created from excel tab header *;
* O cln_adamanalysisresults.xml .. *;
* O cln_adamcompmethod.xml *;
* O cln_adamitemdef.xml *;
* O cln_adamitemdef_value.xml *;
* O cln_adamitemgroupdef.xml *;
* O cln_adamleaves.xml *;
* O cln_adamvaluelist.xml *;
* O cln_sdtmcompmethod.xml *;
* *;
* O cln_sdtmDefine.xml * concatenation of all stdm sections below *;
* *;
* O cln_sdtmdefine_header.xml * excel odm section created from excel tab header *;
* O cln_sdtmitemdef.xml .. *;
* O cln_sdtmitemdef_value.xml *;
* O cln_sdtmitemgroupdef.xml *;
* O cln_sdtmvaluelist.xml *;
* <DIR> xpt *;
* O cln_adamadtte.xpt *;
* O cln_adamadae.xpt *;
* O cln_adamadef.xpt *;
* O cln_adamadsl.xpt *;
* *;
* O cln_sdtmAe.xpt *;
* O cln_sdtmdm.xpt *;
* O cln_sdtmsEx.xpt *;
* O cln_sdtmsuppdm.xpt *;
* O cln_sdtmsXp.xpt *;
* *;
* *;
*******************************************************************************************************************;
* *;
* Version History *;
* *;
* Version Date Programmer Description *;
* ------- --------- ---------- ----------- *;
* 1.0 24JAN2021 rdeangel creation *;
* *;
* *;
*******************************************************************************************************************;
/*___ _____ _____ _ _ ____
/ ___|| ____|_ _| | | | _ \
\___ \| _| | | | | | | |_) |
___) | |___ | | | |_| | __/
|____/|_____| |_| \___/|_|
*/
%symdel
gbl_root
gbl_tools;
%let gbl_root =d:/cln; * autocall library;
%let gbl_tools=https://raw.githubusercontent.com/rogerjdeangelis/CostReports/master/cst_010.sas;
%put &=gbl_root;
%put &=gbl_tools;
* create folder even if it does not exist;
options dlcreatedir;
libname cln "&gbl_root";
options nodlcreatedir;
/* Just in case you rerun */
proc datasets lib=work kill;
run;quit;
proc catalog cat=work.sasmacr et=macro;
delete
cln_100makedefine
cln_110mkesdtmdy
cln_120mkesrtodr
cln_130cfb
cln_140mkeMtyTbl
cln_150domainx
cln_160dtc2dt
cln_170mrgsup
cln_500sdtmDm
cln_525sdtmAe
cln_550sdtmEx
cln_575sdtmLb
cln_600sdtmXp
cln_615adamAdsl
cln_625adamAdae
cln_650adamAdef
cln_675adamAdTte
;run;quit;
%utlfkil(&gbl_root/oto/cln_100makedefine.sas);
%utlfkil(&gbl_root/oto/cln_110mkesdtmdy.sas);
%utlfkil(&gbl_root/oto/cln_120mkesrtodr.sas);
%utlfkil(&gbl_root/oto/cln_130cfb.sas);
%utlfkil(&gbl_root/oto/cln_140mkeMtyTbl.sas);
%utlfkil(&gbl_root/oto/cln_150domainx.sas);
%utlfkil(&gbl_root/oto/cln_160dtc2dt.sas);
%utlfkil(&gbl_root/oto/cln_170mrgsup.sas);
%utlfkil(&gbl_root/oto/cln_500sdtmDm.sas);
%utlfkil(&gbl_root/oto/cln_525sdtmAe.sas);
%utlfkil(&gbl_root/oto/cln_550sdtmEx.sas);
%utlfkil(&gbl_root/oto/cln_575sdtmLb.sas);
%utlfkil(&gbl_root/oto/cln_600sdtmXp.sas);
%utlfkil(&gbl_root/oto/cln_615adamAdsl.sas);
%utlfkil(&gbl_root/oto/cln_625adamAdae.sas);
%utlfkil(&gbl_root/oto/cln_650adamAdef.sas);
%utlfkil(&gbl_root/oto/cln_675adamAdTte.sas);
proc datasets lib=cln nolist;
delete
adae
adef
adsl
adtte
ae
cln_rawadverse
cln_rawdemographic
cln_rawdosing
cln_rawlabs
cln_rawpain
dm
ex
lb;
run;quit;
libname sdtm "&gbl_root";
libname adam "&gbl_root";
data _null_;
length newdir $64;
newdir=dcreate('oto',"&gbl_root"); /* autocall folder */
run;quit;
options fmtsearch=(cln.cln_fmt work.formats) xwait xsync
sasautos=("&gbl_root./oto", sasautos) ;
/*
Folders so far
Directory of d:\cln
<DIR> oto * autocall library
/* _ __ _ _
___ _ __ ___ __ _| |_ ___ / _| ___ | | __| | ___ _ __ ___
/ __| `__/ _ \/ _` | __/ _ \ | |_ / _ \| |/ _` |/ _ \ `__/ __|
| (__| | | __/ (_| | || __/ | _| (_) | | (_| | __/ | \__ \
\___|_| \___|\__,_|\__\___| |_| \___/|_|\__,_|\___|_| |___/
*/
* create root folder for sas datasets;
* create subfolders;
data _null_;
length newdir $64;
newdir=dcreate('oto',"&gbl_root"); /* autocall folder */
newdir=dcreate('xls',"&gbl_root"); /* input excel image of define xml - key metatdata */
newdir=dcreate('pdf',"&gbl_root"); /* documentation */
newdir=dcreate('zip',"&gbl_root"); /* zipped saves */
newdir=dcreate('rtf',"&gbl_root"); /* tables and listings */
newdir=dcreate('png',"&gbl_root"); /* figures */
newdir=dcreate('log',"&gbl_root"); /* batch logs */
newdir=dcreate('b64',"&gbl_root"); /* base 64 binaries from github - github does not support auto downloads */
newdir=dcreate('sas',"&gbl_root"); /* base 64 binaries from github - github does not support auto downloads */
newdir=dcreate('xml',"&gbl_root"); /* define xml and define.xml odm sections */
newdir=dcreate('xpt',"&gbl_root"); /* SAS V5 export */
newdir=dcreate('pin',"&gbl_root"); /* pinacle 21 */
run;quit;
/* OUTPUT DIRECTORY STRUCTURE
Directory of &gbl_root
<DIR> b64
<DIR> log
<DIR> oto ** autocall library
<DIR> pdf
<DIR> pin
<DIR> png
<DIR> rtf
<DIR> sas
<DIR> xls
<DIR> xml
<DIR> xpt
<DIR> zip
*/
/* _ _ _
__ _ ___| |_ | |_ ___ ___ | |___
/ _` |/ _ \ __| | __/ _ \ / _ \| / __|
| (_| | __/ |_ | || (_) | (_) | \__ \
\__, |\___|\__| \__\___/ \___/|_|___/
|___/
*/
filename _bcot "&gbl_root/oto/cln_010.sas";
proc http
method='get'
url="&gbl_tools"
out= _bcot;
run;quit;
/* OUTPUT
Directory of &gbl_root
<DIR> b64
<DIR> log
<DIR> oto
cln_010.sas * many macro tools;
<DIR> pdf
<DIR> pin
<DIR> png
<DIR> rtf
<DIR> xls
<DIR> xml
<DIR> xpt
<DIR> zip
*/
/* _ _ _ _
___ ___ _ __ ___ _ __ (_) | ___ | |_ ___ ___ | |___
/ __/ _ \| `_ ` _ \| `_ \| | |/ _ \ | __/ _ \ / _ \| / __|
| (_| (_) | | | | | | |_) | | | __/ | || (_) | (_) | \__ \
\___\___/|_| |_| |_| .__/|_|_|\___| \__\___/ \___/|_|___/
|_|
*/
* compile utility macros;
filename cin "&gbl_root./oto/cln_010.sas" lrecl=4096 recfm=v;
%inc cin / nosource;
/*__ _
/ _| ___ _ __ _ __ ___ __ _| |_ ___
| |_ / _ \| `__| `_ ` _ \ / _` | __/ __|
| _| (_) | | | | | | | | (_| | |_\__ \
|_| \___/|_| |_| |_| |_|\__,_|\__|___/
*/
proc format lib=cln.cln_fmt;
value visit_labs_month
0=baseline
1=3 months
2=6 months;
value _0n1y 0 = 'N'
1 = 'Y'
;
value avisitn 1 = '3'
2 = '6'
;
value popfl 0 - high = 'Y'
other = 'N'
;
value $trt01pn 'Analgezia HCL 30 mg' = '1'
'Placebo' = '0'
;
value agegr1n 0 - 54 = "1"
55-high= "2"
;
value agegr1_ 1 = "<55 YEARS"
2 = ">=55 YEARS"
;
value $aereln 'NOT' = '0'
'POSSIBLY' = '1'
'PROBABLY' = '2'
;
value $aesevn 'MILD' = '1'
'MODERATE' = '2'
'SEVERE' = '3'
;
value relgr1n 0 = 'NOT RELATED'
1 = 'RELATED'
;
value evntdesc 0 = 'PAIN RELIEF'
1 = 'PAIN WORSENING PRIOR TO RELIEF'
2 = 'PAIN ADVERSE EVENT PRIOR TO RELIEF'
3 = 'COMPLETED STUDY PRIOR TO RELIEF'
;
run;
/*
Note: Formatds are not sent to FDA we provide code/decode pairs
proc catalog cat=cln.cln_fmt;
contents;
run;quit;
Contents of Catalog CLN.CLN_FMT
# Name Type
------------------------------------
1 AGEGR1N FORMAT
2 AGEGR1_ FORMAT
3 AVISITN FORMAT
4 EVNTDESC FORMAT
5 POPFL FORMAT
6 RELGR1N FORMAT
7 VISIT_LABS_MONTH FORMAT
8 _0N1Y FORMAT
9 AERELN FORMATC
10 AESEVN FORMATC
11 TRT01PN FORMATC
*/
/* _ _ _ _ _
__ _ ___| |_ __ _| |_ __ __ ___ __| | _| |__ ___ ___ | | _____
/ _` |/ _ \ __| \ \/ / \ \/ / \ \ /\ / / `__| |/ / `_ \ / _ \ / _ \| |/ / __|
| (_| | __/ |_ > <| |> < \ V V /| | | <| |_) | (_) | (_) | <\__ \
\__, |\___|\__| /_/\_\_/_/\_\ \_/\_/ |_| |_|\_\_.__/ \___/ \___/|_|\_\___/
|___/
*/
* DOWNLOAD EXCEL WOOKBOOKS WITH META DATA;
* you can download the workbooks manually and place in /cln/xls or run these scripts;
* The code below downloads the base64 text and converts the text to binary zipped xml ;
* Only if you need to encode the excel workbooks in bas64 for upload/download from GitHub. Bas64 is text;
%*utl_b64encode(&gbl_root/xls/cln_adamMetadata.xlsx,&gbl_root/b64/cln_adamMetadata.b64); /*only if I need to upload another newer version of the data */
%*utl_b64encode(&gbl_root/xls/cln_sdtmMetadata.xlsx,&gbl_root/b64/cln_sdtmMetadata.b64); /*only if I need to upload another newer version of the data */
* download cln.sdtmMetadata.b64 and convert to sdtmMetadata.xlsx;
filename download clear;
filename download "&gbl_root/b64/cln_sdtmMetadata.b64";
proc http
method='GET'
url="https://raw.githubusercontent.com/rogerjdeangelis/utl-end-to-end-cdisc-SDTM-ADaM-processing/master/cln_sdtmMetadata.b64"
out=download;
run;quit;
filename download clear;
%utl_b64decode(&gbl_root/b64/cln_sdtmMetadata.b64,&gbl_root/xls/cln_sdtmMetadata.xlsx);
* download cln.adamMetadata.b64 and convert to adamMetadata.xlsx;
filename download clear;
filename download "&gbl_root/b64/cln_adamMetadata.b64";
proc http
method='GET'
url="https://raw.githubusercontent.com/rogerjdeangelis/utl-end-to-end-cdisc-SDTM-ADaM-processing/master/cln_adamMetadata.b64"
out=download;
run;quit;
filename download clear;
%utl_b64decode(&gbl_root/b64/cln_adamMetadata.b64,&gbl_root/xls/cln_adamMetadata.xlsx);
run;quit;
/* structure
- added files
Directory of &gbl_root/cln
cln_fmt.sas7bcat
* formats
<DIR> b64
- cln_adamMetadata.b64 *base 64 of excel mata data for adam datasets;
- cln_sdtmMetadata.b64 *base 64 of excel mata data for sdtm datasets;
<DIR> log
<DIR> oto
cln_010.sas * many macro tools;
<DIR> pdf
<DIR> pin
<DIR> png
<DIR> rtf
<DIR> sas
<DIR> xls
- cln_adamMetadata.xlsx * excel mata data for adam datasets;
- cln_sdtmMetadata.xlsx * excel mata data for sdtm datasets;
<DIR> xml
<DIR> xpt
<DIR> zip
*/
/* _ __ _ _
__| | ___ / _(_)_ __ ___ __ ___ __ ___ | |
/ _` |/ _ \ |_| | `_ \ / _ \ \ \/ / `_ ` _ \| |
| (_| | __/ _| | | | | __/ > <| | | | | | |
\__,_|\___|_| |_|_| |_|\___| /_/\_\_| |_| |_|_|
*/
filename ft15f001 "&gbl_root/oto/cln_100makedefine.sas";
parmcards4;
%macro cln_100makedefine(path=,metadata=,out=);
/*---------------------------------------------------------------*;
* %make_define generates parts of the define.xml file for the SDTM and ADaM.
* The parts can be concatenated into the define.xml file using shell scripts.
*
* PARAMETERS:
* path = System path to where the SDTM or ADaM metadata
* file exists as well as where the define.xml
* file will be stored.
* metadata = The name of the metadata spreadsheet.
*
* It requires that the following tabs exist in the metadata file:
* DEFINE_HEADER_METADATA = define file header metadata
* TOC_METADATA = "table of contents" dataset metadata
* VARIABLE_METADATA = variable/column level metadata
* VALUELEVEL_METADATA = value/parameter level metadata
* COMPUTATIONAL_MKETHOD = computational methods
* CODELISTS = controlled terminology metadata
* ANALYSIS_RESULTS = ADaM analysis metadata. [Only for ADaM define]
* EXTERNAL_LINKS = ADaM results file pointers. [Only for ADaM define]
INPUT
=====
TWO EXECL WORKBOOKS WITH THE SHEETD THAT MAP TO THE DEFINE XLM ODM SECTIONS
cln_sdtmMetadata.xlsx
sheets
DEFINE_HEADER_METADATA
TOC_METADATA
VARIABLE_METADATA
VALUELEVEL_METADATA
COMPUTATIONAL_MKETHOD
CODELISTS
cln_adamMetadata.xlsx
sheets
DEFINE_HEADER_METADATA
TOC_METADATA
VARIABLE_METADATA
VALUELEVEL_METADATA
COMPUTATIONAL_MKETHOD
CODELISTS
ANALYSIS_RESULTS
EXTERNAL_LINKS
OUTPUT
======
Directory of d:\cln\xml
-----------------------
cln_sdtmDefine.xml 62,255 SDTM DEFINE XLM
cln_adamDefine.xml 47,905 ADAM DEFILE XML
cln_adamanalysisresults.xml 8,700 concatenate these 8 xlm sections into cln_adamDefine.xml
cln_adamcompmethod.xml 385
cln_adamdefine_header.xml 1,738
cln_adamitemdef.xml 21,165
cln_adamitemdef_value.xml 1,118
cln_adamitemgroupdef.xml 12,744
cln_adamleaves.xml 1,463
cln_adamvaluelist.xml 592
cln_sdtmcompmethod.xml 290 concatenate these 6 xlm sections into cln_sdtmDefine.xml
cln_sdtmdefine_header.xml 1,677
cln_sdtmitemdef.xml 27,026
cln_sdtmitemdef_value.xml 2,778
cln_sdtmitemgroupdef.xml 29,536
cln_sdtmvaluelist.xml 948
*/
**** GET DEFINE FILE HEADER INFORMATION METADATA;
proc import
out = define_header
datafile = "&path\&metadata"
dbms=xlsx
replace;
sheet="DEFINE_HEADER_METADATA";
run;
**** DETERMINE IF THIS IS A SDTM DEFINE FILE OR AN ADAM DEFINE FILE
**** AND SET THE STANDARD MACRO VARIABLE FOR THE REST OF THE PROGRAM;
data _null_;
set define_header;
if upcase(standard) = 'ADAM' then
call symput('standard','ADAM');
else if upcase(standard) = 'SDTM' then
call symput('standard','SDTM');
else
put "ERR" "OR: CDISC standard undefined in define_header_metadata";
run;
**** GET "TABLE OF CONTENTS" LEVEL DATASET METADATA;
proc import
out = toc_metadata
datafile = "&path\&metadata"
dbms=xlsx
replace;
sheet = "TOC_METADATA" ;
run;
**** GET THE VARIABLE METADATA;
proc import
out = VARIABLE_METADATA
datafile = "&path\&metadata"
dbms=xlsx
replace;
sheet = "VARIABLE_METADATA";