SAS编程-宏:Source侧与QC侧程序运行时间检查

目前SAS编程工作一般都要求双侧独立编程,这就像双盲试验一样,减少偏倚,提升输出结果的准确性。双侧编程中,QC侧程序运行时间需在Source侧之后。这不仅是一个很好的编程习惯,不少公司也将这一要求添加到工作手册中,程序提交之前会有专门的系统来检测。

这篇文章运用SAS编程来完成这样的程序运行时间检查,为方便程序调用,将代码封装为宏,宏程序的完整代码在第4节汇总

1. 宏程序的整体思路

宏程序主要分为两部分,第一部分,获取SAS程序以及SAS日志的末次修改时间;第二部分,比较各个末次修改时间,得出对应的结论

第一部分内容,参考文章SAS编程:如何获取某路径下所有文件的修改时间?

第二部分,我把各个末次修改时间的比较划分为4类:

  1. 程序修改时间缺失,需创建缺失程序;
  2. 日志修改时间缺失,程序需要Batch Run;
  3. SAS程序修改时间,晚于日志修改时间,程序需要Batch Run;
  4. Source侧日志修改时间,晚于QC侧修改时间,QC侧程序需要Batch Run。

演示文件夹如下,两个文件夹,一个是Source侧,一个是QC侧,SAS程序对应的日志文件在同一文件夹中

Source侧
QC侧

2. 获取SAS程序以及日志的末次修改时间

这一部分分为两个步骤,先使用Dopen系列函数获取所有文件的名称(文件路径),再使用Fopen系列函数获取所有文件的修改时间

直接参照之前的文章代码,整合成一个宏程序,宏程序中做2点额外的考量。

第一点,考虑Windows和UNIX系统中,文件地址的斜杠不同,以及在输入文件夹地址时,地址末尾可能添加斜杠,也可能不添加。根据输入的地址判断斜杠的类型,同时,统一将输入地址末尾的斜杠移除,后续使用在手动添加

第二点,因为函数FINO返回的信息,受SAS语言的影响,且返回的中文时间不方便读入。所以,在调用FINO时将SAS系统语言设置为英文(options locale = EN_US),调用结束后还原之前的选项值。这个操作是不是跟之前提到的ods listing close;ods listing;options mprint;options nomprint有异曲同工之妙?

第一部分,该程序如下:

%macro  get_last_mod_date( dirpath =, suffix =, outdt =  );

%if  "&dirpath." ne "" %then %do;

%local dirpath_tmp slash;

%let slash = %substr(%sysfunc(compress(&dirpath., : _ , a d)), 1, 1);

*Remove trailing slash;
%if "%substr(&dirpath.,%length(&dirpath.),1)" = "&slash." %then %let dirpath_tmp=%substr(&dirpath.,1,%length(&dirpath.)-1);
%else %let  dirpath_tmp = &dirpath.;


**Dopen--Get filepath;
data _tmp1;
  fileres = filename("dirpath", "&dirpath_tmp");
  dirid = dopen("dirpath");
  num = dnum(dirid);

  length direct filename filepath $200;

  if dirid > 0 and num >0 then do;
    do i = 1 to num;
      direct = "&dirpath_tmp.";
      filename = dread(dirid, i);
      filepath = catx("&slash.", direct, filename);

      if strip(scan(filename, 2, "."))="sas" or strip(scan(filename, 2, "."))="log"  then output;
    end;
  end;

  keep filename filepath;

  proc sort;
    by filename;
run;

*Set SAS language;
 %local locale_sys ;

 %let locale_sys = %sysfunc(getoption(locale));
options locale = EN_US;

**Fopen--Get Last Modified date;
data _tmp2;
  set _tmp1;

  *Get fileID;
  fileres = filename("filepath", filepath);
  fileid = fopen("filepath");

  *Get Last Modified date;
  if fileid > 0 then do;
    length lmdtc $200;
    lmdtc = finfo(fileid, "Last Modified"); 
    if lmdtc ne "" then lmdtm = input(lmdtc, datetime19.);
  end;

  *Close fileID;
  fileid_c = fclose(fileid);
  
  format lmdtm e8601dt.;

  keep filename filepath lmdtc lmdtm;
run;

options locale = &locale_sys.;


**3. Combine lmdtm of .sas and .log file;
proc sql noprint;
  create table &outdt. as
    select scan(a.filename, -2, ".") as domain_&suffix., a.lmdtm as lmdtm_sas_&suffix.,   b.lmdtm as lmdtm_log_&suffix.
  from _tmp2 as a
    left join
    _tmp2 as b
  on scan(a.filename, 1, ".") = scan(b.filename, 1,  ".") and index(a.filename, ".sas") and index(b.filename, ".log")
  where index(a.filename, ".sas")
  ;
quit;

%end;

%else %put Dirpath is missing ! ;

%mend get_last_mod_date;

将上面两个文件夹地址代入宏程序:

*Source;
%get_last_mod_date(
  dirpath = E:\99_Test\Test\test1\
  ,suffix = S
  ,outdt = Source_lmdtm
);

*QC;
%get_last_mod_date(
  dirpath = E:\99_Test\Test\test1\validation
  ,suffix = QC
  ,outdt = QC_lmdtm
);

结果如下:

Source_lmdtm
QC_lmdtm

3. 比较各文件的末次修改时间

获取Source侧和QC侧程序和日志的末次修改时间后,将两侧获取到的时间数据集拼接到一起(Full join),比较输出第1节提到的4类输出结果。

%macro check_date(resdt=, SourcePath=, QCPath= );

**Get last modified dates of files in each folder;
*Source;
%get_last_mod_date(
  dirpath = &SourcePath.
  ,suffix = S
  ,outdt = Source_lmdtm
);

*QC;
%get_last_mod_date(
  dirpath = &QCPath.
  ,suffix = QC
  ,outdt = QC_lmdtm
);


**Combine source and QC results;
proc sql noprint;
  create table _tmp3 as
    select a.*, b.*
    from source_lmdtm as a
      full join
      QC_lmdtm as b
    on a.domain_S = substr(b.domain_QC, 3) or a.domain_S = b.domain_QC
  ;
quit;


**Create results dataset;
data &resdt.;
  retain domain side resultsn results;

  length domain $64 side $10 results $200;

  set _tmp3;

  *1. SAS missing;
  if missing(lmdtm_sas_s) then do;
    domain = strip(domain_QC);
    side = "Source";
    resultsn = 11;
    results = "Source program for **"||strip(domain)||"** is not created!";

    put "results = " results;
    output;
  end;

  if missing(lmdtm_sas_QC) then do;
    domain = strip(domain_S);
    side = "QC";
    resultsn = 12;
    results = "QC program for **"||strip(domain)||"** is not created!";

    put "results = " results;
    output;
  end;

  *2. Log missing;
if not missing(lmdtm_sas_s)  and missing(lmdtm_log_s) then do;
    domain = strip(domain_S);
    side = "Source";
    resultsn = 21;
    results = "Source program for **"||strip(domain)||"** does not putty run!";

    put "results = " results;
    output;
  end;

  if not missing(lmdtm_sas_QC)  and missing(lmdtm_log_QC) then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 22;
    results = "QC program for **"||strip(domain)||"** does not putty run!";

    put "results = " results;
    output;
  end;

  *3. SAS LM after LOG;
  if lmdtm_sas_s > lmdtm_log_s >. then do;
    domain = strip(domain_S);
    side = "Source";
    resultsn = 31;
    results = "Source program for **"||strip(domain)||"** does not putty run after code update!";

    put "results = " results;
    output;
  end;

  if  lmdtm_sas_QC > lmdtm_log_QC >.  then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 32;
    results = "QC program for **"||strip(domain)||"** does not putty run after code update!";

    put "results = " results;
    output;
  end;

 *4. Source log LM after QC log;
  if  lmdtm_log_S > lmdtm_log_QC >.  then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 41;
    results = "QC program for **"||strip(domain)||"** does not putty run after Source putty run!";

    put "results = " results;
    output;
  end;
run;

%mend check_date;

宏程序的参数为,输出的结果数据集,Source文件夹地址,QC文件夹地址:

%check_date(
    resdt = check_date_SDTM
    ,SourcePath = E:\99_Test\Test\test1\
    ,QCPath = E:\99_Test\Test\test1\validation
);

输出结果为:

Check_date_SDTM

4. 完整宏程序汇总

%macro check_date(resdt=, SourcePath=, QCPath= );
**Author: Jihai;
**Date: 2022-05-22;

***1. Create a macro to get last modified dates of files in each folder;
%macro  get_last_mod_date( dirpath =, suffix =, outdt =  );

%if  "&dirpath." ne "" %then %do;

%local dirpath_tmp slash;

%let slash = %substr(%sysfunc(compress(&dirpath., : _ , a d)), 1, 1);

*Remove trailing slash;
%if "%substr(&dirpath.,%length(&dirpath.),1)" = "&slash." %then %let dirpath_tmp=%substr(&dirpath.,1,%length(&dirpath.)-1);
%else %let  dirpath_tmp = &dirpath.;


**1.1 Dopen--Get filepath;
data _tmp1;
  fileres = filename("dirpath", "&dirpath_tmp");
  dirid = dopen("dirpath");
  num = dnum(dirid);

  length direct filename filepath $200;

  if dirid > 0 and num >0 then do;
    do i = 1 to num;
      direct = "&dirpath_tmp.";
      filename = dread(dirid, i);
      filepath = catx("&slash.", direct, filename);

      if strip(scan(filename, 2, "."))="sas" or strip(scan(filename, 2, "."))="log"   then output;
    end;
  end;

  keep filename filepath;

  proc sort;
    by filename;
run;

*Set SAS language;
 %local locale_sys ;

 %let locale_sys = %sysfunc(getoption(locale));
options locale = EN_US;


**1.2 Fopen--Get Last Modified date;
data _tmp2;
  set _tmp1;

  *Get fileID;
  fileres = filename("filepath", filepath);
  fileid = fopen("filepath");

  *Get Last Modified date;
  if fileid > 0 then do;
    length lmdtc $200;
    lmdtc = finfo(fileid, "Last Modified"); 
    if lmdtc ne "" then lmdtm = input(lmdtc, datetime19.);
  end;

  *Close fileID;
  fileid_c = fclose(fileid);
  
  format lmdtm e8601dt.;

  keep filename filepath lmdtc lmdtm;
run;

options locale = &locale_sys.;


**1.3 Combine lmdtm of .sas and .log file;
proc sql noprint;
  create table &outdt. as
    select scan(a.filename, -2, ".") as domain_&suffix., a.lmdtm as lmdtm_sas_&suffix.,   b.lmdtm as lmdtm_log_&suffix.
  from _tmp2 as a
    left join
    _tmp2 as b
  on scan(a.filename, 1, ".") = scan(b.filename, 1,  ".") and index(a.filename, ".sas") and index(b.filename, ".log")
  where index(a.filename, ".sas")
  ;
quit;

%end;

%else %put Dirpath is missing ! ;

%mend get_last_mod_date;

*Source;
%get_last_mod_date(
  dirpath = &SourcePath.
  ,suffix = S
  ,outdt = Source_lmdtm
);

*QC;
%get_last_mod_date(
  dirpath = &QCPath.
  ,suffix = QC
  ,outdt = QC_lmdtm
);


***2. Combine source and QC results;
proc sql noprint;
  create table _tmp3 as
    select a.*, b.*
    from source_lmdtm as a
      full join
      QC_lmdtm as b
    on a.domain_S = substr(b.domain_QC, 3) or a.domain_S = b.domain_QC
  ;
quit;


***3. Create results dataset;
data &resdt.;
  retain domain side resultsn results;

  length domain $64 side $10 results $200;

  set _tmp3;

  **3.1 SAS missing;
  if missing(lmdtm_sas_s) then do;
    domain = strip(domain_QC);
    side = "Source";
    resultsn = 11;
    results = "Source program for **"||strip(domain)||"** is not created!";

    put "results = " results;
    output;
  end;

  if missing(lmdtm_sas_QC) then do;
    domain = strip(domain_S);
    side = "QC";
    resultsn = 12;
    results = "QC program for  **"||strip(domain)||"** is not created!";

    put "results = " results;
    output;
  end;

  **3.2 Log missing;
if not missing(lmdtm_sas_s)  and missing(lmdtm_log_s) then do;
    domain = strip(domain_S);
    side = "Source";
    resultsn = 21;
    results = "Source program for **"||strip(domain)||"** does not putty run!";

    put "results = " results;
    output;
  end;

  if not missing(lmdtm_sas_QC)  and missing(lmdtm_log_QC) then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 22;
    results = "QC program for **"||strip(domain)||"** does not putty run!";

    put "results = " results;
    output;
  end;

  **3.3 SAS LM after LOG;
  if lmdtm_sas_s > lmdtm_log_s >. then do;
    domain = strip(domain_S);
    side = "Source";
    resultsn = 31;
    results = "Source program for **"||strip(domain)||"** does not putty run after code update!";

    put "results = " results;
    output;
  end;

  if  lmdtm_sas_QC > lmdtm_log_QC >.  then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 32;
    results = "QC program for **"||strip(domain)||"** does not putty run after code update!";

    put "results = " results;
    output;
  end;

 **3.4 Source log LM after QC log;
  if  lmdtm_log_S > lmdtm_log_QC >.  then do;
    domain = strip(domain_QC);
    side = "QC";
    resultsn = 41;
    results = "QC program for **"||strip(domain)||"** does not putty run after Source putty run!";

    put "results = " results;
    output;
  end;
run;

%mend check_date;

***Invoke the macro;
%check_date(
    resdt = check_date_SDTM
    ,SourcePath = E:\99_Test\Test\test1\
    ,QCPath = E:\99_Test\Test\test1\validation
);

总结

这个宏的关键点在于获取特定文件夹下的所有文件末次修改时间,涉及DopenFopen系列函数的使用。

相关阅读:
SAS编程:Dopen系列函数介绍
SAS编程:Fopen系列函数介绍

感谢阅读, 欢迎关注!
若有疑问,欢迎评论交流!

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 202,802评论 5 476
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 85,109评论 2 379
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 149,683评论 0 335
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 54,458评论 1 273
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 63,452评论 5 364
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 48,505评论 1 281
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 37,901评论 3 395
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 36,550评论 0 256
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 40,763评论 1 296
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,556评论 2 319
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,629评论 1 329
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 33,330评论 4 318
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 38,898评论 3 307
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,897评论 0 19
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,140评论 1 259
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 42,807评论 2 349
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 42,339评论 2 342

推荐阅读更多精彩内容