Man vs Db2 SMF records. Man wins.
When you work on performance metrics, you need to check the cpu consumption of some threads for a specific period. For example : Study the evolution of cpu consomption of DISTSERV correlation ID xxx on the last 3 months.
SMF101 is a nightmare on large sites because there is a lot of records to read , to load all your data in a table , and to extract your information from there. As each step, you will have troubles dealing with large data sets because the vendor tools don't provide any possibility to filter the records.
My assembler took 14 seconds cpu and 31 minutes in elapsed time (read from tape) to process +130 millions smf records. (Daily SMF production ...) With a rexx, i will be fired for exploding the monthly cost of the lpar !!!
It took me two nights to find a way to decode the cpu data, and i am proud to get through it. I am the only one to publish how to decode this field, believe me, i've google searched !!! . I am not an assembler expert, so if some one find another way to calculate it with a more efficient code , i am grateful.
The code is here :
https://github.com/ndt98/Db2z/blob/master/File.txt
or below.
Have fun.
1 contributor
| /*JOBPARM SYSAFF=* | |
| //* | |
| //ASM EXEC PGM=ASMA90,PARM='OBJECT,NODECK' 00010500 | |
| //SYSIN DD * 00010600 | |
| BAL OPSYN BAS | |
| BALR OPSYN BASR | |
| SMFSUM START 0 | |
| *----------------------------------------------------------* | |
| * 17 May 2021 * | |
| *----------------------------------------------------------* | |
| * Program : SMFSUM -* | |
| * Written on : 17.05.2021 -* | |
| * Author : Nguyen Duc Tuan -* | |
| * DESCRIPTION : Select 100 & 101 then give totals -* | |
| * selection on DISTSERV and specific correlid -* | |
| * R1 System can't used -* | |
| * R4 Smf record read -* | |
| * R5 Self Def -* | |
| * R6 Branch return register -* | |
| * R7 Active macro -* | |
| * R8/9 Read/write counters -* | |
| * R10 TOT CPU -* | |
| * R12-15 Reserved -* | |
| *----------------------------------------------------------* | |
| BEGIN SAVE (14,12) | |
| BALR 3,0 | |
| USING *,3 | |
| ST 13,SAVE+4 | |
| LA 13,SAVE | |
| * | |
| OPEN (SMFIN,INPUT,SMFOU100,OUTPUT) OPEN INPUT OUTPUT | |
| OPEN (SMFOU101,OUTPUT) | |
| OPEN (OUTDD,OUTPUT) | |
| OPEN (SNAPDD,OUTPUT) | |
| SR 8,8 0=>R8 READ COUNT | |
| SR 9,9 0=>R9 WR COUNT | |
| ZAP TOTCPU,=P'0' Init variables | |
| ZAP TOTCPUD,=P'0' | |
| *----------------------------------------------------------* | |
| *- MAIN PROGRAM -* | |
| *----------------------------------------------------------* | |
| READON DS 0H | |
| BAL 6,READREC READ RECORD | |
| BAL 6,SELREC SELECT RECORD | |
| B READON GET NEXT RECORD | |
| * | |
| FNSELREC DS 0F | |
| BR 6 Retour appelant | |
| * END OF FILE | |
| ENDFILE DS 0H | |
| * | |
| * formatage Total records read | |
| CVD 8,DBLWORD | |
| MVC TOTRD,MASK10DG | |
| * D = 8 caracteres , mais la zone edition est sur 10 chiffres | |
| * 8 caracteres = 16 chiffres, il faut donc avancer de 3 pour | |
| * editer que sur 10 chiffres | |
| ED TOTRD,DBLWORD+3 | |
| MVC PRTTXT,=CL18'SMF RECORDS READ :' | |
| MVC PRTVAL,TOTRD | |
| MVC PRTVAL+10,=CL7' ' | |
| WTO MF=(E,WTOBLOC2),ROUTCDE=11 DISPLAY WTOBLOC2 | |
| * formattage & Display Total records selected | |
| CVD 9,DBLWORD | |
| MVC TOTWR,MASK10DG | |
| ED TOTWR,DBLWORD+3 | |
| MVC PRTTXT,=CL18'RECORDS SELECTED :' | |
| MVC PRTVAL,TOTWR | |
| MVC PRTVAL+10,=CL7' ' | |
| WTO MF=(E,WTOBLOC2),ROUTCDE=11 DISPLAY WTOBLOC2 | |
| * formattage Total cpu DDF | |
| MVC OTOTCPU,MASK12DG | |
| ED OTOTCPU,TOTCPUD 12 DIGITS (6 AFTER COMMA) | |
| * Check if overflow on totcpu | |
| SNAP DCB=SNAPDD,ID=01,PDATA=(REGS), X | |
| STORAGE=(BEGINW,LASTW) | |
| MVC PRTTXT,=CL18'TOTAL CPU DDF :' | |
| MVC PRTVAL,OTOTCPU | |
| WTO MF=(E,WTOBLOC2),ROUTCDE=11 DISPLAY WTOBLOC2 | |
| * | |
| * formattage Total cpu | |
| MVC OTOTCPU,MASK12DG | |
| ED OTOTCPU,TOTCPU 12 DIGITS (6 AFTER COMMA) | |
| * Check if overflow on totcpu | |
| SNAP DCB=SNAPDD,ID=02,PDATA=(REGS), X | |
| STORAGE=(BEGINW,LASTW) | |
| MVC PRTTXT,=CL18'TOTAL CPU :' | |
| MVC PRTVAL,OTOTCPU | |
| WTO MF=(E,WTOBLOC2),ROUTCDE=11 DISPLAY WTOBLOC2 | |
| * | |
| CLOSE (SMFIN,,SMFOU100,,SNAPDD) CLOSE FILES | |
| CLOSE (SMFOU101) | |
| CLOSE (OUTDD) | |
| L 13,SAVE+4 | |
| LM 14,12,12(13) | |
| SR 15,15 | |
| BR 14 R14 = RETURNED ADDRESS | |
| *----------------------------------------------------------* | |
| *- READREC ROUTINE -* | |
| *----------------------------------------------------------* | |
| READREC DS 0H | |
| USING SM100,4 SET UP ADDR.ABILITY | |
| GET SMFIN GET RECORD LOCATE MODE (READ) | |
| LR 4,1 LOAD R4 WITH RECORD | |
| A 8,=F'1' Increment R8 by 1 (Read count) | |
| BR 6 | |
| *----------------------------------------------------------* | |
| *- SELREC ROUTINE -* | |
| *----------------------------------------------------------* | |
| SELREC CLI SM100RTY,X'65' IS THIS SMF101 ? | |
| BNE SEL100 NO | |
| * SNAP DCB=SNAPDD,ID=02,PDATA=(REGS) SNAPSHOT | |
| * smf 101 here | |
| CLC SM100STF,=X'0000' subtype 0 ? (no pack records) | |
| BNE FNSELREC Not a right SMF101 | |
| USING SM101,4 SET UP ADDRESSABILITY | |
| * Select SSID | |
| CLC SM101SSI,=CL4'DBP9' Db2 Name | |
| BNE FNSELREC | |
| * Begin processing | |
| LA 5,SM101END Adresse de debut self-def | |
| USING QWA0,5 --> Self def-Section | |
| L 7,QWA01PSO QWA01PSO est un Offset par rapport | |
| AR 7,4 au debut du record, pour avoir | |
| * adresse absolue ajouter R4 | |
| USING QWHS,7 --> Std header puis Corr header | |
| * Select Plan ? CorrelId ? Conntype | |
| LA 7,QWHSEND Adresse Correlation header | |
| * juste apres standard header | |
| USING QWHC,7 --> adressabilite | |
| * CLC QWHCATYP,=F'1' Conntype Batch | |
| * CLC QWHCATYP,=F'8' CONNTYPE DDF | |
| CLC QWHCPLAN,=CL8'M8CB08PP' | |
| BNE FNSELREC | |
| * MVC OQWHCCV,QWHCCV /* Save first 4 characters corrid*/ | |
| DROP 7 Plus besoin de Corr Header | |
| * Access to DSNDQWAC , R5 stills point to QWA0 (Self Def section) | |
| L 7,QWA01R1O QWA01R1O est l'offset vers DSNDQWAC | |
| AR 7,4 | |
| USING QWAC,7 | |
| * cl1 tcb time | |
| LG 11,QWACBJST | |
| LG 10,QWACEJST | |
| SRLG 10,10,12 SHIFT 12 BITS (3 BYTES) TO THE RIGHT | |
| SRLG 11,11,12 SHIFT 12 BITS (3 BYTES) TO THE RIGHT | |
| SGR 10,11 | |
| CVDG 10,DBLWORD2 Can have truncation if very large value | |
| * ----------------------------- | |
| * cl2 tcb time into R10 | |
| * ----------------------------- | |
| * Variable is on 2 bytes, we take only the second byte (accept | |
| * truncation because we suppose that left byte not use | |
| * FFFF is 65535 too high for our usage which is cpu | |
| * L 10,QWACAJST+4 | |
| * Take only the first 13 octets and / by 1000000 | |
| * Shift Right Logical (SRL) s and / by 1000000 | |
| * Cela revient a decaler de 3 octets vers la droite, | |
| * REXX 000000000013925A => 13 premiers => 0000000000139 | |
| * ASM 0013925A => 0000 0000 0001 0011 1001 0010 0101 1010 | |
| * SRL 12 => 0000 0000 0000 0000 0000 0001 0011 1001 | |
| * SRL 12 => 1 3 9 | |
| * SRL 10,12 Shift 12 bits (3 bytes) to the right | |
| * CVD 10,DBLWORD Go to Pack decimal | |
| * Don't use SRP cause lost of precision | |
| * SRP DBLWORD,64-3,5 Shift 3 digits to right (/1000) | |
| * Write out the value for debug | |
| MVC OTOTCPU,MASK12DG | |
| ED OTOTCPU,DBLWORD | |
| PUT OUTDD,OTOTCPU | |
| * Add to total | |
| AP TOTCPUD,DBLWORD Add to total DDF | |
| * CLC OQWHCCV,=CL4'R4ZC' /*corrid =job/tran name*/ | |
| * BNE FNSELREC | |
| * AP TOTCPU,DBLWORD Add to total selection | |
| ** AR 11,10 Add r10 to r11 | |
| * End processing accounting records | |
| * Write SMF101 | |
| * OK on a choisi ce record | |
| A 9,=F'1' Increment R9 by 1 | |
| * PUT SMFOU101,SM101 | |
| B FNSELREC | |
| SEL100 DS 0H Reconduct SMF100 records | |
| USING SM100,4 adressability | |
| CLI SM100RTY,X'64' IS THIS SMF100 ? | |
| BNE FNSELREC NO READ NEXT RECORD | |
| * PUT SMFOU100,SM100 | |
| B FNSELREC | |
| *----------------------------------------------------------* | |
| * ZONES POUR EDITION * | |
| *----------------------------------------------------------* | |
| MASK10DG DC X'40202020202020202020' MASK 10 DIGITS | |
| MASK12DG DC X'402020202020202020204B202020202020' | |
| TOTRD DS CL10 NBR RECORDS READ | |
| TOTWR DS CL10 NBR RECORDS WRITTEN | |
| DS 0H HALF WORD ALIGNMENT | |
| WTOBLOC2 DC H'39' 18+17+4 | |
| DC H'0' | |
| PRTTXT DS CL18 | |
| PRTVAL DS CL17 | |
| *----------------------------------------------------------* | |
| *- FILE SECTION -* | |
| *----------------------------------------------------------* | |
| *----------------------------------------------------------* | |
| SMFIN DCB DDNAME=SMFIN, X | |
| DSORG=PS, X | |
| MACRF=GL, X | |
| EODAD=ENDFILE, X | |
| BFTEK=A, X | |
| RECFM=VBS,BUFNO=20 | |
| SMFOU100 DCB DDNAME=SMFOU100, X | |
| DSORG=PS, X | |
| MACRF=PM, X | |
| LRECL=32767, X | |
| RECFM=VBS, X | |
| BLKSIZE=27998 | |
| OUTDD DCB DDNAME=OUTDD, X | |
| DSORG=PS, X | |
| MACRF=PM, X | |
| LRECL=14, X | |
| RECFM=FB, X | |
| BLKSIZE=0 | |
| SMFOU101 DCB DDNAME=SMFOU101, X | |
| DSORG=PS, X | |
| MACRF=PM, X | |
| LRECL=32767, X | |
| RECFM=VBS, X | |
| BLKSIZE=27998 | |
| SNAPDD DCB DSORG=PS,RECFM=VBA,MACRF=W,LRECL=125,BLKSIZE=882, X | |
| DDNAME=SNAPDD | |
| *----------------------------------------------------------* | |
| *- WORKING STORAGE SECTION -* | |
| *----------------------------------------------------------* | |
| SAVE DS 18F | |
| OQWHCCV DS CL4 | |
| *FLLWORD DS F Full word | |
| DS 0D Alignment | |
| DBLWORD2 DS 0DL2 | |
| DS D | |
| DBLWORD DS D work | |
| BEGINW DS 0H | |
| DC CL8'>>>>>>>>' | |
| TOTCPUD DS D Total CPu Ddf | |
| TOTCPU DS D Total CPu Db2 | |
| LASTW DS 0H | |
| OTOTCPU DS CL17 TOT CPU | |
| DC CL8'<<<<<<<<' | |
| *----------------------------------------------------------* | |
| *- DSECT/MACRO SECTION -* | |
| *----------------------------------------------------------* | |
| SMFTYPDA DS 0H | |
| DSNDQWAS DSECT=YES,SUBTYPE= | |
| SMFSUM CSECT | |
| SELFDEFS DS 0H | |
| DSNDQWA0 DSECT=YES Self defined | |
| SMFSUM CSECT | |
| SMFTYPDB DS 0H | |
| DSNDQWST DSECT=YES,SUBTYPE= | |
| SMFSUM CSECT | |
| PRODSECT DS 0H Production Standard header | |
| DSNDQWHS DSECT=YES | |
| SMFSUM CSECT | |
| PRODSEC2 DS 0H Production Header type 2 for correlation Id | |
| DSNDQWHC DSECT=YES | |
| SMFSUM CSECT | |
| ACCTSEC DS 0H Accounting ifcid3 | |
| DSNDQWAC DSECT=YES | |
| SMFSUM CSECT | |
| LAST DS CL1 | |
| END BEGIN | |
