Cpt Global Limited · ASX:CGO
Cpt Global — UKCMG Conference 2010
36 slides from Cpt Global's conference presentation, May 18, 2010
Slide 1: UKCMG Conference 2010: Tuning where it matters – a case study
UKCMG Conference 2010 Tuning where it matters – a case study 18 May 2010 Prepared by CPT Global Limited Geoff Scott Telephone: +44 (0) 7758 225899 Email: gscott@cptglobal.com Ed Maurer Telephone: +44 (0) 7720 435879 Email: emaurer@cptglobal.com
Slide 2: Presentation content
Presentation content • Set the scene – the Client, the issue • How we targeted the reduction efforts • Approach taken – top down • Examples of things we looked for / found • Examples of changes recommended • Where we are going next • Results Page 2
Slide 3: Setting the scene
Setting the scene • Client is a large French bank • Completely different culture • Very risk-averse, and change averse • Very untrusting of non-French speaking “experts” Page 3
Slide 4: The issue
The issue • Growing monthly SCRT based software cost • Approaching the point of requiring a large upgrade Page 4
Slide 5: Technical environment
Technical environment • Client had recently centralised a number of region data centres onto 2 CECs in 2 data centres in a central location • Consolidation had just been physical, so LPARs were still regional, though mainly ran the same applications with regional flavours • Throughout the project th
Slide 6: Tuning focus
Tuning focus Typical Challenge Approach • Capacity Issue • Move workloads to quieter times Run own • Tune what is left systems • Reduce software bill • Need to focus on the peak 4HRA and smoothing the workload • Billed based on peak hour • Move workloads to quieter times • Make outsourcer move house
Slide 7: Top down, focused approach
Top down, focused appraoch • Understand the goals of tuning • Understand the workload profile (RMF70, RMF72) • Decide on area of focus • Identify the big burners (SMF30) • Use a sampling tool to understand what they are doing • Build and implement recommendations Page 7
Slide 8: Focus where it matters
Focus where it matters Data for CEC « B », the other CEC has a Peak monthly 4HRA driven similar profile by online morning peak MIPS Hourly intervals for week starting 2 December 2008 Page 8
Slide 9: Focus where it matters
Focus where it matters Peak monthly 4HRA driven by online morning peak MIPS Hourly intervals for week starting 2 December 2008 Page 9
Slide 10: Focus where it matters
Focus where it matters Batch Large online peak Large midday throughout day batch suite MIPS 15 minute intervals for Tuesday 3 December 2008 Page 10
Slide 11: Initial sample – largest CICS region
Initial sample – largest CICS region High RACF overhead MODULE !SECTION !PROCEDURE/FUNCTION CPU TIME PERCENT! NAME NAME NAME !TOTAL ! .SUPERVS!IGC0013{ IRRRFC23 SUPERVISOR SERVICES !17.78! .NUCLEUS!IEAVESL SUSPEND LOCK SERVICE ! 7.36! .DB2 DSNK2DM DSNKNXT2 FETCH NEXT TYPE 2 INDEX ! 4.64 ! .DB2 DSNK2
Slide 12: Reducing the RACF overhead
Reducing the RACF overhead • SIT security parameters • Check the Xxxx parameters in SIT • XPPT and XFCT are very expensive • If they are set, ask why • MRO session security • With ATTACHSEC = IDENTIFY on a CICS MRO link, CICS will perform 2 RACF checks at transaction attach: one for the incoming use
Slide 13: Focus where it matters - 2
Focus where it matters - 2 Peak now overnight batch MIPS Hourly intervals for week starting 2 March 2008 Page 13
Slide 14: Dealing with batch
Dealing with batch • Spread what you can into white space • Tune or squeeze the rest • The difficult bit is convincing the operations area Page 14
Slide 15: Modeling changes to backups
Modeling changes to backups BATMD load shown over both CECs Batch all kicks off at 19h00 Backups around 20% of batch MIPS Hourly intervals for week starting 30 March 2009 Page 15
Slide 16: Modeling changes to backups
Modeling changes to backups 80% of backups now delayed until 02h00 Saving in hour MIPS Hourly intervals for week starting 30 March 2009 Page 16
Slide 17: Modeling changes to backups
Modeling changes to backups Assume 50% compression removed MIPS Hourly intervals for week starting 30 March 2009 Page 17
Slide 18: Modeling changes to backups
Modeling changes to backups BATMD squeezed to 5000 MIPS MIPS Hourly intervals for week starting 30 March 2009 Page 18
Slide 19: System setup - monitors & traces
SYSTEM SETUP - MONITORS & TRACES • The way to reduce CICS Trace overhead is to turn off the all options you can • Default for standard trace in CICS is level 1 for all components (STNTRxx = 1 in SIT) • Trace points from most of these domains are of no use in debugging application problems • For most
Slide 20: System setup - monitors & traces
SYSTEM SETUP - MONITORS & TRACES • The only thing you can go with monitors is try to switch off as many options as you can get away with TXSTGCOL means that TMON645 < 4720 .02 . TMON912 > 968 .17 . TMON GLOBAL TRACE TMON924 > 5128 .21 . is monitoring transaction TM2N950 > 11288 .19 . storage utilisa
Slide 21: Investigating high DB2 activity in CICS
Investigating high DB2 activity in CICS • Use your sampling tool to identify high burning DBRMs ** SQL CPU USAGE SUMMARY ** SQL SQL STMT EXECUTION TIME/COUNT % CPU TIME TYPE NAME CNT AVG-TIME TOTAL …… DBRM NK1K600T 30,306 .0000 09:06:59 21/01/2005 .11 . DBRM NK1NT0AT 194 .0035 16:31:23 13/04/2005 .1
Slide 22: Focus on application code
FOCUS ON APPLICATION CODE • Cover simple code changes • Look for tight loops, definition of index variables MODULE SECTION PROCEDURE/FUNCTION STARTING PROCEDURE CPU TIME NAME NAME NAME LOCATION LENGTH TOTAL .LELIB CEEBINIT CEEVGTSI GET A STACK INCREMENT 13.36 .COBLIB IGZCPAC IGZCFPC COBOL LIBRARY SU
Slide 23: Focus on application code
FOCUS ON APPLICATION CODE • Offset points to area of code doing basic index function IF WSOL-NUCTE-SR > 900 AND < 961 MOVE WPRO TO WPRO5 END-IF. IF ( WSOL-NUCTE-SR > 200 AND < 225 ) OR WSOL-NUCTE-SR = 298 MOVE WSCL TO WSCL5 END-IF. • Variables are declared as characters 20 WSOL-NUCTE PIC 9(10) VALUE
Slide 24: Lingering RACF overhead
LINGERING RACF OVERHEAD • RACF overhead is still high • Noticed odd user sign-on behaviour in CICS logs • Investigated further with Type 80 RACF records • Spotted sign-on trend • Piece together user behavior Page 24
Slide 25: Accessing CICS from branch desktop
Accessing CICS from branch desktop Vtam / GOR Many AORs TELNET Application on desktop TOR GOR CTGs WOR Websphere on AIX Page 25
Slide 26: Logon via GOR
Logon via GOR Vtam / GOR Many AORs Connects to TELNET mainframe Autoinstall Signon IPADDR : Port NETNAME TERMID ACEE TXN1 No TERMID Ship Signon Single TERMID business transaction TERMID ACEE TXN1 TXNn TXNn CESF Breaks Delete Signoff connection Page 26
Slide 27: Logon via GOR
Logon via GOR Vtam / GOR Many AORs TELNET TERMID ACEE IPADDR : New = NEW = NEW = NEW OVERHEAD Port number NETNAME TERMID ACEE TXN1 No TERMID Next Ship Business TERMID Transaction = NEW = NEW TXN1 TERMID ACEE TXNn TXNn CESF Breaks Delete Signoff connection Page 27
Slide 28: Ideal logon via GOR
Ideal logon via GOR Vtam / GOR Many AORs TELNET First NETNAME TERMID ACEE business transaction TXN1 Ship TERMID TERMID ACEE TXN1 Single TXNn Business Day TXNn TXNn TXNn Subsequent TXNn business TXNn transactions TXNn TXNn TXNn Page 28
Slide 29: Measuring results on a region basis
Measuring results on a region basis • A key success point for early opportunities was building confidence within the technical and operational areas • Opportunies were measured at a very granular level to prove the effectiveness Page 29
Slide 30: Transactions per hour
Measuring system wide results Transactions per hour Page 30
Slide 31: Summary of opportunities raised
Summary of opportunities raised Number of Savings to 4HRA Savings to 4HRA Opportunities at midday in batch peak Move 6 200 MIPS 890 MIPS workloads System 43 1,940 MIPS 210 MIPS settings Application 29 244 MIPS 330 MIPS changes Access path 34 1,010 MIPS 110 MIPS improvements Page 31
Slide 32: Results over the project lifetime
Results over the project lifetime CICS CPU hours consumed in the month Millions of transactions in the month Start of tuning implementations + 35% - 13% Page 32
Slide 33: Continue to focus where it matters
Continue to focus where it matters Less batch Peak load around 800 Large midday throughout MIPS lower running 38% batch suite the day more transactions now at 13h00 MIPS 15 minute intervals for Tuesday 2 March 2010 Page 33
Slide 34: Current position
[ 00%%T | sper | Og:eT SUeL | 00:€t | sven oezt 49 | o0:zt SPIT | oett | SUIT | 00:tT Sp:0T | €or XIN [stor o0:0r NY | sre | 086 6 | 006 | sve 088 | sve | 008 X L—~ CECB [f ZL [. 00:%T Sper | Ofer | Stet oo-et rad 43 SUE 00:2t | Svett | O€:TT SUIT | 00%tt | Sv:0T o€:0T | St:OT | 00:01 NWN ts | ST6 |
Slide 35: Summary
Summary • You need to establish the business reason for tuning • Focus where it will achieve the impact you want • Start with big initiatives to gain momentum • Take an iterative approach • Track & publish results • Keep reminding yourself what the goal is Page 35
Slide 36: Q&A
Q&A Page 36 OPT Gionai