Author Biography

Richard L. Sites wrote his first computer program in 1959 and has spent most of his career at the boundary between hardware and software, with a particular interest in CPU/software performance interactions. His past work includes VAX microcode, DEC Alpha co-architect, and inventing the performance counters found in nearly all processors today. He has done low-overhead microcode and software tracing at DEC, Adobe, Google, and Tesla. Dr. Sites earned his PhD at Stanford in 1974; he holds 66 patents and is a member of the US National Academy of Engineering.

Table of Contents

Foreword xix
Preface xxi
Acknowledgments xxv
About the Author xxvii

Part I: Measurement 1
Chapter 1: My Program Is Too Slow 3
Chapter 2: Measuring CPUs 15
Chapter 3: Measuring Memory 31
Chapter 4: CPU and Memory Interaction 49
Chapter 5: Measuring Disk/SSD 61
Chapter 6: Measuring Networks 85
Chapter 7: Disk and Network Database Interaction 111

Part II: Observation 131
Chapter 8: Logging 133
Chapter 9: Aggregate Measures 141
Chapter 10: Dashboards 157
Chapter 11: Other Existing Tools 167
Chapter 12: Traces 193
Chapter 13: Observation Tool Design Principles 209

Part III: Kernel-User Trace 217
Chapter 14: KUtrace: Goals, Design, Implementation 219
Chapter 15: KUtrace: Linux Kernel Patches 227
Chapter 16: KUtrace: Linux Loadable Module 239
Chapter 17: KUtrace: User-Mode Runtime Control 245
Chapter 18: KUtrace: Postprocessing 249
Chapter 19: KUtrace: Display of Software Dynamics 257

Part IV: Reasoning 267
Chapter 20: What to Look For 269
Chapter 21: Executing Too Much 271
Chapter 22: Executing Slowly 279
Chapter 23: Waiting for CPU 289
Chapter 24: Waiting for Memory 299
Chapter 25: Waiting for Disk 307
Chapter 26: Waiting for Network 319
Chapter 27: Waiting for Locks 337
Chapter 28: Waiting for Time 357
Chapter 29: Waiting for Queues 361
Chapter 30: Recap 383

Appendix A: Sample Servers 387
Appendix B: Trace Entries 391

Glossary 397
References 405
Index 415

Review

"Complex software often uses threads, events, and I/O to produce results. Even intermittent performance bugs can lead to functional failures when results are time sensitive. Here Sites shares his methods from decades of experience as a real-world performance detective to enable you to approach the deduction skills of an IT Sherlock Holmes."
—Mark D. Hill, Partner Hardware Architect at Microsoft and Professor Emeritus at the University of Wisconsin-Madison "I think that any senior CS student or professional can benefit by reading this book. While all the material in the first half of the book leads up to the use of KUTrace, the first two parts are worth reading on their own by anyone who wants to better understand the systems they are building and using."—Rik Farrow, ;login

Long Description

From mobile and cloud apps to video games to driverless vehicle control, more and more software is time-constrained : it must reliably deliver results seamlessly, consistently, and virtually instantaneously. If it fails to do so, customers are unhappy -- and sometimes lives are put at risk. When time-constrained software does not meet its requirements, software engineers must quickly identify and address the root causes. This is difficult, and few tools have been available to help. Now, world-renowned software performance expert Richard Sites tackles the problem head on, offering expert methods and advanced tools for understanding complex time-constrained software dynamics, improving software reliability, and troubleshooting challenging performance problems. He addresses CPU, memory, disk, network, and more, helping you fix programs that are always too slow, or work well at times but inexplicably slow down at other times. Sites draws on several decades of experience pioneering software performance optimization, and extensive experience teaching developers at the graduate level. He introduces principles and techniques you can use in any environment, from embedded devices to datacenters, illuminating them with examples based on x86 or ARM processors running Linux and linked by Ethernet. Sites also guides readers through building and applying a powerful new extremely-low-overhead open source software tool, KUtrace, to trace exactly what is executing on every CPU core. Using insights gleaned from this tool, readers can apply nuanced approaches to performance enhancement -- not merely brute-force techniques such as turning off caches or cores.

Review Text

"Complex software often uses threads, events, and I/O to produce results. Even intermittent performance bugs can lead to functional failures when results are time sensitive. Here Sites shares his methods from decades of experience as a real-world performance detective to enable you to approach the deduction skills of an IT Sherlock Holmes." --Mark D. Hill, Partner Hardware Architect at Microsoft and Professor Emeritus at the University of Wisconsin-Madison "I think that any senior CS student or professional can benefit by reading this book. While all the material in the first half of the book leads up to the use of KUTrace, the first two parts are worth reading on their own by anyone who wants to better understand the systems they are building and using."--Rik Farrow, ;login

Feature

Troubleshoot and optimize the performance of complex, time-constrained software Master approaches you can apply anywhere from mobile apps to datacenters and the cloud Find and resolve the root causes of delays and other performance anomalies Gain deep practical experience by building and using an extremely-low-overhead tool for tracing what is executing on every CPU core By Richard Sites, a pioneer in performance monitoring at Google, Adobe, HP, IBM, DEC, and beyond

Details