Performance and Processor Design

Learning outcomes:

  • Describe common performance measures, such as throughput, response time, and utilization, and why GHz alone does not show how fast a computer is
  • Compare ways to evaluate performance, including benchmarks, monitoring, and simulation, and judge whether a benchmark result can be trusted
  • Identify bottlenecks and saturation, and explain how feedback loops affect performance
  • Explain how instruction sets such as x86 and ARM differ, and why programs built for one do not run directly on the other

Would you like to download my PowerPoint to follow along?

  • Performance
    • Performance measures how well a system does its job
    • You usually need several measures, because "fast" can mean different things
    • Performance matters to people buying computers, people building software and systems, and everyday users
    • It depends on the software, especially the operating system, and the hardware, especially the processor
  • Why GHz Alone Doesn't Tell You How Fast a Computer Is
    • Historical note: for years, clock speed (GHz) was the number everyone compared. Still worth knowing because it's still printed on every spec sheet
    • Today, speed also depends on:
      • How many cores there are, and how much work each clock tick gets done (IBM: What is a CPU?)
      • Cache: small, very fast memory right on the processor
      • Memory and storage: for many tasks, an SSD makes a bigger difference than a faster processor
      • Heat: chips slow themselves down when they get too hot (thermal throttling)
      • Software: an efficient operating system and apps
    • That's why we test with benchmarks instead of comparing numbers on a box
  • Why Performance Monitoring and Evaluation Are Needed
    • Common reasons to evaluate performance:
      • Selection: choosing which system to buy
      • Projection: predicting how a system that doesn't exist yet will perform
      • Monitoring: watching a running system to find problems
    • Useful when building a system, deciding whether to buy or upgrade, and tuning a system to run better
  • Performance: External Goals
    • Goals you can measure from the outside, without looking inside the system
      • Throughput: get the most work done
      • Response time: respond quickly
      • Fairness: every user and process gets a fair share
      • Battery life and power use: get the most work done per watt
    • These goals often conflict. Running faster usually uses more battery
  • Performance: Internal Goals
    • Goals for the parts inside the computer
      • CPU utilization: the percent of time the CPU is busy
      • Disk utilization: the percent of time the drive is busy
      • Disk access time: how long one disk request takes
    • Busy isn't always good: 100% disk use in Task Manager usually means things are waiting
  • Performance Measures
    • Absolute measures can be counted, like throughput. Relative measures are judgments, like ease of use
    • Turnaround time: from submitting a job to getting the result
    • Response time: from pressing a key or clicking until the system responds
    • System reaction time: from a request until it first gets CPU time
  • Performance Measures Examples
    • Variance in response time: how consistent response times are. A system that's usually fast but sometimes freezes feels slow
    • Throughput: work done per unit of time
    • Workload: the amount of work given to the system
    • Capacity: the most throughput the system can handle
    • Utilization: the fraction of time a resource is in use
  • Performance Evaluation Techniques
    • Different techniques for different purposes
      • Some test the whole system, and others test one part
      • A thorough evaluation uses more than one technique
    • Before you start, decide:
      • What you're measuring (the metrics)
      • Under what workload
      • And why: what question are you trying to answer?
  • Evaluation Techniques at a Glance
    • A table in the PowerPoint sums up the main techniques: trace, profile, timing, microbenchmarks, application-specific evaluation, analytic modeling, benchmarks, synthetic programs, simulation, and performance monitoring
  • Measuring a Real System
    • Benchmarks: run a standard test and compare the score
    • Monitoring: watch a running system with tools like Task Manager or top
    • Profiling: find which parts of the OS or a program use the most time
    • Tracing: record every request, to replay or study later
    • Timings and microbenchmarks: measure one small thing, like how long it takes to start a process
  • Predicting Performance
    • Simulation: a computer model of the system, useful before it's built
      • Results must be checked against the real system, and bugs in the model give wrong answers
    • Analytic models: math models, such as queuing theory (the math of waiting in lines)
      • Fast, but real systems are often too complex to model exactly
    • Synthetic programs: fake programs built to test one part, such as memory speed
  • Benchmarks
    • Run a standard program and time it, so systems can be compared fairly
    • A good benchmark gives nearly the same result each run, and matches what you'll actually use the system for
    • Industry standards: SPEC for processors and servers, and TPC for databases (SPEC)
    • Ones you may see in reviews: Geekbench, Cinebench, and 3DMark (3DMark)
    • On Linux: the free Phoronix Test Suite (Phoronix Test Suite)
  • Can You Trust a Benchmark?
    • Some companies have tuned devices to score higher on benchmarks than in normal use
    • Ask who ran the test, and why
    • Check whether real-world tests agree with the benchmark score
  • Performance Monitoring
    • Monitoring finds slow spots that administrators and developers can fix
    • Software monitors
    • Hardware monitors
      • Counters built into the processor record events like cache misses
      • On Linux, the perf tool reads them (perf wiki)
  • Bottlenecks and Saturation
    • Bottleneck: the slowest part, holding everything else back
      • Work arrives faster than it can be handled
      • Fixing one bottleneck can reveal the next one
    • Everyday examples
      • Task Manager shows the disk at 100% while everything feels slow
      • RAM is full, so the system starts using the drive as extra memory (swapping)
      • Slow Wi-Fi makes a fast computer feel slow online
    • Saturated resource: so busy that processes get in each other's way. When memory is saturated, it's called thrashing (Week 11)
  • Feedback Loops
    • Feedback loop: information about how busy the system is changes where new work goes
    • Negative feedback pushes things back toward balance (negative doesn't mean bad)
    • Positive feedback pushes things further the same way, and can be misleading
      • If the CPU looks idle, the scheduler may add more processes
      • But low CPU use can be caused by thrashing, so adding processes makes it even worse
  • What Is an Instruction Set?
    • An instruction set is the list of basic commands a processor understands, such as add, compare, or load from memory
    • Programs are translated (compiled) into machine code: the 1s and 0s for one instruction set
    • The two big families today are x86 and ARM
    • An operating system has to be built separately for each one. That's why Linux distros offer x86 (amd64) and ARM (arm64) versions (Debian: Ports)
  • CISC and RISC
    • CISC (complex instruction set computer): many instructions, some doing a lot of work in one step
      • x86 processors from Intel and AMD
    • RISC (reduced instruction set computer): fewer, simpler instructions that each run very fast
    • The line has blurred: modern x86 chips break complex instructions into simpler, RISC-like steps inside
  • RISC and CISC: The Classic Comparison
    • Historical note: a classic comparison table (in the PowerPoint) is from the mid-2000s. It compares instruction length, how instructions are decoded, how many instructions and registers there are, and how well each handles pipelining. Its examples (Pentium, Athlon, MIPS, SPARC, G5) are mostly retired
    • Still worth knowing because terms like pipelining and registers are still used today
  • Processors Today
    • x86-64 (Intel and AMD): most Windows PCs, older Macs, and many servers
    • ARM: almost every phone and tablet, Apple silicon Macs, Windows on ARM laptops, and cloud servers like AWS Graviton (AWS Graviton)
      • Known for good performance per watt, which means long battery life
    • RISC-V: an open design anyone can use without paying license fees, growing especially in small and embedded devices (RISC-V International: About RISC-V)
  • Why Programs Don't Just Swap Between ARM and x86
    • A program's machine code is written for one instruction set, like a recipe in a language the cook can't read
    • Ways around it:
    • Web apps, and programs in languages like Python and Java, run anywhere, because the browser or the language translates for them
    • What still breaks: drivers can't be translated, so some printers, and games whose anti-cheat runs as a driver, may not work on ARM laptops
  • Beyond the CPU
    • GPUs (graphics processing units): thousands of small cores, built for graphics and now also used for AI (IBM: What is a GPU?)
    • NPUs (neural processing units): chips built for AI tasks, now in many laptops and phones (Microsoft: Copilot+ PCs developer guide)
      • Windows Copilot+ PCs require one, and Apple calls theirs the Neural Engine
    • The OS has to decide which work goes to which chip, and manage the power each one uses

Suggested Activities and Discussion Topics:

  • Discussion: what makes a computer feel slow? Which of the performance measures from this week would show the problem?
  • Activity: open Task Manager (Windows), Activity Monitor (Mac), or htop (Linux), then open a big app or game. Watch the CPU, memory, and disk. Which one hits its limit first? That is your bottleneck
  • Activity: find out what processor your phone and your computer use. Is each one ARM or x86? On Windows, look in Settings, System, About. On a Mac, choose About This Mac. On Linux, run uname -m (x86_64 means x86, aarch64 means ARM)
  • Discussion: a phone review says one phone scored 20% higher on a benchmark than another. What questions would you ask before believing it means the phone is 20% faster?
  • Discussion: you want to play a Windows game on a new ARM laptop. What might work, what might not, and why?

Would you like to see some more classes? Click here