Gustafson’s Law Scaled Speedup Calculator for Parallel HPC

Think about how faster machines, double the cores, for example; will speed up your simulations. The classic solution is Amdahl’s Law, which says there is a brick wall defined by fraction of your program that can’t be run in parallel. This is valid reasoning if your program is fixed size. But in today’s world of large-scale computation, the size of the problem expands as the compute power does.

Gustafson’s Law flips this notion on its head: what happens if you increase the size of the problem as well? Then there is no ceiling on speedup which scales almost directly with machine. This website computes that for you. Play around and see how much more efficient you’ll be if you let the problem size grow to match compute resources at hand.

Why Gustafson’s Law Is Better for Big Computers

That’s a fairly simple equation, but one with wide consequences for systems design. You can think about it this way: scaled speedup equals (number of processors), (penalty for serial fraction of code x rest-of-cores). You can also say it this way: (serial part) + ((processor-count) x (parallel-fraction)). Both equations gets you to the same place, just with a slightly different understanding of what might be going wrong.

On 16 processors, you have 95% of your scaled workload running in parallel; so the serial slice accounts for five percent. From the speedup calculator, we find speedup of approximately fifteen point two five times, not the expected twenty times. The difference there is the price of code that cannot run in parallel no matter how many core you throw at it.

In that case, our efficiency behaves in an interesting way compared to the traditional scaling laws. Since the parallel work grows along with the machine, the efficiency metric converges towards the parallel fraction from above for increasing processors. As long as the serial part is sufficiently small, hardware use will remain high even on very large clusters. The tool calculates this efficiency and provides it to you automatically.

This allows you to get a clear signal whether you struggle with overhead or not, did your parallelization effort pay off? Does the number go down substantially below the parallel fraction? Then probably the distribution of work over nodes are wrong.

This is unlike Amdahl’s Law, which says supercomputers will hit a hard limit on speedup; instead, Gustafson’s Law explains why they don’t hit diminishing returns and can simply keep getting bigger and faster. In Amdahl’s world, there is a hard upper bound on speedup, determined purely by the amount of serial work in the task. Gustafson’s world assumes an ever-increasing problem size. That removes the limit entirely.

The calculator has a switch for displaying these results side-by-side. The discrepancy becomes obvious. When you have very little parallel work, the difference isn’t big. At high parallel fractions, the difference is staggering: you might get only ten times speedup with Amdahl but over two hundred times with Gustafson on the same hardware configuration.

It’s why we don’t see weather models and fluid dynamics simulations stall when ported to bigger computers. When using the presets, you get an instant idea about what to expect for typical types of work. Database queries may only hit 80 percent parallelism, whereas weather simulations often hit nearly ninety-eight. Why does this matter? These are realistic representations of your code’s architecture instead of theoretical perfection. Over-estimating the parallel fraction leads to overly optimistic estimates of speedup. Under-estimating it risks turning down hardware that could benefit you.

The tool can also be used to solve for processor count. Need a certain amount of speedup by X date? The tool reverses the equation and tells you how many cores you should of provision.

But more than math, this shows the power of how you look at things. Serial isn’t something you can eliminate from your code. But do you want it to feel like a tiny cost for huge potential? Or is it a wall stopping progress? As you scale the problem, that little bit of serial cost becomes negligible compared to the overall time. This is a real example of an otherwise-abstract idea. Instead of guessing at capacity, you have numbers you can trust to plan confidently. And you see that parallel computing is a must and not just an option if you want any real speedup.