NonTech | Why software can run slow – Part 5/5 | Sudden meltdowns

Welcome to one of my NonTech Posts. This type of post is targeting a non-developer audience and tries to explain specific Topics on a high-flier level.

As a developer who is writing custom software I am aware that under normal circumstances not many people at my company even know what I am doing. Even less know what I worked or am working on. Most applications just run. They are those little workers inside an organization, which keep a lot of things running.

But sometimes developers like myself get their 5 minutes of fame, when the little workers we build do not behave. For example they produce weird results or are just running slowly without an announcement.

But what causes this behavior? Why can an application have a random and sudden performance breakdown? After all, the instructions have not changed.

Software at it’s core is a complex web of algorithms which work with data structures. Every algorithm has a critical runtime path. The more complex the software is, the more different critical paths you have.

If we boil it down to the essentials you have 2 possible general causes for something in a critical path to experience or cause a performance meltdown:

  1. It breaks – Every component can either have a hidden bug or a physical defect. Sometimes they just malfunction and your shiny performance tuning efforts get flushed down the toilet.
  2. The algorithm stops scaling – This is pretty common. Algorithms have a scaling attribute, which describes how the workload of that algorithm develops with a rising amount of data to process. Some scale quite well, having a constant or linear runtime development. Some do not, their runtime development looks more like an exponential curve. If you go beyond the magical point here, you get a performance meltdown.

The first cause can be solved by replacing the malfunctioning hardware or shipping a fix for the bug.

If the second cause was not a result of a bug, it usually means that a partial redesign of the software is required. Please do not try to solve it with temporary assigning more hardware to the application, an exponential runtime curve rises faster than you can even think about new hardware…

NonTech | Why software can run slow – Part 4/5 | Data structure

Welcome to one of my NonTech Posts. This type of post is targeting a non-developer audience and tries to explain specific Topics on a high-flier level.

 

When reduced to a minimum, an application transforms data from one form into another.

Algorithms are very important for that, but algorithms are designed to work with data. And data is structured.

How an application structures it‘s data has a great influence on it‘s performance. But why?

Imagine two trucks, both need to transport cars from Point A to Point B.

Both have the same amount of cars to transport.

The difference is that the first Trucker did not manage to load all of his cars onto his truck for his first run, he must drive two times to transport all of his cars.

The second Trucker does, so he needs to drive only once. He managed to structure his payload better and is therefore faster.

Now imagine the same situation with two applications. Both request data from RAM, but the first application has a data structure that allows it to fetch only once.

The second application does not access such a tight packaged data structure and therefore must either request twice, or fill up the internal Bus system to get that data.

What is a Bus System? Oversimplified, it is what connects your RAM, CPU and all other components. Think of it as the highway, which you have to take to move data around.

In a modern computer, CPU Power is not what holds you back. Actually your CPU is ridicilously fast.

The problem is that your applications do not manage to get that data through the Bus System fast enough. Your CPU typically waits 50%-75% of runtime for data.

So the goal is to design data structures which have a very high information density.

This is especially important in regard to the very small but fast L1 and L2 Caches of your CPU.

What are L1 and L2 Caches?

Think of it as very small, fast and expensive Memory right next to the CPU. L1 Cache is smaller, faster and more expensive than the L2 Cache. It also is closer to the CPU.

The Goal is to keep the data you touch often in a L1 or L2 Cache, because RAM access is 100 times slower than an access to L1 Cache.

If your data structure is chubby and has a low information density, you will not be able to fit it in one of these very fast caches and performance will suffer.

Your application will then move data in and out of RAM and thereby not only fail to provide the CPU with data in time, it will also fill that Highway up so no other processes can get their data through as well (Think of really bad L.A. Traffic).

As a side note your data structures also influence your algorithms. If your data structures are chubby, your algorithms will have to do more work to access all of the relevant data (or change it).

NonTech | Why software can run slow – Part 3/5 | Algorithms

Welcome to one of my NonTech Posts. This type of post is targeting a non-developer audience and tries to explain specific Topics on a high-flier level.

 

Algorithms describe what an application does. They lay out what and how much work the application needs to do.

An Example could look like this:

  1. Get input from user
  2. Select data from a Database
  3. Process each Dataset according to input – if input is invalid return to user
  4. Return to user with results

This is oversimplified, but I think you get the Idea.

Now imagine that every single one of those points above are just a collection of algorithms themselves. There is no magic command I can give a computer to just „process data“ or „check the user input“.

So what about these algorithms is relevant for performance?

Well, algorithms determine how much work your application has to do.

So the goal here would be to design algorithms which produce the required result, but require less work to do so.

How about the example above, did you find something strange in it?

If not, look at point 3.) - do you really want the application to check your input every time it processes a Dataset? What, if your input is wrong? How much runtime did you already waste, before the application would tell you that your input was nonsense?

And if your input is right, why would you do a repetitive task in your main data-stream?

The answer is you would not, you would change the algorithm:

  1. Get input from user
  2. Check input – if invalid return to user
  3. Select data from a Database
  4. Process each Dataset according to input
  5. Return to user with results

Now you achieve the same result but with less work, because you perform your input check only once.

The basic Message I want you to take with you is that algorithms determine how much work your application has to do. If the algorithm is more efficient, your application needs to do less and performance improves.

If the algorithm on the other hand is bad, you are facing a serious uphill battle which you will in almost all cases lose. Bad algorithms are a guarantee for a bad runtime.

NonTech | Why software can run slow – Part 2/5 | Resources

Welcome to one of my NonTech Posts. This type of post is targeting a non-developer audience and tries to explain specific Topics on a high-flier level.

 

Before we explore what makes an application unbearably slow, we should clarify what an application is.

An application is developed by humans (for now at least) to solve a defined problem. For example the Google Search engine: If you want to search for stuff on the Internet, it enables you to do it.

Every big company has it‘s own custom problems, with it‘s own gazillion programs which solve these problems (report this, print that etc.). You me be familiar with some of these little fellows, one of them probably brought you here.

Your application runs on a physical existing computer (yes, cloud included), this computer runs on electricity and has a certain spec or configuration.

Like the way a car has a almost distinct configuration (form, color, engine), a computer also has one.

We want to focus at the resources a computer provides an application:

  • CPU – Central processing Unit

This is the main engine, it provides your application with the possibility to compute things. The goal is to feed this with as much information as you can and always keeping it busy.

  • RAM – Random-Access Memory

If your application consists of more than adding two numbers, it needs memory to store its internal data. The goal is to use only as much as you need, it is a finite resource. Furthermore you want to access it in a fast manner, so that you do not keep the CPU waiting.

  • Network Connection

Used for useful things like for example an Internet access. It is a good idea to reduce the time spent here.

  • Access to a Storage Device

This can range from a simple hard drive to a very expensive SSD or another external storage device. The point is that the information does not disappear when you turn the lights off. It is a good idea to reduce the time spent here.

 

In essence the idea is to keep the CPU busy at all times and reducing the time spent working with other resources.

By using those resources in a smart way, an application can get away with transferring and computing a lot of data in a very short time frame – it runs blazingly fast.

But if an application does not do that, the runtime is sometimes hair-raising.

If you remember nothing else from these posts, just remember this:

An application, which uses the hardware poorly, will never be fast.

 

NonTech | Why software can run slow – Part 1/5 | Overview

Welcome to one of my NonTech Posts. This type of post is targeting a non-developer audience and tries to explain specific Topics on a high-flier level.

 

Did you ever wonder why some piece of software is really fast in performing its task, but another just run for hours? In this series of posts I will present you some general concepts of software performance.

If you are not a developer, but still would like to know why some applications run fast or slow, this series is for you.

If you expect deep technical details, you will not discover anything interesting.

Let me give you an overview of the main topics:

  1. Computer resources - What resources is an application using?
  2. Algorithms - Here I will introduce you to how an application does it‘s work.
  3. Data structure - The only Task of an application is to transform data from one form into another, we will talk about how it affects performance.
  4. Sometimes things go really wrong - how performance goes out of the window in production.

My Goal: After reading this series, you will have a general understanding of why software can run slow. You will also be in a better position to understand your developers, when they talk about a possible performance improvement.

 

Each of the above Topics will be discussed in a separate post. Have fun!