Memory vs. Performance

Question

This has always been on my mind while programming, so I thought I'd spit it out before it actually gets to me.

What should i be more worried about? The memory the application consumes, or the performance it takes. By this I mean should I be focused upon using less memory for of the application and using more performance (eg. loading via database, and dumping it after use), or using less performance and using more memory (eg. caching)

My conditions for the application: - It's a server application, so it's not meant to be ran on desktops, etc, I have 6GB of ram, and I have a quad-core.

Answer 1

Right now, I am dealing with the same dilemma at my current project. There are three aspects: readability, speed, and memory. Among alternative solutions and libraries that I have, each is good in one area. My conclusion was (sorted descending):

• readability (good syntax and logic)
• memory (limit memory of things to 1% of RAM)
• speed (the more, the merrier)

The main purpose is to to write a future proof code. Code has an urge to survive, live, and thrive. Nothing beats a good, simple code: beautiful syntax, easy to follow logic, and intuitive steps. Choose the most readable code.

Services and applications share CPU sequentially: by running one after another and often many microseconds of idling and waiting. They share RAM in parallel: all take their share of the memory.

Implement a solution that makes sure memory never exceed the limit in practical cases. Remember OS and other apps share the memory with you. A few percentage of RAM should be plenty for your application. Then you can think about working on seed bottlenecks (too many loops or too much waiting).

Answer 2

Like others already pointed out it just depends on the requirements of your application. When you are dealing with big data you probably will not load everything into the RAM at the programs initial load.

It is best to design programms according to hardware-requirements right from the beginning. Refactorings take a substential amount of time especially in very big programs!

I'll list the different approaches that you can take and there pros and cons, so that it is much easier for you to make a decision.

Approaches

1 - Is the best rating
3 - Is the worst rating

1) Load huge objects (the whole database) into the RAM at the beginning via ORM

Initial loading time: 3
Performance after the initial load: 1
RAM requirements: 3

Comments:

Performance after the initial load:

• After the initial database-query + ORM it is not necessary to query the database anymore.

Remarks:

• MVVMC possible
  
• Not suitable for applications with big data.

2) Fetch data only when required and use ORM

Initial loading time: 1/2
Performance after the initial load: 3
RAM requirements: 2

Comments:

Initial loading time:

• 1 or 2 Depending on if MVVMC is used or not
  
• When using MVVMC, the Model and ViewModel objects have to be created.
  Performance after the initial load:
  
• Database-queries

RAM requirements:

• When using MVVMC, the Model and ViewModel objects have to be created.
  
• Temporary RAM requirements for the fetched data due to ORM

Remarks:

• MVVMC possible

3) Fetch data only required data and do not use ORM but instead functional programming

Initial loading time: 1
Performance after the initial load: 3
RAM requirements: 1

Comments:

Performance after the initial load:

• Database-queries

Remarks:

• Useful when using a functional programming style over MVVMC and therefore populating the View directly.
• More database-query code

Answer 3

It's best not to think about it in the abstract, but in terms of a concrete design.

1. If you run out of RAM, you'll be sorry, so keep your data structure as clean and simple as possible, even if it seems that you might have to code some loops that might seem inefficient. Complexifying code and data structure because of concerns about performance is the essence of premature optimization. And, as much as people inveigh against premature optimization and claim they don't do it, they do it anyway, to a scary degree.

2. When it is working and doing what you need it to do, and if you actually have one or more performance problems, then deal with performance. The usual method is to use a profiling tool, but this is the method I prefer.

Be careful of multi-cores. Parallelism and threads allow you to get multiple agents working overlapped in time, such as disk heads, CPUs, or human clients. If, for example, your processes are I/O bound, trying to do them on multiple cores won't help much and might hurt. If there is only a single physical disk drive, you might not gain much by trying to overlap I/O bound threads, and it might hurt. On the other hand, if you've got a thread per user, that might make sense because such threads spend most of their time waiting for the user.

Answer 4

What do your customers require?

You should have at least some idea of what platform your users will be running it on. You also need to have some idea of the performance requirements (transactions per second, or whatever). Produce some conservative estimates of the minimum spec platform you will require, then design to that.

You also seem to be a bit confused in your post - using less memory is not an end goal if the purpose is to use it for caching (i.e. you are actually using the memory saved to improve performance). In which case, go for whatever gives you most bang-per-developer-hour.

Answer 5

It really depends on the kind of program. If you can control the target machines it makes it a bit easier. If you know that even at the extremes, you aren't going to run out of memory, then you ight as well use all you want. There is no advantage in memory not used by anything.

In general I think of things in several categories.

Supplemental programs, If the program is not performing the primary use of the machine then It should try and conserve memory, While not a server thing the examples I usually think of in this case are Desktop widgets and Tomboy.. They are not the primary use, so they shouldn't take too many resources away from the system, that could potentially impair the performance of the primary application.

General applications, These have simple priorities. Firstly do the job required, then if it's slow, make it faster. You shouldn't need to worry about memory too much unless you are being foolhardy (or using python or java :-) )

Many instance applications. If you expect the user to have many instances of the application either as multiple tasks or just multiple instances within the same task (like multiple firefox windows/tabs), Because things multiply you need to keep a handle on memory usage. Speed s not so much an issue of making operations faster as ensuring that idle instances are not actually doing any processing.

Jumbo applications, If your application actually has a huge task to perform, like say image manipulation, then you should consider memory usage from the outset. I suspect Evolution consumes a lot of ram (currently 142 meg on my machine) because they had a jumbo task but didn't realise it. I have a lot of email, mostly incoming from lists,

If you can control your target environment then you can just have as much ram as needed, It's easier for you. If other users are going to have your program then requiring more memory is still easier for you, but it's not friendly to the users.

I'm doing development on an OLPC XO, Largely trying to make the system nice with supplemental programs. That means I'm really focused on low memory usage, but even on a system that is that memory restricted I find that there is not much use in further reducing memory usage. After boot it has over 150 meg free. Which is enough to run all of the lightweight apps you want but most of the heavier apps will be a strain. There is very little middle ground. Further optimising a 2 meg app to only use one meg doesn't give you that much more elbowroom if you are running an app like firefox.

Answer 6

Your question has drawn a lot of Zen Buddhism-like responses. I hope to do better.

Your memory limit is hard: If you exceed it, even if there's virtual memory, your app will crawl and you'll be the laughing stock of all.

Your CPU time is unlimited: Your app will take whatever time it needs; hopefully it will be sufficiently parallel that all 4 CPUs will be, for the most part, cooking with full steam until your app is finished.

Many Computer Science problems have a variety of solutions with various tradeoffs of memory against time. So: Be generous with memory until you use at least about half of it (if that helps; don't waste memory for the heck of it!) but stop while there's enough memory left that you don't need to worry about exceeding the limit, even in exceptional cases or accidentally.

Now that you've allocated your memory resources, you can try to tweak some more small performance gains out of your code. But don't bother overdoing it.

Done.

P.S. If it doesn't work correctly and reliably, all the preceding effort is worthless. Keep that in mind all the time!

Good luck.

Answer 7

This question is as old as programming itself. The answer is, unfortunately, "it depends." If you are writing an application for a system that has 32 GB of RAM, and your software is the only thing that will be running, you should write your code to take advantage of that. If, on the other hand, you are writing code that will run on an embedded system, you should probably use as little memory as possible. What's MOST important is that you are aware of these trade offs, profile your code, and optimize whatever the biggest bottleneck is.

Answer 8

You can think of performance in terms of throughput and response time. Find ways to measure these two factors, and to setup the type of load that your system needs to handle and work from there. The memory/processing time (what you are calling "performance") decisions come once you have measured your throughput/response time under load. In general, you should try to use as much of the CPU as possible (to get the best throughput), so you can exploit all of the memory you have available.

Answer 9

I think you should work to achieve a balance between both memory and processor usage.

If you're working on a server component, I would be worried about making it work with multiple users. How many users can your application serve? Can you bring more users using same resources?

Answer 10

They both are important. You might want to cache certain objects in memory for having a better performance which might increase the memory footprint. On the other hand if your application is spending a lot of time with garbage collection (as in .net), or having unmanaged resources which have not yet released the memory, you are going to have performance issues

Answer 11

Even with your specs given this still depends on the workload your application is going to see.

• If you are processing small amounts of data at a time, you may optimize performance by prefetching the next N chunks and thereby increasing memory consumption.
• If your data is rather large, it may pretty soon fill your main memory completely and reading ahead will lead to thrashing (e.g., reading ahead forces writing data back to disk/database before it is completely processed; then you need this data back in memory and thereby force swapping of these read-ahead values).

So, first get a working version of your app. Then do profiling and see what the bottlenecks are. (Premature optimization is the root of all evil! -- Donald E. Knuth)

Answer 12

It depends

Ask a tangible question!

EDIT: If you think about caching at the design phase, then go back to start and start it again (caching is always a compromise solution)!

Answer 13

Consider the amount of data you will be dealing with and the responsiveness you require. Put some thought into the design. Build it to be maintainable, and get it working.

Then profile, and address your actual bottlenecks.

Answer 14

It depends on many factors. Which of the two will be limiting first? Do other applications need to run on the same server? Which is the harder to extend?

Answer 15

1. Make it work.

You're going to get different answers and it honestly depends per-application. There is no blanket answer that covers all cases except for:

1. Make it work.

Software can be overthought.

Getting specific, most operating systems are pretty good at caching disk I/O -- if you're doing a lot of heavy work with, say, a SQLite database, by the time you're a few connections in the operating system will likely have thrown the whole thing in memory anyway. Don't outthink the OS, in most cases.

Answer 16

There's no one right choice - it depends on your application and its requirements. However, it's often a one-or-the-other choice - you can't often (if ever) maximize performance and reduce memory consumption. If this is any kind of critical system, the maximum bounds for memory and/or lowest bounds for performance should be specified by the customer - if they aren't, they should be.