How much memory does a variable take?

Question

The variable "a=b" contains 1 char 'a' for the name, and 1 char 'b' for the value.

Together 2 bytes.

How many characters can you store with one byte ?

The variable needs a pointer. 8 bytes.

How many bytes do pointers take up ?

Together 10 bytes.

Does a variable "a=b" stored in memory take about 10 bytes ? And would 10 variables of the same size take about 100 bytes ?

So 1000 variables of 1000 bytes each would be almost 1MB of memory ?

I have a file data.sh that only contains variables. I need to retrieve the value of one variable in that file. I do this by using a function. (called by "'function-name' 'datafile-name' 'variable-name'")

#!/usr/pkg/bin/ksh93

readvar () {
    while read -r line
    do
            typeset "${line}"
    done < "${1}"

    nameref indirect="${2}"
    echo "${indirect}"
 }

readvar datafile variable

The function reads the file data.sh line by line. While it does that is typesets each line. After it's done with that, it makes a namereference from the variable-name in the function-call, to one of the variables of the file data.sh. To finally print the value of that variable.

When the function is finished it no longer uses up memory. But as long as the function is running it does.

This means all variables in the file data.sh are at some point stored in memory.

Correct ?

In reality I have a file with ip-addresses as variable name and a nickname as values. So I suppose this will not be such a problem on memory. But if I use this also for posts of visitors variable values will be of larger sizes. But then it would be possible to have this function only store for instance 10 variables in memory each time. However I wonder if my way of calculating this memory usage of variables is making any sense.

Edit:

This might be a solution to avoid loading the whole file in memory.

#!/bin/ksh

readvar () {
input=$(print "${2}" | sed 's/\[/\\[/g' | sed 's/\]/\\]/g')
line=$(grep "${input}" "${1}")
typeset ${line}
nameref indirect="${2}"
print "${indirect}"
}

readvar ./test.txt input[0]

With the input test.txt

input[0]=192.0.0.1
input[1]=192.0.0.2
input[2]=192.0.0.2

And the output

192.0.0.1

Edit:

Of course !!! In the original post Bash read array from an external file it said:

# you could do some validation here

so:

    while read -r line
do
    # you could do some validation here
    declare "$line"
done < "$1"

lines would be declared (or typeset in ksh) under a condition.

Answer 1

Your real concern seems not to be "how much memory does this take?" but "how can I avoid taking uselessly much memory for this?". I'm going to answer this one first. For a bunch of thoughts about the original question, see the end of my answer.

For avoiding to use up memory I propose to use grep to get the one line which is of interest to you and ignore all the others:

line=$(grep "^$2=" "$1")

Then you can extract the information you need from this line:

result=$(echo "$line" | cut -d= -f 2)

Now the variable result contains the value which would have been assigned to $2 in the file $1. Since you have no need to store more than one such result value you definitely have no memory issue.

Now, to the original question:

To find out how much memory a shell uses up for each variable is tricky. You would need to have a look into the source of the shell to be sure on the implementation. It can vary from shell to shell (you appear to be using ksh which can be different from bash in this aspect). It also can vary from version to version.

One way to get an idea would be to watch a shell process's memory usage while making the shell set variables in large amounts:

bash -c 'a="$(head -c 1000 /dev/zero | tr "\0" x)"; for ((i=0; i<1000; i++)); do eval a${i}="$a"; done; grep ^VmPeak /proc/$$/status'
bash -c 'a="$(head -c 1000 /dev/zero | tr "\0" x)"; for ((i=0; i<10000; i++)); do eval a${i}="$a"; done; grep ^VmPeak /proc/$$/status'
bash -c 'a="$(head -c 1000 /dev/zero | tr "\0" x)"; for ((i=0; i<100000; i++)); do eval a${i}="$a"; done; grep ^VmPeak /proc/$$/status'
bash -c 'a="$(head -c 1000 /dev/zero | tr "\0" x)"; for ((i=0; i<200000; i++)); do eval a${i}="$a"; done; grep ^VmPeak /proc/$$/status'

This prints the peak amount of memory in use by a bash which sets 1000, 10000, 100000, and 200000 variables with a value of 1000 x characters. On my machine (using bash 4.2.25(1)-release) this gave the following output:

VmPeak:    19308 kB
VmPeak:    30220 kB
VmPeak:   138888 kB
VmPeak:   259688 kB

This shows that the memory used is growing more or less in a linear fashion (plus a fixed offset of ~17000k) and that each new variable takes ~1.2kB of additional memory.

But as I said, other shells' results may vary.