CODEWITHSUNDEEP
bashshellunixksh
javapadawan
javapadawan

Reputation: 907

What is the maximum number of characters that the ksh variable accepts?

I am trying to load and parse a really large text file. Although the loading is not a problem, but there are particular lines that have 2908778 characters on a single line.

This is causing an error in my script.

On the script below, I removed all logic and just got straight to read line. I also removed all valid lines and just left the really long line in one text file. When running I get the below error :

$ dowhiledebug.sh dump.txt
dowhiledebug.sh[6]: no space
Script Ended dump.txt

The actual script:

 #!/bin/sh
 filename=$1
 count=1
 if [ -f ${filename} ]; then
    echo "after then"
    while read line;
            do
            echo "$count"
            count=$((count+1))
            done < $filename
 else
    echo "Could not open file $filename"
 fi
 echo "Script Ended $filename"

Updated (2013-01-17)

Follow up question : Is it possible to increase the maximum number of characters that ksh variable accepts?

Upvotes: 3

Views: 12226

Answers (2)

shellter
shellter

Reputation: 37278

what OS and version of ksh? Can you echo ${.sh.version} and get a value? If so, please include in your question above. Or could this be pdksh?

Here's a test that will get you in the ballpark, assuming a modern ksh supporting (( i++ )) math evaluations:

#100 char var
var=1234578901234456789012345678901234567890123456789012345789012344567890123456789012345678901234567890

$ while (( i++ < 10000 )) ;do  var="$var$var" ; print "i=$i\t" ${#var} ; done
i=1      200
i=2      400
i=3      800
i=4      1600
i=5      3200
i=6      6400
i=7      12800
i=8      25600
i=9      51200
i=10     102400
i=11     204800
i=12     409600
i=13     819200
i=14     1638400
i=15     3276800
i=16     6553600
i=17     13107200
i=18     26214400
i=19     52428800
i=20     104857600
i=21     209715200
i=22     419430400
-ksh: out of memory

$ print -- ${.sh.version}
Version JM 93t+ 2010-05-24

AND that is just the overall size of the environment that can be supported. When dealing with the command-line environment and "words" after the program name, there is a limit to the number of words, regardless of overall size.

Some shells man page will have a section LIMITS that may show something like max-bytes 200MB, max-args 2048. This information may be in a different section, it will definitely have different labels and different values I have included, OR it may not be there at all, hence the above code loop, so look carefully around and if you find a source for this info, either add an answer to this Q, or update this one.

The bash 4.4 std man page doesn't seem to have this information and its harder to find a ksh doc all the time. Check your man ksh and hope that you can find a documented limit.

IHTH

Upvotes: 6

David W.
David W.

Reputation: 107040

The limit for any shell is the limit of the C command line maximum. Here's a little program that pulls the information out of /usr/include/limits.h for you:

cpp <<HERE | tail -1
#include <limits.h>
ARG_MAX
HERE

Mine gives me (256 * 1024) or 262144 characters.

Doesn't work if the C compiler isn't installed, but it's probably a similar limit.

Upvotes: 1

Related Questions