CODEWITHSUNDEEP
linuxbashhttpreadlinesocat
mike123
mike123

Reputation: 21

Socat - Poor man's http server

I have been bashing my brains for the past few hours trying to build a poor man's http server using socat. Don't ask why or suggest alternatives. I need to do this in pure bash. So I run socat like this:

socat -v TCP-LISTEN:1234,reuseaddr,fork SYSTEM:./httpd.sh

The httpd.sh is supposed the read the http request, parse it and then send a response.

e.g. GET /index.html will output Index! and GET /random will output a random number.

The problem I have is reading the entire http request. Consider the following code used to read each line of the http request:

while read -r LINE
do
    echo "$LINE" 
done

Normally it should output back the request to the browser. The problem is after I open 127.0.0.1:1234 it just hangs waiting for something. If I CTRL + C socat, the connection closes and the response shows up in the browser. I think the while loop continues forever thus preventing the transmission of the response.

If I use the code below:

while read -r LINE
do
    echo "$LINE" 
    [ -z "$LINE" ] && break
done

the while loop breaks and the browser doesn't hang anymore. Seems like a good solution. But... in the case of a POST request the POST data isn't recorded because the break occurs right after the headers (blank line)...

POST /index.html HTTP/1.0
User-Agent: Firefox
Content-Type: application/x-www-form-urlencoded

parameter1=test

What can I do to read the entire http request via the shell script, process it and send the response without any hanging?

EDIT:

Here is something that I think might hold the answwer. If I run this command:

socat TCP4-LISTEN:1234,reuseaddr,fork,crlf SYSTEM:"echo hello world"

Everything works just fine, hello world is outputed everytime.

How does socat know when the HTTP request ends?

Upvotes: 2

Views: 2144

Answers (3)

Oki Abrian
Oki Abrian

Reputation: 21

u must detect if data Header is finish,

while read -r LINE
do

    if [[ $LINE =~ "Content-Length:" ]] ; then // check if line is content-length data
          contentLength=("$( cut -d ':' -f2 <<< $LINE |tr -d ' ')"); // get data content-length
    fi          
    if [ -z "$LINE" ]; then // detect if line is empty, this means the header data is completed
            limit=500; // set limit time, limit x 0.05s, this optional u can remove it
            li=0;
            postLength=0;
                    contentLength=$((contentLength-1))
                    while [ $postLength -lt $contentLength ]; do // Check the amount of length that has been obtained
                            postData="$postData$(timeout 0.05 cat|sed 's/\x0/*#Mystring#*/g')"; // reading post data
                            postLength=$(sed 's/\*#Mystring#\*/\x0/g' <<< "$postData" |wc -c); // get the amount of length that has been obtained 
                            li=$((li+1));
                            if [ $li -eq $limit ] ; then //remove this if, if you remove limit=, or when you not need this
                               break;// remove this line, if u not need limit timeout
                            fi;
                    done
          break; //stop while, and then stop script
      fi
done

this is example socat http POST

#!/bin/bash
contains() {
    string="$1"
    substring="$2"
    if test "${string#*$substring}" != "$string"
    then
        return 0    # $substring is in $string
    else
        return 1    # $substring is not in $string
    fi
}
Methods=()
REQUESTs=()
HOSTs=()
contentLengths=()
contentTypes=()
connections=()
referers=()
ranges=()
encodings=()
cookies=()
line=""
    timeout="timeout=5, max=1000"
while read -t 5 linec
do
linec=$(echo "$linec" | tr -d '\r\n')
line=$(echo -e "$line\n$linec")

if grep -qE '^GET /' > /dev/null <<< "$linec" ; then
    Methods+=("GET")
    REQUESTs+=("$(awk '{if(NR==1)print $0}' <<< $linec | cut -d ' ' -f2)") # extract the request
elif echo "$linec" | grep -qE '^POST /' > /dev/null ; then
    Methods+=("POST")
    REQUESTs+=("$(awk '{if(NR==1)print $0}' <<< $linec | cut -d ' ' -f2)") # extract the request
elif echo "$linec" | grep -qE '^OPTIONS /' > /dev/null ; then
        Methods+=("OPTIONS")
        REQUESTs+=("$(awk '{if(NR==1)print $0}' <<< $linec | cut -d ' ' -f2)") # extract the request
fi

if [[ $linec =~ "Host:" ]] ; then
    HOSTs+=("$( cut -d ':' -f2,3 <<< $linec|tr -d ' ')");
fi
if [[ $linec =~ "Content-Type:" ]] ; then
    contentTypes+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')");
fi

if [[ $linec =~ "Content-Length:" ]] ; then
    contentLengths+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')");
fi
if [[ $linec =~ "Connection:" ]] ; then
    connections+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')");
fi

if [[ $linec =~ "Referer:" ]] ; then
    referers+=("$( cut -d ':' -f2,3,4 <<< $linec|tr -d ' '| sed "s|https://${HOSTs[0]}||g" )")
fi

if [[ $linec =~ "Range:" ]] ; then
    ranges+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')")
fi

if [[ $linec =~ "Accept-Encoding:" ]] ; then
    encodings+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')");
fi

if [[ $linec =~ "Cookie:" ]] ; then
    cookies+=("$( cut -d ':' -f2 <<< $linec|tr -d ' ')");
fi

if [[ $linec =~ "Android" ]] ; then
    androidDevice="true"
fi



#HOST=$(echo $line | awk '{if(NR==2)print $0}' | cut -d ':' -f2 | sed -e 's/^[[:space:]]*//')

if [ -z "$linec" ]; then
    echo "$line" >> /root/range10
    line=""
    #echo  >> /root/range
    proc=true
index=0
for Method in "${Methods[@]}" ; do
    REQUEST=${REQUESTs[index]}
    HOST=${HOSTs[index]}
    contentType=${contentTypes[index]}
    #echo "$contentType" >/root/ct
    contentLength=${contentLengths[index]}
    connection=${connections[index]}
    referer=${referers[index]}
    range=${ranges[index]}
    encoding=${encodings[index]}
    cookie=${cookies[index]}
    if [ "$Method" = "POST" ] ; then
        echo "AAA:$contentType" >> /root/testResult2;
            limit=500;
            li=0;
            postLength=0;
            if [[ $contentType =~ "application/x-www-form-urlencoded" ]] ; then
                    contentLength=$((contentLength-1))
                    while [ $postLength -lt $contentLength ]; do
                            postData="$postData$(timeout 0.05 cat|sed 's/\x0/*#Mystring#*/g')";
                            postLength=$(sed 's/\*#Mystring#\*/\x0/g' <<< "$postData" |wc -c);
                            li=$((li+1));
                            if [ $li -eq $limit ] ; then 
                                break; // remove this line, if u not need limit timeout
                            fi;
                    done
            elif [[ $contentType =~ "application/json" ]] ; then
                    contentLength=$((contentLength-1))
                    while [ $postLength -lt $contentLength ]; do
                            postData="$postData$(timeout 0.05 cat|sed 's/\x0/*#Mystring#*/g')";
                            postLength=$(sed 's/\*#Mystring#\*/\x0/g' <<< "$postData" |wc -c);
                            li=$((li+1));
                            if [ $li -eq $limit ] ; then
                                break;// remove this line, if u not need limit timeout
                            fi;
                    done
            fi
    elif [ "$Method" = "OPTIONS" ] ; then
        echo 'HTTP/1.1 204 No Content';
        echo "Access-Control-Allow-Origin: *";
        echo 'Access-Control-Allow-Methods: GET, POST, OPTIONS';
        echo 'Access-Control-Allow-Headers: X-PINGOTHER, Content-Type';
        echo 'Access-Control-Max-Age: 86400';
        echo 'Vary: Accept-Encoding, Origin';
        echo 'Keep-Alive: timeout=2, max=100';
        echo 'Connection: Keep-Alive';
        echo;
    fi
index=$(( index + 1 ));
done
if [ -z "$line" ]; then
    exit
fi
if [ "$connection" = "keep-alive" ] && [ "$proc" = "true" ] ; then
    Methods=()
    REQUESTs=()
    HOSTs=()
    contentLengths=()
    connections=()
    ranges=()
    encodings=()
    cookies=()
    line=""
    
elif [ "$proc" = "true" ]  ; then
    exit;
fi
fi
done

Upvotes: 0

Armali
Armali

Reputation: 19375

What can I do to read the entire http request via the shell script, process it and send the response without any hanging?

As you write, you already have a solution for GET; in the case of a POST request, you just have to read one more line (multiple data values are & separated on one line). After sending the response, you have to exit httpd.sh or at least close its output.

How does socat know when the HTTP request ends?

socat knows that the response ended when the data pipe from echo is closed on termination of that process.

Upvotes: 1

Ricardo
Ricardo

Reputation: 490

You cannot read the request line by line. HTTP requests might contain line breaks. You need to read the whole request and parse that.

On GET requests, or Head requests, you only need the headers, so you can read only one line.

For Posts, you need to read either the number of bytes stated in the content-length header, or until you receive an EOF.

Suffices to say, your script becomes more complicated.

Upvotes: 0

Related Questions