How to setup a Message Head on a network level in Perl Socket programming

Question

I have a requirement about sending socket data over the TCP/IP Protocol to a different party/server, and one of the core requirements is setting up the message length in the message head. Can we set such an operation in Perl Socket Programming where both the header and body can be set?

I have seen a reference in the PHP language where the message length can be sent as described in the examples - https://www.php.net/manual/en/sockets.examples.php, and it would be helpful if we have something in that front in Perl.

I verified with Socket, IO::Socket::INET modules but hardly could find anything related to this point

Answer 1

Understanding your requirement

With a streaming protocol like TCP, information needs to be inserted into the stream which allows the reader to split the stream into individual messages.

It could be something as simple as inserting line feeds between messages. That is the approach used by the examples to which you link. Contrary to your claim, the sender does not provide the length of the messages in those examples.^[1]

The problem with that approach is that line feeds can't appear as part of the message. One could introduce an escaping mechanism, but this slows down reading.

Another very common approach is to send a length prefix. This refers to sending the length of the message in some kind of header before the message. Reading a message from the stream involves reading the header first, determining the size of the message, then reading the rest of the message.

This is the approach you have chosen to take.

---

Sending and receiving a length-prefixed message

The following senders use this technique:

print $fh pack( "N", length( $msg ) ) . $msg;

print $fh pack( "N/a", $msg );

These identical snippets precede the message with its length (as a 32-bit BE integer). The reader would start by reading the length. Armed with that information, it knows how much it needs to read to obtain the full message. The corresponding reader might therefore look like the following:^[2]

my $buf = '';
while ( 1 ) {
   my $n = read_uint32be( $sock, $buf );
   die $! if !defined( $n );

   my $msg = read_exactly( $sock, $buf, $n );
   die $! if !defined( $msg );

   ...
}

---

1. And the readers are buggy, relying on the completely bogus assumption that socket_read will return exactly one line.

2. With the help of this:

   use Errno qw( ENODATA );
   
   use constant CHUNCK_SIZE => 64 * 1024;
   
   # Returns `undef` and sets `$!` to `ENODATA` on eof.
   # Returns `undef` and sets `$!` on error.
   sub read_exactly {
      my $fh      =  shift;
      my $buf_ref = \shift;
      my $n       =  shift;
   
      $$buf_ref //= '';
   
      my $to_read = $n - length( $$buf_ref );
      while ( $to_read > 0 ) {
         my $rv = sysread( $fh, $$buf_ref, CHUNK_SIZE, length( $$buf_ref ) );
         if ( !$rv ) { 
            $! = ENODATA if defined( $rv );  # EOF
            return undef;
         }
   
         $to_read -= $rv;
      }
   
      return substr( $$buf_red, 0, $n );
   }
   
   # Returns `undef` and sets `$!` to `ENODATA` on eof.
   # Returns `undef` and sets `$!` on error.
   sub read_uint32be {
      my $packed = read_exactly( $_[0], $_[1], 4 );
      return undef if !defined( $packed );
      return unpack( "N", $packed );
   }