LWIP ECHO SERVER: How to increase the buffer size in itoa function?

Question

I am working with Xilinx Ethernetlite (LWIP) design. I am able to transfer the data from KC board to PC (Hercules) through Ethernet only if buf =32. But my actual buffer size is 1024. How to increase the buffer size from 32 to 1024

I am unable to make sure whether the mistake is in code or in hercules. To read the values (integer) in hercules i am doing this function.

initially, From hercules i will send the Hello command to Board,then board with accept that request. After that, board will output the data (integer values) to Hercules.

C code for itoa

char* itoa(int val, int base)
{
static char buf[32] = {0};          //buf size 
int i = 30;

for(; val && i ; --i, val /= base)
buf[i] = "0123456789abcdef"[val % base];
return &buf[i+1];
}

Modified code

    #define  DAQ_FIFO_DEPTH  128

  int transfer_data() 
  {
  return 0;
  }

  err_t tcp_write_u32_string(struct tcp_pcb *pcb, unsigned char   prefix, u32_t value)
   {
    unsigned char  buf[11]; /* enough room for prefix and value. */
    err_t          result;
    u16_t          len;
    unsigned char *p = buf + sizeof buf;
  do {
    /* ASCII encoding: '0' = 48, '1' = 49, ..., '9' = 57. */
    *(--p) = 48 + (value % 10u);
    value /= 10;
     } while (value);
     if (prefix)
    *(--p) = prefix;
 len = buf + sizeof buf - p;
  if (tcp_sndbuf(pcb) < len) 
     {
    result = tcp_output(pcb);
    if (result != ERR_OK)
        return result;
    }
 return tcp_write(pcb, p, len, TCP_WRITE_FLAG_COPY | TCP_WRITE_FLAG_MORE);
  }

   err_t send_list(struct tcp_pcb *pcb, const u32_t data[], u16_t len)
  {
  static const char  newline[2] = { 13, 10 }; /* ASCII \r\n */
  err_t              result;

    if (len > 0) {
     u16_t  i;


      result = tcp_write_u32_string(pcb, 0, data[0]);
      if (result != ERR_OK)
        return result;
    for (i = 1; i < len; i++) 
   {
        /* ASCII comma is code 44. (Use 32 for space, or 9 for tab.) */
        result = tcp_write_u32_string(pcb, 44, data[i]);
        if (result != ERR_OK)
            return result;
     }
   }
   result = tcp_write(pcb, newline, 2, 0);
    if (result)
    return result; 
   return tcp_output(pcb);
  }
 int application_connection(void *arg, struct tcp_pcb *conn, err_t err)
 {
  struct netif *netif = arg; /* Because of tcp_arg(, netif). */
  u32_t         data[DAQ_FIFO_DEPTH];
  u32_t         i, n;
 if (err != ERR_OK) {
    tcp_abort(conn);
    return ERR_ABRT;
   }
  err = daq_setup();
  if (err != ERR_OK) 
  {
    tcp_abort(conn);
    return ERR_ABRT;
 }
 while (1) 
    {
    xemacif_input(netif);
    tcp_tmr();
    tcp_output(conn);
    n = daq_acquire(data, DAQ_FIFO_DEPTH);
    if (n > DAQ_FIFO_DEPTH)
        break;
    if (tcp_write(conn, data, n * sizeof data[0], TCP_WRITE_FLAG_COPY) != ERR_OK)
        break;
     }
// daq_close();

/* Close the TCP connection. */
    if (tcp_close(conn) == ERR_OK)
     return ERR_OK;

/* Close failed. Abort it, then. */
    tcp_abort(conn);
    return ERR_ABRT;
    }
  int application_main(struct netif *netif, unsigned int port)
  {
   struct tcp_pcb *pcb;
   err_t           err;
   pcb = tcp_new();
   if (!pcb) {
    /* Out of memory error */
    return -1;
      }
    err = tcp_bind(pcb, IP_ADDR_ANY, port);
    if (err != ERR_OK) {
    /* TCP error */
    return -1;
     }
   pcb = tcp_listen_with_backlog(pcb, 1);
  if (!pcb) {
    /* Out of memory. */
    return -1;
    }
  tcp_arg(pcb, netif); 
  tcp_accept(pcb, application_connection);
  while (1)
  xemacif_input(netif);
  }

Hercules output enter image description here

Answer 1

So, this is a continuation from the discussion at Xilinx forums?

The itoa() function converts an unsigned integer (stored in an int) into the first 30 or so characters in the buffer buf.

The recv_callback() function makes little to no sense.

The call to aurora_rx_main() is documented as a "FUNCTION CALL", which is rather less than helpful (because we have no idea what it does), and even its return value is completely ignored.

The first for loop dumps the contents of the first 100 u32's in DestinationBuffer[] for debugging purposes, so that code is unrelated to the task at hand. However, we don't know who or what filled DestinationBuffer. It might or might not have been filled by the aurora_rx_main() call; we're not told either way.

(The tcp_*() functions seem to follow the API described in lwIP Wiki at Wikia.)

If the p parameter is NULL, then tcp_close(tcpb) is called, followed by a tcp_recv(tcpb, NULL) call. This makes the least sense of all: why try to receive anything (and why the NULL parameter) after a close?

The next part is similarly baffling. It looks like the if test checks if the TCP send buffer is over 1024 bytes in size. If not, the p buffer is freed. Otherwise, the for loop tries to convert each u32 in DestinationBuffer to a string, write that string to the TCP buffer; however, rather than the proper api flags, it uses the constant 1, and does not even check if appending to the TCP send buffer works.

In summary, this looks like a pile of copy-pasted code that does nothing sensible. Increasing the buffer size in itoa function is not only unnecessary (an u32, even when converted to an int, will always fit within 12 characters (excluding either the minus sign, or the nul byte at end, so make that 13 characters total), but completely unrelated to the problem it is supposed to fix.

The root problem is that the code is horrible. Modifying it is like putting car body filler over a piece of old chewing gum, in an effort to "fix" it. The proper fix is to rip out that junk code altogether, and use something better instead.

Edit: The OP states that they're a new programmer, so the comments above should be taken as a direct, honest opinion of the code shown, and not about OP themselves. Let's see if we can help OP produce better code.

---

First, the itoa() function shown is silly. Assuming the intent is really to send back the u32_ts in the DestinationBuffer as decimal strings, it is much better to implement a helper function for doing the conversion. Since the value is to be preceded with a comma (or some other separator), we can add that trivially as well. Since it will be sent using tcp_write(), we'll combine the functionality:

err_t tcp_write_u32_string(struct tcp_pcb *pcb,
                           unsigned char   prefix, /* 0 for none */
                           u32_t           value)
{
    /* Because 0 <= u32_t <= 4294967295, the value itself is at most 10 digits long. */
    unsigned char  buf[11]; /* enough room for prefix and value. */
    err_t          result;
    u16_t          len;
    unsigned char *p = buf + sizeof buf;

    /* Construct the value first, from right to left. */
    do {
        /* ASCII encoding: '0' = 48, '1' = 49, ..., '9' = 57. */
        *(--p) = 48 + (value % 10u);
        value /= 10;
    } while (value);

    /* Prepend the prefix, if any. */
    if (prefix)
        *(--p) = prefix;

    /* Calculate the length of this part. */
    len = buf + sizeof buf - p;

    /* If the TCP buffer does not have enough free space, flush it. */
    if (tcp_sendbuf(pcb) < len) {
        result = tcp_output(pcb);
        if (result != ERR_OK)
            return result;
    }

    /* Append the buffer to the TCP send buffer.
       We also assume the packet is not done yet. */
    return tcp_write(pcb, p, len, TCP_WRITE_FLAG_COPY | TCP_WRITE_FLAG_MORE);
}

so that to send num u32_ts from a specified array as decimal strings, with a newline at end, you could use

err_t send_list(struct tcp_pcb *pcb,
                const u32_t data[],
                u16_t len)
{
    static const char  newline[2] = { 13, 10 }; /* ASCII \r\n */
    err_t              result;

    if (len > 0) {
        u16_t  i;

        /* The first number has no prefix. */
        result = tcp_write_u32_string(pcb, 0, data[0]);
        if (result != ERR_OK)
            return result;

        /* The following numbers have a comma prefix. */
        for (i = 1; i < len; i++) {
            /* ASCII comma is code 44. (Use 32 for space, or 9 for tab.) */
            result = tcp_write_u32_string(pcb, 44, data[i]);
            if (result != ERR_OK)
                return result;
        }
    }

    /* We add a final newline.
       Note that this one can be referenced,
       and it does complete what we wanted to send thus far. */
    result = tcp_write(pcb, newline, 2, 0);
    if (result)
        return result;

    /* and flush the buffer, so the packet gets sent right now. */
    return tcp_output(pcb);
}

Now, I haven't written C for Xilinx or used the lwIP stack at all, so the above code is written blind. Yet, I'm pretty confident it works (barring any typos or thinkos; if you find any, report them in a comment, and I'll verify and fix).

The two buffers (buf and newline) are declared static, so that although they're only visible within their respective functions, their value is valid in the global scope.

Because TCP is a stream protocol, it is not necessary to fit each response to a single packet. Other than the 11-character (for each number and its prefixing character) and the 2-character (newline) buffers, the only large buffer you need is the TCP send buffer (the maximum transmission unit or maximum segment size as I'm not sure how lwIP uses the buffer internally), typically between 536 and 1518 bytes.

The two above functions try to split packets between numbers, but that's just because it's easier than to try and fill each packet exactly. If the next (comma and) value fit in the buffer, then it is added to the buffer; otherwise the buffer is flushed first, and then the next (comma and) value added to the buffer.

From the recipient side, you should obtain a nice, readable stream using e.g. netcat. (I have no idea if Hercules is an application, or just the name of your local machine.) Because TCP is a stream protocol, the recipient cannot (reliably) detect where the packet boundaries were (unlike in, say, UDP datagrams). In practice, a TCP connection is just two streams of data, each going one way, and the split into packets is just a protocol detail application programmers don't need to worry about. For lwIP, because it is such a low-level library, a little bit of care need to be taken, but as one can see from the above code, it's really not that much.

---

In a comment, OP explained that they are not very experienced, and that the overall intent is to have the device accept a TCP connection, and stream data (samples acquired by a separate acquisition board) as unsigned 32-bit integers via the connection.

Because I would love to have one of those FPGA boards (I have several tasks I could see if I could offload to an FPGA), but no resources to get one, I shall try to outline the entire application here. Do note that the only information I have to go on, is the 2018 version of Xilinx OS and Libraries Document Collection (UG643) (PDF). It looks like OP wants to use the raw API, for high performance.

Converting the samples to text is silly, especially if high performance is desired. We should just use raw binary, and whatever endianness the KC705 uses. (I didn't see it at a quick glance from the documentation, but I suspect it is little-endian).

According to the documentation, the raw API main() is something similar to following:

int main(void)
{
    /* MAC address. Use an unique one. */
    unsigned char  mac[6] = { 0x00, 0x0A, 0x35, 0x00, 0x01, 0x02 };

    struct netif  *netif = NULL;
    ip_addr_t      ipaddr, netmask, gateway;

    /* Define IP address, netmask, and gateway. */
    IP4_ADDR(&ipaddr,  192, 168,   1,  1);
    IP4_ADDR(&netmask, 255, 255, 255,  0);
    IP4_ADDR(&gateway,   0,   0,   0,  0);

    /* Initialize lwIP networking stack. */
    lwip_init();

    /* Add this networking interface, and make it the default one */
    if (!xemac_add(netif, &ipaddr, &netmask, &gateway, mac, EMAC_BASEADDR)) {
        printf("Error adding network interface\n\r");
        return -1;
    }
    netif_set_default(netif);

    platform_enable_interrupts();

    /* Bring the network interface up (activate it) */
    netif_set_up(netif);

    /* Our application listens on port 7. */
    return application_main(netif, 7);
}

In the documentation examples, rather than return application_main(netif);, you'll see a call to start_application(), and then an infinite loop that regularly calls xemacif_input(netif) instead. It just means that out application_main() must call xemacif_input(netif) regularly, to be able to receive data. (The lwIP documentation says that we should also call sys_check_timeouts() or tcp_tmr() at regular intervals.)

Note that I've omitted error reporting printfs, and that rather than recovering from errors gracefully, this will just return (from main()); I'm not certain whether this causes the KC705 to restart or what.

int application_main(struct netif *netif, unsigned int port)
{
    struct tcp_pcb *pcb;
    err_t           err;

    pcb = tcp_new();
    if (!pcb) {
        /* Out of memory error */
        return -1;
    }

    /* Listen for incoming connections on the specified port. */
    err = tcp_bind(pcb, IP_ADDR_ANY, port);
    if (err != ERR_OK) {
        /* TCP error */
        return -1;
    }
    pcb = tcp_listen_with_backlog(pcb, 1);
    if (!pcb) {
        /* Out of memory. */
        return -1;
    }

    /* The accept callback function gets the network interface
       structure as the extra parameter. */
    tcp_arg(pcb, netif);

    /* For each incoming connection, call application_connection(). */
    tcp_accept(pcb, application_connection);

    /* In the mean time, process incoming data. */
    while (1)
        xemacif_input(netif);
}

For each TCP connection to the port, we get a call to application_connection(). This is the function that sets up the data acquisition board, and transfers the data for as long as the recipient wants it.

/* How many DAQ samples to process in each batch.
 * Should be around the DAQ FIFO depth or so, I think. */
#define  DAQ_FIFO_DEPTH  128

err_t application_connection(void *arg, struct tcp_pcb *conn, err_t err)
{
    struct netif *netif = arg; /* Because of tcp_arg(, netif). */
    u32_t         data[DAQ_FIFO_DEPTH];
    u32_t         i, n;

    /* Drop the connection if there was an error. */
    if (err != ERR_OK) {
        tcp_abort(conn);
        return ERR_ABRT;
    }

    /* Setup the data aquisition. */
    err = daq_setup();
    if (err != ERR_OK) {
        tcp_abort(conn);
        return ERR_ABRT;
    }

    /* Data acquisition to TCP loop. */
    while (1) {

        /* Keep the networking stack running. */
        xemacif_input(netif);
        tcp_tmr();

        /* Tell the networking stack to output what it can. */
        tcp_output(conn);

        /* Acquire up to DAQ_FIFO_DEPTH samples. */
        n = daq_acquire(data, DAQ_FIFO_DEPTH);
        if (n > DAQ_FIFO_DEPTH)
            break;

        /* Write data as-is to the tcp buffer. */
        if (tcp_write(conn, data, n * sizeof data[0], TCP_WRITE_FLAG_COPY) != ERR_OK)
            break;
    }

    /* Stop data acquisition. */
    daq_close();

    /* Close the TCP connection. */
    if (tcp_close(conn) == ERR_OK)
        return ERR_OK;

    /* Close failed. Abort it, then. */
    tcp_abort(conn);
    return ERR_ABRT;
}

There are three more functions to implement: daq_setup(), which should setup the data acquisition and FIFOs; daq_acquire(u32_t *data, u32_t count) that stores up to count samples to data[], and returns the actual number of samples stored -- it would be best if it just drained the FIFO, rather than waited for new samples to arrive --, and finally daq_close(), that stops the data acquisition.

I believe they should be something like this:

XLlFifo         daq_fifo;

err_t daq_setup(void)
{
    XLlFifo_Config *config = NULL;

    config = XLlFifo_LookupConfig(DAQ_FIFO_ID);
    if (!config)
        return ERR_RTE;

    if (XLlFifo_CfgInitialize(&daq_fifo, config, config->BaseAddress) != XST_SUCCESS)
        return ERR_RTE;
}

u32_t daq_acquire(u32_t *data, u32_t max)
{
    u32_t len, have;

    have = XLlFifo_iRxGetLen(&daq_fifo);
    if (have < 1)
        return 0;
    else
    if (have < max)
        max = have;

    for (len = 0; len < max; len++)
        data[len] = XLlFifo_RxGetWork(&daq_fifo);

    return len;
}

err_t daq_close(void)
{
    /* How to stop the FIFO? Do we need to? */
}

That's about it.