client socket does not receive exactly what the server side socket sends

Question

I have been developing an Android audio chatting program which behaves like a walkie talkie. After a user presses the talk button then the audio recorder starts to record what the user is saying and writes the audio bytes to a remote server through a socket. On the server side, the server socket just sends the audio bytes it received to the other client sockets.

I do not have a good way to control the behavior of these sockets. For example, to identify a client socket belongs which user? The socket does not has any field to carry the additional information other than the data it writes. So in the end, I worked out the solution is to use the same socket which transfer the audio data to transfer something like a username string. And this works well as the android client sends out a username string in cases like a client socket creates connection to server socket successfully.

The disaster happens when I try to send a username string to inform other clients who is talking when the user presses the talk button. Let me give you an example to make this clearer:

1. A user who's name is "user1" presses the talk button to talk.
2. The application sends the string "usr:user1" to the server side.
3. It then starts to send the audio data generated by the audio recorder.

On the server side, the server received the exact "user1" and the following audio data and resend to the other connected clients. But the problem is the client does not seem to be receiving "usr:user1" all of the time.

Here is how I check the received data:

 is = socket.getInputStream();
 byte[] buffer = new byte[minBufSize];
 numOfReceived = is.read(buffer);
 if(numOfReceived!=-1&&numOfReceived!=minBufSize){
     byte[] ub = new byte[numOfReceived];
     for(int i=0;i<numOfReceived;i++){
         ub[i]=buffer[i];
     }
     String usersString = new String(ub, "UTF-8");
     if(usersString.contains("hj:")){
         System.out.println("current:");
         final String userOfTalking=usersString.substring(3,usersString.length());
         runOnUiThread(new Runnable() {
             @Override
             public void run() {
                 whoIsTalking.setText(userOfTalking+" is talking");
                 whoIsTalking.setVisibility(View.VISIBLE);
             }
         });
         continue;
     }

Actually, I have no idea whether the input stream contains audio data or string data. So I tried to use the return of inputstream.read() to find out how many bytes the inputstream read:

• If the return number does not equal to -1 (socket closed) or the buffersize, I set in the outputstream.write, then I assume it a string.

But this is highly unreliable. For example, if I loop the command socket.getoutstream.write(buffer,0,100), then I am supposed to read a buffer 100 length from input stream. But it's not like this. I often got buffers which length are 60, or 40, or any number less than 100.

It's like the outputstream does not send exactly 100 bytes data as it declares. So my string data just mixes with the following audio data. So when the application sends the username when it just connects to the server, the others clients will receive the correct string because there is no following audio data to interfere with it.

Can you guys give me some of your opinions? Is my guessing right? How can I solve this problem? I managed to call Thread.sleep(300) after the application send the username string when the user pressed the talk button to make some room between sending the audio data in case they mix. But it does not work. Any help is much appreciated!

Answer 1

It's been quite a while since I asked this question and I am gonna give my own answer right now. Hopefully its not too late.

Actually @Philip Couling shed some very valuable insights in his answer, it helped me confirmed my guess about the cause of this issue - "the OS is fragmenting the data between packets". Thanks for his contribution again.

The approach to resolve this problem is from one of my friend. He told me that I could create a new socket in the client to connect to the same server socket to transfer some control information in string format to tell the server like who starts to talk,who stopped talking or even to allow people chatting over it. Each socket will send a string to the server to tell what they are doing and who they are belong to in the format like "audio stream: username" or "control info: username". And The server just store them into two arraylist or hashmap respectively. So every time a user presses the button to stream the audio, the corresponding control information string will be sent to the server to tell it the stream is from who and then the server redirects this information to other clients over sockets for controlling. So now we transfer the string data in a dedicated socket other than the one transferring audio stream. As a result, "The Os fragments the data" is no longer a problem because string data is too short to trigger the OS fragmenting them and also because we just send them on specific event, not as continuously as sending the audio stream.

But the new socket also brings a side effect. Because of the network delay, people may find they are still receiving the voice for a while after the application tell them someone stopped talking. The delay could be over 10 seconds in extreme network condition and may lead to strong noise if some one starts to talk during his phone is playing receiving voice.

For fixing this problem, transferring string informing in the audio socket may be the only choice to keep each side in sync. But I think we could insert some empty bytes in between the audio data and string data to make sure the string wont be mixed with other data.(empty bytes should not change the string.) However I have not tried this method yet. I will add the result after I have examined it.

Answer 2

If I've read throug this properly... You send exactly 100 bytes, but the subsiquent read doesn't get 100, it gets less?

There can be a number of reasons for this. One is that you are not calling flush() when you write. If that's the case then you have a bug and you need to put an appropriate flush() call in your sending code.

Alternativly it could be because the OS is fragmenting the data between packets. This is unlikely for small packets (100 bytes) but very likely / necessary for large packets...

You should never rely on ALL your data turning up in a single read... you need to read multiple times to assemble all the data.