The problem of sticky packets and half packets is a common problem in data transmission. The so-called sticky packet problem means that when data is transmitted, part of the data of another message is read in one message. This phenomenon is called sticky packets. For example, if two messages are sent, namely "ABC" and "DEF", then the receiver should also receive two messages "ABC" and "DEF" under normal circumstances, but the receiver receives "ABCD". A situation like this is called a sticky package, as shown in the following figure:

The half-packet problem means that the receiver only receives part of the data, but not the complete data, which is called a half-packet. For example, if a message is sent "ABC", but the receiver receives two pieces of information, "AB" and "C", this situation is called a half packet, as shown in the following figure:

PS: In most cases, we regard the sticky package problem and the half-package problem as the same problem, so the following will use the "sticky package" problem to replace the "sticky package" and "half-package" problems.

1. Why is there a sticky bag problem?

The packet sticking problem occurs in TCP/IP protocol. Because TCP is a connection oriented transmission protocol that transmits data in the form of "streams", and "streams" have no clear start and end boundaries, There is a sticky package problem

2. Code demonstration of sticky package problem

Next, we will use code to demonstrate the sticky package and half-package problem. For the intuitiveness of the demonstration, I will set two roles:

• The server is used to receive messages;
• The client is used to send a fixed message.

Then observe the sticky packet problem by printing the information received on the server side.
The server-side code is implemented as follows:

/**
 * Server side (only responsible for receiving messages)
 */
class ServSocket {
    // length of byte array
    private static final int BYTE_LENGTH = 20;  
    public static void main(String[] args) throws IOException {
        // Create Socket server
        ServerSocket serverSocket = new ServerSocket(8888);
        // Get client connection
        Socket clientSocket = serverSocket.accept();
        // Get the stream object sent by the client
        try (InputStream inputStream = clientSocket.getInputStream()) {
            while (true) {
                // Loop to get the information sent by the client
                byte[] bytes = new byte[BYTE_LENGTH];
                // Read the information sent by the client
                int count = inputStream.read(bytes, 0, BYTE_LENGTH);
                if (count > 0) {
                    // A valid message is successfully received and printed
                    System.out.println("The information received from the client is:" + new String(bytes));
                }
                count = 0;
            }
        }
    }
}

The client implementation code is as follows:

/**
 * Client (only responsible for sending messages)
 */
static class ClientSocket {
    public static void main(String[] args) throws IOException {
        // Create a Socket client and try to connect to the server side
        Socket socket = new Socket("127.0.0.1", 8888);
        // Sent message content
        final String message = "Hi,Java."; 
        // Send messages using output streams
        try (OutputStream outputStream = socket.getOutputStream()) {
            // Send 10 messages to the server
            for (int i = 0; i < 10; i++) {
                // Send a message
                outputStream.write(message.getBytes());
            }
        }
    }
}

The execution result of the above program is shown in the following figure:

From the above results, we can see that the Service Terminal has a packet sticking problem because the client has sent 10 fixed "Hi,Java." messages. The correct result is that the Service Terminal has also received 10 fixed messages "Hi,Java." is correct, but the actual execution result is not the case

3. Solutions

There are 3 common solutions to the sticky package problem:

1. The sender and the receiver fix the size of the data to be sent. When the character length is not enough, use null characters to make up for it. After the fixed size is obtained, the specific boundary of each message can be known, so that there is no problem of sticking packets;

2. Encapsulate a layer of user-defined data protocol on the basis of TCP protocol. In the user-defined data protocol, the data header (the size of the stored data) and the specific content of the data are included. After the server gets the data, it can know The specific length of the data, there is no problem of sticking packets by parsing the data header;

3. End with a special character, such as "\n", so that we know the specific boundary of the data, thus avoiding the sticky package problem (recommended solution).

   Solution 1: Fixed data size

   To receive and send fixed-size data, the server-side implementation code is as follows:

   /**
   * Server side, improved version one (only responsible for receiving messages)
   */
   static class ServSocketV1 {
    private static final int BYTE_LENGTH = 1024;  // Byte array length (for receiving messages)
    public static void main(String[] args) throws IOException {
        ServerSocket serverSocket = new ServerSocket(9091);
        // get connected
        Socket clientSocket = serverSocket.accept();
        try (InputStream inputStream = clientSocket.getInputStream()) {
            while (true) {
                byte[] bytes = new byte[BYTE_LENGTH];
                // Read the information sent by the client
                int count = inputStream.read(bytes, 0, BYTE_LENGTH);
                if (count > 0) {
                    // print message received
                    System.out.println("The information received from the client is:" + new String(bytes).trim());
                }
                count = 0;
            }
        }
    }
   }
   

   The implementation code of the client is as follows:

   /**
   * Client, improved version one (only responsible for receiving messages)
   */
   static class ClientSocketV1 {
    private static final int BYTE_LENGTH = 1024;  // length in bytes
    public static void main(String[] args) throws IOException {
        Socket socket = new Socket("127.0.0.1", 9091);
        final String message = "Hi,Java."; // Send a message
        try (OutputStream outputStream = socket.getOutputStream()) {
            // Assemble data into fixed-length byte array
            byte[] bytes = new byte[BYTE_LENGTH];
            int idx = 0;
            for (byte b : message.getBytes()) {
                bytes[idx] = b;
                idx++;
            }
            // Send 10 messages to the server
            for (int i = 0; i < 10; i++) {
                outputStream.write(bytes, 0, BYTE_LENGTH);
            }
        }
    }
   }
   

   The execution result of the above code is shown in the following figure:
   

   Analysis of advantages and disadvantages

   It can be seen from the above code that although this method can solve the problem of sticky packets, this fixed data size transmission method will use null characters to fill in when the amount of data is relatively small, so it will increase the burden of network transmission, so Not an ideal solution.

   Solution 2: Custom Request Protocol

   The implementation idea of ​​this solution is to encapsulate the requested data into two parts: the message header (the size of the data sent) + the message body (the specific data sent). Its format is shown in the following figure:
   
   The implementation of this solution is divided into the following 3 parts:

4. Write a message wrapper class

5. write client

6. write server side

Next, we will implement them one by one.

① Message encapsulation class

The encapsulation class of the message provides two methods: one is to convert the message into message header + message body, and the other is to read the message header. The specific implementation code is as follows:

/**
 * message encapsulation class
 */
class SocketPacket {
    // Length of message header storage (8 bytes)
    static final int HEAD_SIZE = 8;

    /**
     * Encapsulate the protocol as: protocol header + protocol body
     * @param context Message body (String type)
     * @return byte[]
     */
    public byte[] toBytes(String context) {
        // Protocol body byte array
        byte[] bodyByte = context.getBytes();
        int bodyByteLength = bodyByte.length;
        // final package object
        byte[] result = new byte[HEAD_SIZE + bodyByteLength];
        // Convert int to byte[] with the help of NumberFormat
        NumberFormat numberFormat = NumberFormat.getNumberInstance();
        numberFormat.setMinimumIntegerDigits(HEAD_SIZE);
        numberFormat.setGroupingUsed(false);
        // protocol header byte array
        byte[] headByte = numberFormat.format(bodyByteLength).getBytes();
        // encapsulation protocol header
        System.arraycopy(headByte, 0, result, 0, HEAD_SIZE);
        // encapsulation protocol body
        System.arraycopy(bodyByte, 0, result, HEAD_SIZE, bodyByteLength);
        return result;
    }

    /**
     * Get the content of the message header (that is, the length of the message body)
     * @param inputStream
     * @return
     */
    public int getHeader(InputStream inputStream) throws IOException {
        int result = 0;
        byte[] bytes = new byte[HEAD_SIZE];
        inputStream.read(bytes, 0, HEAD_SIZE);
        // Get the length in bytes of the message body
        result = Integer.valueOf(new String(bytes));
        return result;
    }
}

② Client

In the client, we add a group of messages to be sent, and randomly send a message to the server. The implementation code is as follows:

/**
 * client
 */
class MySocketClient {
    public static void main(String[] args) throws IOException {
        // Start Socket and try to connect to server
        Socket socket = new Socket("127.0.0.1", 9093);
        // Send a collection of messages (send a random message)
        final String[] message = {"Hi,Java.", "Hi,SQL~", "Pay attention to the public account|Java Chinese community."};
        // Create a protocol wrapper object
        SocketPacket socketPacket = new SocketPacket();
        try (OutputStream outputStream = socket.getOutputStream()) {
            // Send 10 messages to the server
            for (int i = 0; i < 10; i++) {
                // send a random message
                String msg = message[new Random().nextInt(message.length)];
                // Encapsulate the content as: protocol header + protocol body
                byte[] bytes = socketPacket.toBytes(msg);
                // Send a message
                outputStream.write(bytes, 0, bytes.length);
                outputStream.flush();
            }
        }
    }
}

③ Server side

The server side uses the thread pool to process the business request of each client. The implementation code is as follows:

/**
 * Server side
 */
class MySocketServer {
    public static void main(String[] args) throws IOException {
        // Create Socket Server
        ServerSocket serverSocket = new ServerSocket(9093);
        // Get client connection
        Socket clientSocket = serverSocket.accept();
        // Handle more clients with thread pools
        ThreadPoolExecutor threadPool = new ThreadPoolExecutor(100, 150, 100,
                TimeUnit.SECONDS, new LinkedBlockingQueue<>(1000));
        threadPool.submit(() -> {
            // client message handling
            processMessage(clientSocket);
        });
    }
    /**
     * client message handling
     * @param clientSocket
     */
    private static void processMessage(Socket clientSocket) {
        // Socket wrapper object
        SocketPacket socketPacket = new SocketPacket();
        // Get the message object sent by the client
        try (InputStream inputStream = clientSocket.getInputStream()) {
            while (true) {
                // Get the message header (that is, the length of the message body)
                int bodyLength = socketPacket.getHeader(inputStream);
                // message body byte array
                byte[] bodyByte = new byte[bodyLength];
                // The actual number of bytes read each time
                int readCount = 0;
                // Message body assignment subscript
                int bodyIndex = 0;
                // Loop through the length defined in the message header
                while (bodyIndex <= (bodyLength - 1) &&
                        (readCount = inputStream.read(bodyByte, bodyIndex, bodyLength)) != -1) {
                    bodyIndex += readCount;
                }
                bodyIndex = 0;
                // Successfully received the client's message and printed it
                System.out.println("information received from the client:" + new String(bodyByte));
            }
        } catch (IOException ioException) {
            System.out.println(ioException.getMessage());
        }
    }
}

The execution result of the above program is as follows:

It can be seen from the above results that the message communication is normal, and there is no sticky packet problem in the interaction between the client and the server.

Analysis of advantages and disadvantages

Although this solution can solve the sticky packet problem, the design and code implementation complexity of the message is relatively high, so it is not an ideal solution.

Solution 3: End with special characters

The boundary of the stream can be known by ending with a special character. Its specific implementation is to use the BufferedReader and BufferedWriter that come with Java, that is, the input character stream and the output character stream with a buffer, and end with n when writing, and use readLine to read data by line when reading, so that the boundary of the stream can be known, thus solving the problem of package sticking
The server-side implementation code is as follows:

/**
 * Server side, improved version three (only responsible for receiving messages)
 */
static class ServSocketV3 {
    public static void main(String[] args) throws IOException {
        // Create Socket Server
        ServerSocket serverSocket = new ServerSocket(9092);
        // Get client connection
        Socket clientSocket = serverSocket.accept();
        // Handle more clients with thread pools
        ThreadPoolExecutor threadPool = new ThreadPoolExecutor(100, 150, 100,
                TimeUnit.SECONDS, new LinkedBlockingQueue<>(1000));
        threadPool.submit(() -> {
            // message processing
            processMessage(clientSocket);
        });
    }
    /**
     * message processing
     * @param clientSocket
     */
    private static void processMessage(Socket clientSocket) {
        // Get the message flow object sent by the client
        try (BufferedReader bufferedReader = new BufferedReader(
                new InputStreamReader(clientSocket.getInputStream()))) {
            while (true) {
                // Read messages sent by the client by line
                String msg = bufferedReader.readLine();
                if (msg != null) {
                    // Successfully received the client's message and printed it
                    System.out.println("information received from the client:" + msg);
                }
            }
        } catch (IOException ioException) {
            ioException.printStackTrace();
        }
    }
}

PS: The above code uses a thread pool to solve the problem of multiple clients accessing the server side at the same time, thus realizing a one-to-many server response.

The implementation code of the client is as follows:

/**
 * Client, improved version three (only responsible for sending messages)
 */
static class ClientSocketV3 {
    public static void main(String[] args) throws IOException {
        // Start Socket and try to connect to server
        Socket socket = new Socket("127.0.0.1", 9092);
        final String message = "Hi,Java."; // Send a message
        try (BufferedWriter bufferedWriter = new BufferedWriter(
                new OutputStreamWriter(socket.getOutputStream()))) {
            // Send 10 messages to the server
            for (int i = 0; i < 10; i++) {
                // Note: The trailing \n cannot be omitted, it means to write by line
                bufferedWriter.write(message + "\n");
                // Flush the buffer (this step cannot be omitted)
                bufferedWriter.flush();
            }
        }
    }
}

The execution result of the above code is shown in the following figure:

Analysis of advantages and disadvantages

The biggest advantage of using special symbols as a sticky package solution is that it is simple to implement, but there are certain limitations. For example, if a terminator appears in the middle of a message, it will cause a half-package problem, so if it is a complex string, you must correct it. The content is encoded and decoded to ensure the correctness of the terminator.

Summarize

The problem of sticky packets and half packets is a common problem in data transmission. There are many solutions. The more common solutions are: setting a fixed data transmission size, customizing the encapsulation of the request protocol, and adding transmission data to the request header. length, use special symbols as terminators, etc.

It is up to oneself to judge right and wrong, to listen to others, and to count the gains and losses.

Official Account: Analysis of Java Interview Questions

Interview collection: https://gitee.com/mydb/interview