Localhost UDP client not receiving packets from UDP proxy

Question

I have a proxy that acts between a Minecraft (Windows 10 Edition) client and a server, the protocol is UDP. The client broadcasts an unconnected ping, then server replies with an unconnected pong, everything with that is okay. The thing is when the server sends the packet to my proxy, my proxy sends it to client but for some reason something happens during that part; either my proxy is not sending the packet or the client is not receiving the packet, but most likely it's the second option.

Edit: I got the unconnected pings and pongs working, now the server shows online on the server list, now the problem is mainly the open connection requests/replies. How I got the the pings and pongs working was I re-encoded the buffers and send them instead of sending them raw.

Here you can see from wireshark, the minecraft client sends an unconnected ping the proxy:

NO. Time.       Source.     Dest.       Proto.  Len. Packet
417 10.452413   10.0.0.248  10.0.0.255  RakNet  75   Unconnected Ping (client -> proxy)
430 10.457000   10.0.0.248  x.x.x.x     RakNet  610  Unconnected Ping (proxy -> server)
431 10.587214   x.x.x.x     10.0.0.248  RakNet  212  Unconnected Pong (server -> proxy -> client)

Now the proxy receive the unconnected pong, send it to the client, and the client doesn't receive it, I can confirm this because on the game the server shows offline and doesn't show any data:

Localhost Server Screenshot

In my code, I first bind the proxy on port 19132 and set the server address I want to communicate with:

var config = NewConfig()
var proxy = Proxy{}

var err error

proxy.UDPConn, err = net.ListenUDP("udp", &net.UDPAddr{IP: net.ParseIP(config.BindAddr), Port: config.BindPort})

if err != nil {
    Panic(err.Error())
    os.Exit(1)
}

Info("Starting proxy on " + config.BindAddr + ":" + strconv.Itoa(config.BindPort))

addrs, err := net.LookupHost(config.ServerAddr)

if err != nil {
    Panic(err.Error())
    os.Exit(1)
}

conn := NewConnection(&proxy)
conn.server.SetAddress(net.UDPAddr{IP: net.ParseIP(addrs[0]), Port: config.ServerPort})

conn.HandleIncomingPackets()

Now my proxy starts receiving and sending packets automatically.

for true {
    buffer := make([]byte, 2048)
    _, addr, err := conn.proxy.UDPConn.ReadFromUDP(buffer)

    if err != nil {
        Alert(err.Error())
        continue
    }

    MessageId := buffer[0]

    Debug("Message Id : " + strconv.Itoa(int(MessageId))) // this is the packet id

    if conn.client.IsConnected() { // client is connected
        if conn.IsServer(*addr) {
            conn.pkHandler.HandleIncomingPacket(buffer, conn.client) // if server send to client
        }else{
            conn.pkHandler.HandleIncomingPacket(buffer, conn.server) // if client send to server
        }
    } else {
        switch MessageId {
        case byte(IdUnconnectedPingOpenConnection):
            conn.handleUnconnectedPing(*addr, buffer) // send this server
            break
        case byte(IdUnconnectedPongOpenConnection):
            conn.handleUnconnectedPong(*addr, buffer) // parse server data and send to client
            break
        case byte(IdOpenConnectionRequest1):
            conn.handleConnectionRequest1(*addr, buffer) // connect client and send to server
            break
        //case byte(IdOpenConnectionReply1):
        //  conn.handleConnectionReply1(*addr, buffer)
        //  break
        }
    }
}

This is the message id log:

[2018-06-10 13:52:12][Log/DEBUG]: Message Id : 1
[2018-06-10 13:52:12][Log/INFO]: Received unconnected ping from client address: 10.0.0.248
[2018-06-10 13:52:12][Log/DEBUG]: Message Id : 28
[2018-06-10 13:52:12][Log/INFO]: Received unconnected pong from server address: x.x.x.x
[2018-06-10 13:52:13][Log/DEBUG]: Message Id : 1
[2018-06-10 13:52:13][Log/INFO]: Received unconnected ping from client address: 10.0.0.248
[2018-06-10 13:52:13][Log/DEBUG]: Message Id : 28
[2018-06-10 13:52:13][Log/INFO]: Received unconnected pong from server address: x.x.x.x

Another way I confirmed the client is not receiving packets is that when in the game I click the server in the server list, the client sends an open connection request 1 (5) and the server replies with open connection reply 1 (6), the client is supposed to receive this and continue with an open connection request 2 (7) then finally the server replies with open connection reply 2 (8), but the client never sends open connection request 2 (7), because it never got the open connection reply 1 (6) from the proxy and hence times out and disconnects, here is a log showing this:

[2018-06-10 11:07:46][Log/DEBUG]: Message Id : 5
[5 0 255 255 0 254 254 254 254 253 253 253 253 18 52 86 120 8]
[2018-06-10 11:07:47][Log/DEBUG]: Message Id : 6
[6 0 255 255 0 254 254 254 254 253 253 253 253 18 52 86 120 36 149 162 237 197 55 226 161 0 8 28]
[2018-06-10 11:07:47][Log/DEBUG]: Message Id : 5
[5 0 255 255 0 254 254 254 254 253 253 253 253 18 52 86 120 8]
[2018-06-10 11:07:47][Log/DEBUG]: Message Id : 6
[6 0 255 255 0 254 254 254 254 253 253 253 253 18 52 86 120 36 149 162 237 197 55 226 161 0 8 28]
[2018-06-10 11:07:47][Log/DEBUG]: Message Id : 1
[1 0 0 0 0 3 3 91 191 0 255 255 0 254 254 254 254 253 253 253 253 18 52 86 120 191 216 14 215 31 123 8 249]

Answer 1

After so much debugging and testing, I figured I just needed to continue sending the datagram buffers, if it came from the server send it to client and vice-versa. If I cancelled the packet from sending I needed to send an ACK. It was all that simple.