CODEWITHSUNDEEP
javamemorynettydirectmemory
nikai
nikai

Reputation: 27

PooledUnsafeDirectByteBuf not found in DirectMemory

I am using Netty 4.1.17-Final.

I wrote the code to send and receive 100 MB random ASCII. Decoder does not read until ByteBuf becomes 100 MB.

@Override
public void decode(ByteBuffer _in, List<Object> _out) {
    if (_in.remaining() == size) {   // size = 100 * 1024 * 1024
        _in.get(new byte[size]);
    }
}

Therefore, Netty buffers 100 MB, but it was not found even by monitoring Direct Memory.

System.out.println(sun.misc.SharedSecrets.getJavaNioAccess().getDirectBufferPool().getMemoryUsed());
ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class).forEach(buf -> {
    System.out.printf("%s: Used=%d, TotalCap=%d%n", buf.getName(), buf.getMemoryUsed(), buf.getTotalCapacity());
});

// Result
// 10
// direct: Used=9, TotalCap=8
// mapped: Used=0, TotalCap=0

Netty used PooledUnsafeDirectByteBuf, but where do you buffer the received data? Or is the direct buffer monitoring method incorrect?

Upvotes: 0

Views: 3697

Answers (2)

nikai
nikai

Reputation: 27

Self-answer, In my code Netty had acquired DirectMemory using com.sun.Unsafe. This area can not be captured with BufferPoolMXBean or SharedSecrets.

I tried to get memory with Unsafe, but it was not output to the monitoring result.

Field unsafeField = Unsafe.class.getDeclaredField("theUnsafe");
unsafeField.setAccessible(true);
Unsafe unsafe = (Unsafe) unsafeField.get(null);
unsafe.allocateMemory(1000);

I found the following request.
extension for tracking off-heap memory use for netty · Issue #475 · Netflix/spectator

Please tell me if there is a way to get the size obtained by Unsafe.

Upvotes: 0

Norman Maurer
Norman Maurer

Reputation: 23567

You can tell Netty to not use Unsafe for the direct buffers and so have it show up in JMX again. Unfortunally this also will have some performance impact.

Just use -Dio.netty.maxDirectMemory=0.

See: https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java#L153

Upvotes: 2

Related Questions