Java advanced usage: Memory and Pointer in JNA

Introduction

We know that there are many pointers in the native code, and these pointers are mapped to Pointer in JNA. In addition to Pointer, JNA also provides a more powerful Memory class. This article will discuss the use of Pointer and Memory in JNA.

Pointer

Pointer is a class introduced in JNA to represent pointers in native methods. Recall that what is the pointer in the native method?

The pointer in the native method is actually an address, which is the memory address of the real object. So a peer property is defined in Pointer to store the memory address of the real object:

 protected long peer;

In real time, the constructor of Pointer needs to pass in this peer parameter:

 public Pointer(long peer) {
        this.peer = peer;
    }

Next, let's take a look at how to get a real object from Pointer. Here is an example of a byte array:

 public void read(long offset, byte[] buf, int index, int length) {
        Native.read(this, this.peer, offset, buf, index, length);
    }

In fact, this method calls the Native.read method, let's continue to look at this read method:

 static native void read(Pointer pointer, long baseaddr, long offset, byte[] buf, int index, int length);

You can see that it is a real native method to read a pointer object.

In addition to Byte arrays, Pointer also provides many other types of reading methods.

Read and write again, let's look at how Pointer writes data:

 public void write(long offset, byte[] buf, int index, int length) {
        Native.write(this, this.peer, offset, buf, index, length);
    }

Similarly, the Native.write method is called to write data.

Here the Native.write method is also a native method:

 static native void write(Pointer pointer, long baseaddr, long offset, byte[] buf, int index, int length);

Pointer also provides writing methods for many other types of data.

Of course, there are more direct get* methods:

 public byte getByte(long offset) {
        return Native.getByte(this, this.peer, offset);
    }

Special Pointer: Opaque

In Pointer, there are also two createConstant methods, which are used to create a Pointer that is neither readable nor writable:

 public static final Pointer createConstant(long peer) {
        return new Opaque(peer);
    }

    public static final Pointer createConstant(int peer) {
        return new Opaque((long)peer & 0xFFFFFFFF);
    }

In fact, what is returned is the Opaque class, which inherits from Pointer, but all read or write methods in it will throw UnsupportedOperationException:

 private static class Opaque extends Pointer {
        private Opaque(long peer) { super(peer); }
        @Override
        public Pointer share(long offset, long size) {
            throw new UnsupportedOperationException(MSG);
        }

Memory

Pointer is a basic pointer mapping. If for the memory space allocated by using the native malloc method, in addition to the starting position of the Pointer pointer, we also need to know the size of the allocated space. So a simple Pointer is not enough.

In this case, we need to use Memory.

Memory is a special Pointer that saves the allocated space size. Let's take a look at the definition of Memory and the properties it contains:

 public class Memory extends Pointer {
...
    private static ReferenceQueue<Memory> QUEUE = new ReferenceQueue<Memory>();
    private static LinkedReference HEAD; // the head of the doubly linked list used for instance tracking
    private static final WeakMemoryHolder buffers = new WeakMemoryHolder();
    private final LinkedReference reference; // used to track the instance
    protected long size; // Size of the malloc'ed space
...
}

There are 5 data defined in Memory, we will introduce them one by one.

The first is the most important size. Size represents the size of the memory space in Memory. Let's look at the constructor of Memory:

 public Memory(long size) {
        this.size = size;
        if (size <= 0) {
            throw new IllegalArgumentException("Allocation size must be greater than zero");
        }
        peer = malloc(size);
        if (peer == 0)
            throw new OutOfMemoryError("Cannot allocate " + size + " bytes");

        reference = LinkedReference.track(this);
    }

It can be seen that the memory type data needs to be passed in a size parameter, indicating the size of the space occupied by the Memory. Of course, this size must be greater than 0.

Then call the malloc method of the native method to allocate a memory space, and the returned peer saves the start address of the memory space. If peer==0, the allocation failed.

If the allocation is successful, the current Memory is saved to the LinkedReference to track the current position.

We can see that there are two LinkedReferences in Memory, one is HEAD and the other is reference.

LinkedReference itself is a WeakReference, and the object referenced by weekReference will be recycled as long as garbage collection is performed, regardless of whether there is insufficient memory.

 private static class LinkedReference extends WeakReference<Memory>

Let's take a look at the constructor of LinkedReference:

 private LinkedReference(Memory referent) {
            super(referent, QUEUE);
        }

This QUEUE is the ReferenceQueue, which represents the list of objects to be recycled by the GC.

We see that the constructor of Memory calls in addition to setting the size:

 reference = LinkedReference.track(this);

Take a closer look at the LinkedReference.track method:

 static LinkedReference track(Memory instance) {
            // use a different lock here to allow the finialzier to unlink elements too
            synchronized (QUEUE) {
                LinkedReference stale;

                // handle stale references here to avoid GC overheating when memory is limited
                while ((stale = (LinkedReference) QUEUE.poll()) != null) {
                    stale.unlink();
                }
            }

            // keep object allocation outside the syncronized block
            LinkedReference entry = new LinkedReference(instance);

            synchronized (LinkedReference.class) {
                if (HEAD != null) {
                    entry.next = HEAD;
                    HEAD = HEAD.prev = entry;
                } else {
                    HEAD = entry;
                }
            }

            return entry;
        }

The meaning of this method is to first take out those Memory objects that are ready to be garbage collected from QUEUE, and then unlink them from LinkedReference. Finally, the newly created object is added to the LinkedReference.

Because QUEUE and HEAD in Memory are class variables, this LinkedReference saves all Memory objects in the JVM.

Finally, the corresponding read and write methods are also provided in Memory, but the methods in Memory are different from Pointer. The methods in Memory have an additional boundsCheck, as shown below:

 public void read(long bOff, byte[] buf, int index, int length) {
        boundsCheck(bOff, length * 1L);
        super.read(bOff, buf, index, length);
    }

    public void write(long bOff, byte[] buf, int index, int length) {
        boundsCheck(bOff, length * 1L);
        super.write(bOff, buf, index, length);
    }

Why is there boundsCheck? This is because Memory is different from Pointer, and there is a size attribute in Memory, which is used to store the allocated memory size. The use of boundsCheck is to determine whether the accessed address is out of bounds to ensure the security of the program.

Summarize

Pointer and Memory are advanced functions in JNA. If you want to map with the native alloc method, you should consider using them.

This article has been included in http://www.flydean.com/06-jna-memory/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!