28

I am writing a very simple RMI server, and I am seeing intermittent java.rmi.NoSuchObjectExceptions in the unit tests.

I have a string of remote method calls on the same object, and while the first few go through, the later ones will sometimes fail. I am not doing anything to unregister the server object in between.

These error do not appear always, and if I put in breakpoints they tend to not appear. Are those Heisenbugs, whose race conditions dissolve when looking at them through the slowed down execution of the debugger? There is no multi-threading going on in my test or server code (though maybe inside of the RMI stack?).

I am running this on Mac OS X 10.5 (Java 1.5) through Eclipse's JUnit plugin, and the RMI server and client are both in the same JVM.

What can cause these exceptions?

Greg Mattes
  • 30,462
  • 13
  • 66
  • 103
Thilo
  • 241,635
  • 91
  • 474
  • 626

7 Answers7

68

Keep a strong reference to the object that implements the java.rmi.Remote interface so that it remains reachable, i.e. ineligible for garbage collection.

Below is a short program that demonstrates a java.rmi.NoSuchObjectException. The script is self-contained, creating an RMI registry as well as a "client" and a "server" in a single JVM.

Simply copy this code and save it in a file named RMITest.java. Compile and invoke with your choice of command line arguments:

  • -gc (default) Explicitly instruct the JVM to make "a best effort" to run the garbage collector after the server is started, but before the client connects to the server. This will likely cause the Remote object to be reclaimed by the garbage collector if the strong reference to the Remote object is released. A java.rmi.NoSuchObjectException is observed when the client connects after the Remote object is reclaimed.
  • -nogc Do not explicitly request garbage collection. This will likely cause the Remote object to remain accessible by the client regardless of whether a strong reference is held or released unless there is a sufficient delay between the server start and the client call such that the system "naturally" invokes the garbage collector and reclaims the Remote object.
  • -hold Retain a strong reference to the Remote object. In this case, a class variable refers to the Remote object.
  • -release (default) A strong reference to the Remote object will be released. In this case, a method variable refers to the Remote object. After the method returns, the strong reference is lost.
  • -delay<S> The number of seconds to wait between server start and the client call. Inserting a delay provides time for the garbage collector to run "naturally." This simulates a process that "works" initially, but fails after some significant time has passed. Note there is no space before the number of seconds. Example: -delay5 will make the client call 5 seconds after the server is started.

Program behavior will likely vary from machine to machine and JVM to JVM because things like System.gc() are only hints and setting the -delay<S> option is a guessing game with respect to the behavior of the garbage collector.

On my machine, after javac RMITest.java to compile, I see this behavior:

$ java RMITest -nogc -hold
received: foo
$ java RMITest -nogc -release
received: foo
$ java RMITest -gc -hold
received: foo
$ java RMITest -gc -release
Exception in thread "main" java.rmi.NoSuchObjectException: no such object in table
    at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:255)
    at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
    at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:178)
    at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:132)
    at $Proxy0.remoteOperation(Unknown Source)
    at RMITest.client(RMITest.java:69)
    at RMITest.main(RMITest.java:46)

Here is the source code:

import java.rmi.Remote;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.rmi.server.UnicastRemoteObject;
import static java.util.concurrent.TimeUnit.*;

interface RemoteOperations extends Remote {
    String remoteOperation() throws RemoteException;
}

public final class RMITest implements RemoteOperations {
    private static final String REMOTE_NAME = RemoteOperations.class.getName();
    private static final RemoteOperations classVariable = new RMITest();

    private static boolean holdStrongReference = false;
    private static boolean invokeGarbageCollector = true;
    private static int delay = 0;

    public static void main(final String... args) throws Exception {
        for (final String arg : args) {
            if ("-gc".equals(arg)) {
                invokeGarbageCollector = true;
            } else if ("-nogc".equals(arg)) {
                invokeGarbageCollector = false;
            } else if ("-hold".equals(arg)) {
                holdStrongReference = true;
            } else if ("-release".equals(arg)) {
                holdStrongReference = false;
            } else if (arg.startsWith("-delay")) {
                delay = Integer.parseInt(arg.substring("-delay".length()));
            } else {
                System.err.println("usage: javac RMITest.java && java RMITest [-gc] [-nogc] [-hold] [-release] [-delay<seconds>]");
                System.exit(1);
            }
        }
        server();
        if (invokeGarbageCollector) {
            System.gc();
        }
        if (delay > 0) {
            System.out.println("delaying " + delay + " seconds");
            final long milliseconds = MILLISECONDS.convert(delay, SECONDS);
            Thread.sleep(milliseconds);
        }
        client();
        System.exit(0); // stop RMI server thread
    }

    @Override
    public String remoteOperation() {
        return "foo";
    }

    private static void server() throws Exception {
        // This reference is eligible for GC after this method returns
        final RemoteOperations methodVariable = new RMITest();
        final RemoteOperations toBeStubbed = holdStrongReference ? classVariable : methodVariable;
        final Remote remote = UnicastRemoteObject.exportObject(toBeStubbed, 0);
        final Registry registry = LocateRegistry.createRegistry(Registry.REGISTRY_PORT);
        registry.bind(REMOTE_NAME, remote);
    }

    private static void client() throws Exception {
        final Registry registry = LocateRegistry.getRegistry();
        final Remote remote = registry.lookup(REMOTE_NAME);
        final RemoteOperations stub = RemoteOperations.class.cast(remote);
        final String message = stub.remoteOperation();
        System.out.println("received: " + message);
    }
}
Greg Mattes
  • 30,462
  • 13
  • 66
  • 103
  • 1
    So what you are saying is that I need (on the server) to manually retain a reference to my server object, because the exported UnicastRemoteObject that I put into the Registry will not keep that object from being garbage-collected, which would leave me with a dangling reference in the Registry? – Thilo May 12 '09 at 21:45
  • 2
    I would have hoped that until I "unbind" the object, the RMI system would keep it alive. – Thilo May 12 '09 at 21:48
  • 8
    Bingo. The responsibility for keeping the reference seems to fall on the programmer. I had the same hope about the RMI system keeping the reference while binding is in effect. There's probably a good reason for it being the way it is (I've not studied the RMI sources). In any case, it doesn't seem to be overly intuitive. – Greg Mattes May 13 '09 at 00:29
  • Great analysis and a very illustrative example. You deserve more than a measly +1 for this ;) – Steen Apr 19 '10 at 21:04
  • Thank you! Hope this helped you. :) – Greg Mattes Apr 20 '10 at 16:10
  • This only happens if you use UnicastRemoteObject.exportObject() an bind the stub to the registry. If you inherit from UnicastRemoteObject and bind the real object to the registry, it will not get GCd even if you don't have a static ref. This is from experience (Sun Java 1.6.22 Ubuntu64) and not necessarily always true. I would use the static ref as a precaution. – FkYkko Nov 01 '10 at 12:25
  • Just to be clear, the object must be "reachable" so that it is not reclaimed by the garbage collector. Whether the reference to the object is a static (type-level) variable is irrelevant. http://java.sun.com/docs/books/jls/third_edition/html/execution.html#12.6 – Greg Mattes Nov 15 '10 at 19:25
  • What's the best way holding a strong reference? I'm wildly guessing JIT optimisations could destroy some that should otherwise work. – Bart van Heukelom Jan 25 '11 at 14:35
  • The object has to be reachable so that it won't be finalized and reclaimed by the garbage collector. There is link at the top of my answer to the definition of reachability in the Java language spec, this information might be helpful to you. Also, consider reading chapter 11 "References in Four Flavors" from the book "Hardcore Java" http://oreilly.com/catalog/9780596005689 – Greg Mattes Jan 25 '11 at 14:44
  • 8
    This answer is not correct. It completely ignores the effects of DGC. The existence of a DGC client for a remote object is sufficient to prevent it being locally GC'd. In this case the Registry becomes a DGC client of the remote object by virtue of being fed a remote stub for it. The reason why this code behaves as it does is because the *Registry* itself gets GC'd, which both unexports it and releases all its bindings, which in turn releases its DGC hold on the remote object, which in turn allows the remote object to be locally GC'd. – user207421 Aug 23 '12 at 06:53
  • @EJP. Hmm. Of course, if there is an active client, DGC will keep the server alive. That was not being questioned. Some of us were just expecting to have the server object alive even when there are no more clients (by virtue of it still being bound, and hence "accessible"). But that does not seem to happen unless we explicitly create another reference on the server for the sole purpose of not having it garbage-collected. – Thilo Aug 23 '12 at 09:11
  • @Thilo The RMI Registry *is* a DGC client, hence the server object *is* alive 'by virtue of its being bound'; and I have already explained why the code in this answer behaves as it does. – user207421 Aug 23 '12 at 09:26
  • Hmm. How come then that the bug disappeared by just making an extra reference to the UnicastRemoteObject (and *not* the Registry)? This should not affect how/when/if the Registry is garbage collected. (Maybe this code does not present this scenario properly, where the Registry is not garbage collected, but the UnicastRemoteObject is. You say that can't happen?) – Thilo Aug 23 '12 at 09:37
  • @Thilo Adding the extra reference to the remote object prevents GC of the remote object happening once the Registry disappeared, along with its reference to the remote object. It doesn't affect how/when/if the Registry itself is garbage-collected, and I didn't say otherwise. – user207421 Aug 23 '12 at 11:11
  • @EJP: I agree that the example script is a bit unfortunate (because it allows Registry to be garbage-collected). But the same error occurs when you put Registry in a static field, so that it does not disappear. Registry is still alive, but you get a " java.rmi.NoSuchObjectException: no such object in table". – Thilo Aug 23 '12 at 11:58
  • 1
    @EJP I'm interested to learn more. Will you please write another script that demonstrates your ideas and post it as another answer to this question? – Greg Mattes Aug 23 '12 at 14:59
  • @Thilo Not according to the RMI Specification, or to my experience of RMI over fifteen years. It would take a network partition problem of several minutes duration to prevent the DGC lease from being renewed for it to be possible for the server to be DGC'd, which is a precondition for it being locally GC'd, as the DGC server end holds a local reference. DGC protocol failures do happen, but not unless there is a network partition to fail, which there isn't in this case. – user207421 Aug 23 '12 at 22:36
  • @GregMattes I suggest it would be simpler all round for you to read up on DGC in the RMI Specification. – user207421 Aug 23 '12 at 22:41
  • 1
    @EJP in a Q&A site (like this one), nothing would be better than an comprehensive answer, instead of telling us to read the specs... – Rafael Eyng Sep 21 '16 at 17:43
  • @RafaelEyng There is a limit. What I have already provided seems extremely comprehensive to me. Any further information required beyond what I've already provided here should certainly be sought in the RMI Specification rather than here, as all I would be doing would be repeating it, or quoting it. – user207421 Jan 18 '17 at 07:34
8

Some other questions to consider - First are you referencing an object instance or is the stub interface itself gone? If some object instance is gone, its for the usual reasons, it got dereferenced and GC'd, but if it's the interface then your RMI server end point loop quit for some reason.

The best debugging tool I've found so far is to turn on the java.rmi.server.logCalls=true property (see http://java.sun.com/j2se/1.5.0/docs/guide/rmi/javarmiproperties.html) and watch all the wonderfull information stream down your log window. This tells me what's up every time.

jos

jottos
  • 19,028
  • 9
  • 28
  • 26
2

I have the same problem and now I've solved it. The solution is simple, you MUST create strong reference 'object' to avoid the object being GC'd.

for example in your server class:

...
private static ServiceImpl serviceImpl = null;

public static void register (int port) {
    serviceImpl = new ServiceImpl();
    Registry registry = LocateRegistry.createRegistry(port);
    registry.rebind ("serviceImpl", serviceImpl);
}

public static void main(String[] args) throws RemoteException, NotBoundException {
    register(1099);    
    ...the rest of your code...
}

So, it protects "serviceImpl" object from being GC'd. CMIIW

Fahmi
  • 21
  • 1
1

there is one point missing in the above discussion. There is something that is called distributed garbage collection (DGC). If there are no living local and remote references to a distributed object the GC is allowed to remove the object from memory. There is a sophisticated algorithm to verify this. The nice code snippet from above is indeed a good demonstration of the effectiveness of the DGC.

What somehow looks like a feature is nothing but the designed behavior!

Frank

Frank Z
  • 11
  • 2
0

While using spring remoting (rmi) i bumped into this error. My service wasn't garbage collected.

After turning on debug logging for "org.springframework" i discovered that my server was registering the service on the default port (1099) instead of the port the client was trying to connect to.

I thought everything port wise was ok cause "java.rmi.server.logCalls=true" did show some output on server when client was trying to connect.

When getting this error double check the ports (the service and registry one).

Tinus Tate
  • 1,988
  • 2
  • 9
  • 26
0

It's difficult to answer this question without looking at the code (which I guess will be big enough to not be publishable here). However, using Occam's razor, you have two possibilies

  • Server objects must be getting unregistered somehow
  • Since breakpoints stop the errors, it's definitely a race condition.

I would suggest you go over the code paths carefully keeping the two points above in mind.

talonx
  • 1,751
  • 1
  • 14
  • 28
-1

Got the same error but probably for the other (yet unknown) reason.

I was casting exported object to the type of my remote interface and then while binding to name I was getting NoSuchObjectException. Removing casting fixed the problem.

Briefly:

public interface MyRemoteInterface extedns Remote {
    ...
}

public class MyRemoteObject implements MyRemoteInterface {
    ...
}

public static MyRemoteObject obj = new MyRemoteObject();

public static void main(String[] args) {
    //removing cast to MyRemoteInterface fixes the problem
    this.obj = UnicastRemoteObject.exportObject((MyRemoteInterface) this.obj, 0);

    //unless the above cast is removed, this throws NoSuchObjectException occasionally
    LocateRegisry.getRegistry("127.0.0.1", 1099).bind("name", this.obj);
}
  • Removing the cast fixed nothing. The result of `exportObject()` is a `Remote,` not `MyRemoteObject,` and it is in fact a stub, not the `Remote` object argument supplied. This code cannot possibly compile, let alone execute. – user207421 Sep 25 '14 at 00:23