Provides classes for robust usage of RMI, primarily for streaming data. One common usage is streaming files to/from a remote EJB or some other RMI server.

Package Specification

RemoteRetry

This facility provides a basis for robust communication using RMI. Any type of remote communication has the possibility of intermittent errors. Naive programs using RMI often ignore this possibility, thus creating very fragile implementations. This class makes it easy to add appropriate retry abilility to RMI method calls. Note that calls which are retryable must be idempotent (see {@link com.healthmarketscience.rmiio.RemoteRetry} for more details). Also included is a base class {@link com.healthmarketscience.rmiio.RemoteWrapper} which simplifies wrapping remote interfaces for use with the RemoteRetry facility. This allows code to use a remote interface without having to implement the retry code directly (see {@link com.healthmarketscience.rmiio.RemoteInputStreamWrapper} for example usage).

Remote Streams

The bulk of this package is code to support streaming data over RMI. There are many ways to do this the wrong way. Let us explore some possibilities. (Note, all of these "solutions" were found online when searching for an existing solution to this problem).

Idea: Send a File object over the wire.
Problem: A File is only a String which describes the file, so you are not actually sending any data, just a reference.
Idea: Send a URL/File object, and let the server access the file directly
Problem: Only works if the server can access the same resources as the client, which is very limiting in any enterprise scenario (cannot assume common disk access, or even common host access)
Idea: Send a byte[] to the server
Problem: This is the first scenario that kind of works... at least until your files get to 1GB (or some large size which will not fit in memory), in which case your client box may run out of memory trying to create the byte[]. Again, not a very robust solution.
Idea: Open a socket and stream the data to the server
Problem: Well, we are getting the right idea, but again, opening a socket between an arbitrary client and server can involve negotiating proxies and firewalls, which is difficult at best. Not to mention the additional work of implementing socket based streaming, etc.
Idea: Why not use the existing RMI connection to stream the data, by sending the client chunks of the file at a time?
Finally, the right idea. But still not trivial to implement. Hopefully, though, this package should fill that need in a way that is accessible for anyone who can use java.io based streams.

The {@link com.healthmarketscience.rmiio.RemoteInputStream} and {@link com.healthmarketscience.rmiio.RemoteOutputStream} interfaces provide the basis of the functionality for streaming data over RMI. However, most users of this package should never have to use or implement these interfaces directly. There are wrappers for both the client and server implementations which wrap these interfaces as regular java InputStreams or OutputStreams (including usage of the RemoteRetry utility to automatically retry certain remote exceptions), making them very easy to work with.

The examples below assume that the reader has knowledge of how RMI works.


  //
  // The "server" is reading a file from the "client"
  //
  
  // "server" implementation

  public void sendFile(RemoteInputStream inFile) {
    // wrap RemoteInputStream as InputStream (all compression issues are dealt
    // with in the wrapper code)
    InputStream istream = RemoteInputStreamClient.wrap(inFile);

    // ... read file here using normal InputStream code ...
  }

  // "client" implementation

  // create a RemoteStreamServer which uses compression over the wire (note
  // the finally block to release RMI resources no matter what happens)
  RemoteInputStreamServer istream = null;
  try {
    istream = new GZIPRemoteInputStream(new BufferedInputStream(
      new FileInputStream("myFile.txt")));
    // call server (note export() call to get actual remote interface)
    server.sendFile(istream.export());
  } finally {
    if(istream != null) istream.close();
  }


  //
  // The "server" is writing a file to the "client"
  //

  // "server" implementation

  public void getFile(RemoteOutputStream outFile) {
    // wrap RemoteOutputStream as OutputStream (all compression issues are
    // dealt with in the wrapper code)
    OutputStream istream = RemoteOutputStreamClient.wrap(inFile);

    // ... write file here using normal OutputStream code ...
  }


  // "client" implementation

  // create a RemoteStreamServer which uses no compression over the wire (note
  // the finally block to release RMI resources no matter what happens)
  RemoteOutputStreamServer ostream = null;
  try {
    ostream = new SimpleRemoteOutputStream(new BufferedOutputStream(
      new FileOutputStream("myResults.txt")));
    // call server (note export() call to get actual remote interface)
    server.getFile(ostream.export());
  } finally {
    if(ostream != null) ostream.close();
  }


  //
  // A more complicated example:  The "server" returns data streamed directly
  // out of a database.  The interesting situation here is that the database
  // connection lives longer than the initial method call, and will need to be
  // closed when the stream is closed.  To implement this, we use a
  // RemoteStreamMonitor which will be notified when the stream is closed
  // (among other things).
  //

  // "server" implementation (using jdbc)

  public RemoteInputStream getFile(String fileName) {
    Connection conn = null;
    try {

      conn = getConnection();
      Blob fileBlob = null;

      // ... do db work to get handle to a Blob ...
      
      // create RemoteInputStreamMonitor to close db connection when stream is
      // done
      final Connection streamConn = conn;
      RemoteInputStreamMonitor monitor = new RemoteInputStreamMonitor() {
        public void closed(RemoteInputStreamServer stream, boolean clean) {
          streamConn.close();
        }
      };

      // create server to stream blob bytes back using compression
      RemoteInputStream istream = new GZIPRemoteInputStream(
        fileBlob.getBinaryStream(), monitor).export();

      // the monitor has taken control of the db connection (note, we don't
      // set the conn to null until after the RemoteInputStream has been
      // exported, in case any exceptions occur in that process)
      conn = null;

      // return already exported RemoteInputStream
      return istream;

    } finally {
      if(conn != null) conn.close();
    }
  }

  // "client" implementation

  InputStream istream = null;
  try {

    istream = RemoteInputStreamClient.wrap(server.getFile("importFile.txt"));

    // ... read file here using normal InputStream code ...

  } finally {
    // do all we can to close the stream when finished
    if(istream != null) istream.close();
  }

Remote Iterators

The {@link com.healthmarketscience.rmiio.RemoteIterator} facility is built on top of the remote streams to enable streaming a large collection of objects over RMI as easily as streaming a file. Possible applications of this facility might be reading a large collection of objects out of a remote database, where the entire collection may not fit in memory at once. While the implementor of the RemoteIterator is free to use whatever method available to convert the objects to a stream, the common usage would be to use Serializable objects with the {@link com.healthmarketscience.rmiio.SerialRemoteIteratorServer} and {@link com.healthmarketscience.rmiio.SerialRemoteIteratorClient}. Note that the RemoteIterator interface is not actually a Remote interface. Instead, the RemoteIterator implementation is a Serializable object which gets copied across the wire. Internally, it contains a reference to a RemoteInputStream, from which it reads the objects being transferred.

The examples below assume that the reader has knowledge of how RMI works (as well as how the remote streams above work).


  //
  // The "server" returns a RemoteIterator of Strings from a database to the
  // "client".  We will use a similar trick used in the previous example to
  // close the db resources when the iterator is closed.
  //

  // "server" implementation (using jdbc)

  public RemoteIterator<String> getResults(String searchParam) {

    Connection conn = null;
    Statement stmt = null;
    ResultSet rs = null;
    try {

      // ... create ResultSet with relevant results ...
      
      // note, this is an *example* ResultSet iterator implementation, not the
      // cleanest in the world (doesn't handle all close scenarios, left as an
      // exercise to the reader)...
      final Connection iterConn = conn;
      final Statement iterStmt = stmt;
      final ResultSet iterRs = rs;
      IOIterator<String> dbIter = new IOIterator<String>() {
        private boolean _hasNext = iterRs.next();
        private String _next = (_hasNext ? iterRs.getString(1) : null);
        private boolean _isClosed = false;
        public boolean hasNext() throws IOException {
          if(!_hasNext && !_isClosed) {
            _isClosed = true;
            try {
              try {
                iterRs.close();
              } finally {
                try {
                  iterStmt.close();
                } finally {
                  iterConn.close();
                }
              }
            } catch(SQLException e) {
              throw new IOException(e);
            }
          }
          return _hasNext;
        }

        public String next() throws IOException {
          String cur = _next;
          try {
            if(_hasNext = iterRs.next()) {
              _next = iterRs.getString(1);
            }
          } catch(SQLExcepion e) {
            throw new IOException(e);
          }
          return cur;
        }
      };

      // create RemoteIterator client/server objects.  Note that the
      // RemoteInputStream export() call is handled internally to these
      // objects.
      SerialRemoteIteratorServer<String> stringServer =
        new SerialRemoteIteratorServer<String>(dbIter);
      SerialRemoteIteratorClient<String> stringClient =
        new SerialRemoteIteratorClient<String>(stringServer);

      // the RemoteIterator now owns the db resources
      rs = null;
      stmt = null;
      conn = null;
        
      return stringClient;

    } finally {
      try {
        if(rs != null) rs.close();
      } finally {
        try {
          if(stmt != null) stmt.close();
        } finally {
          if(conn != null) conn.close();
        }
      }
    }
  }

  // "client" implementation

  RemoteIterator<String> resultIter = null;
  try {
    resultIter = server.getResults("SELECT THE_STRING");
    while(resultIter.hasNext()) {
      System.out.println("Got string: " + resultIter.next() +
                         " from the server");
    }
  } finally {
    // do all we can to close the iterator when finished
    if(resultIter != null) resultIter.close();
  }
  

Usage Notes

Writing distributed applications is not easy, and adding streaming to the mix further complicates things. These utilities should make life easier, if used correctly. Included below are some recommendations learned through the shedding of much blood, sweat, and many tears.

General

JDBC

EJB