[QFJ-224] Automatically close logs and message stores when session connector is closed. Created: 14/Aug/07  Updated: 15/Nov/12  Resolved: 20/May/11

Status: Closed
Project: QuickFIX/J
Component/s: Networking
Affects Version/s: 1.2.1
Fix Version/s: 1.5.1

Type: Improvement Priority: Default
Reporter: Toli Kuznets Assignee: Steve Bate
Resolution: Fixed Votes: 0
Labels: None
Environment:

Linux


Issue Links:
Duplicate
is duplicated by QFJ-350 Sessions do not cleanup after themsel... Closed
is duplicated by QFJ-406 No way for FileStore to be closed Closed
Relates
relates to QFJ-287 Provide hook for manually closing Fil... Closed

 Description   

When you use a FileStoreFactory as a MessageFactory for "Acceptor" QFJ applications, it does not seem to close or delete files associated with a particular session.

For example, when i connect to my acceptor (running on Linux) with a new dynamic session, i can see the new set of files being created for it.
I can run 'lsof +p <pid> ' and see the files being open for the session.

When i kill my initiator and force a disconnect, the files still remain open - i can still see them in the output of lsof command.

So after a long running time and lots of incoming connections the acceptor application ends up running out of file descriptors.

I stepped through the code and it looks like the files are being closed in the Session.disconnect() --> FileStore.reset() call, but somehow the file descriptors still show up in use.



 Comments   
Comment by Steve Bate [ 14/Aug/07 ]

The acceptor does no management of the message stores.

To implement this feature, we'd need to create some type of new or extended interface that some of the message stores could implement that would allow them to clean up their resources based on specific session events.

As a workaround, you could access the session's state in the Application onLogout callback, cast it to a FileStore and then call deleteFiles() on it.

Comment by Toli Kuznets [ 15/Aug/07 ]

The above workaround works for me, as long I have the following vars setup:
ResetOnLogin=Y
ResetOnDisconnect=N
ResetOnLogout=N

Since Session.disconnect() calls into application.onLogout() before it does the check/reinitalization on disconnect, i had to set ResetOnDisconnect=N.

My quickfix.Application.onLogout() code:
public void onLogout(SessionID sessionId) {
FileStore store = (FileStore) Session.lookupSession(sessionId).getStore();
try {
store.closeFiles();
store.deleteFiles();
if(LoggerAdapter.isDebugEnabled(this))

{ LoggerAdapter.debug("Closed and deleted files for session "+sessionId, this); }

} catch (IOException e)

{ LoggerAdapter.error(e.getMessage(), e, this); }

}

Comment by Steve Bate [ 06/Oct/07 ]

We have the listener interface to implement this feature now. I think it should be configurable (default to off) since the FileStore loads all sent messages when it initializes. This could cause a significant delay during counterparty login. However, closing the files and clearing the caches makes a lot of sense for other applications, like simulators.

Comment by André Malenfant [ 21/Nov/07 ]

The same applies to the quickfix.Log class but the workaround cannot be used since the close() method is protected.

Comment by André Malenfant [ 21/Nov/07 ]

Sorry, It is atually package scope.

Comment by André Malenfant [ 21/Nov/07 ]

Sorry fro the multiple comments in a row...

To be more precise, I don't talk specifically about deleting the files. This can be a nice developement feature but you definitely don't want that in a prod environment. What is most important is that the store AND the logger close their files if the initiatior or acceptor is stopped. This way, if a web application gracefully responds to a stop event from the container, it can stop all connections opened in quickfixj and expect the files to be unlocked.

Comment by André Malenfant [ 21/Nov/07 ]

And B.T.W. (sorry again) to close the files on logout is not enough because the session may not be logged on at all at the time the initiator or acceptor is closed. This should be done also or at least on close.

Comment by Steve Bate [ 11/Jan/08 ]

I split this into a separate issue for 1.3.1 to expose hooks for manually closing FileLog files. See the related issue.

Comment by Steve Bate [ 06/Apr/10 ]

Added support to Session for disposable message stores and logs. However, we need to think more about how to support these in the current file-based implementations. There are still potential issues with the FileStore if it's required to load it's caches on every session connect event. FileLogs might be a little easier to handle.

Comment by Steve Bate [ 26/Apr/11 ]

Modified the title to more accurately reflect the feature request. One approach for this is to check in the session connector (initiator or acceptor) whether the message log and/or store implements the java.io.Closeable interface. If so, the connector will call close on the log or store.

Comment by Steve Bate [ 20/May/11 ]

This issue has become confusing over the last four years. I'm think that we want to close the file-based resources when a connector is stopped. When a connector is stopped, the Sessions are unregistered. If we "close" the sessions at this point then we can release resources if the plugins (store, log) support it. Let me know if anyone else has a different interpretation.

Comment by Steve Bate [ 20/May/11 ]

If a MessageStore or Log implementation implements the java.io.Closeable interface then the Session will close them (the connector tells the session to close them after unregistering the session). The FileLog and FileStore have been modified to implement the Closeable interface.

Comment by Mate Varga [ 08/Aug/12 ]

Am I right thinking that (initiator means SocketInitiator in the following)

  • an initiator creates the sessions in start() -> initialize() -> createSessionInitiators()
  • if an initiator is stopped, then the sessions will be unregistered
  • if an initialized initiator is started, then it only re-registers the sessions, and does not recreate them (see initialize())
  • a Session closes it's Log if it's closeable (see Session -> unregisterSessions() -> close() -> closeIfCloseable(getLog())
  • a FileLog only opens it's streams in clear() and in it's constructor

— therefore, if SocketInitiator using a FileLog is stopped, then next time it gets started it will throw exceptions as the underlying file handle is closed.

Opinions?

Comment by Mate Varga [ 08/Aug/12 ]

By the way, this is in 1.5.2.

Generated at Fri May 17 05:24:24 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.