Uploaded image for project: 'QuickFIX/J'
  1. QuickFIX/J
  2. QFJ-923

FileStore is leaking file handles

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.6.2, 1.6.3
    • Fix Version/s: 1.6.4
    • Component/s: Engine
    • Environment:
      All

      Description

      The FileStore like all the other stores are not synchronized.
      The access to this store, at times, can be done by 2 threads concurrently.

      When a session is about to be estabilished, logon request is sent to the server, if the request is not answered by the server in within the timeout period the connection is closed and the reconnect timer starts.

      The problem is that when the Timeout Timer [QFJ Timer] fires it also releases [QFJ Message Processor] thread which tries to process EndOfFille/Stream event, both threads will try to reset the MessageStore on next().

      During the reset() the FileStore is (re)initialized which causes the following actions: close existing files, open the files, read stores.

      When done in parallel, one thread may try to open or read while the other is closing the files resulting in:
      1. Read corrupted data from message stores
      2. Leak file handles

      Some proof:
      16:18:28.763 [QFJ Timer] INFO quickfix.FileStore - initialize() called from.
      at quickfix.FileStore.initialize(FileStore.java:111) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.FileStore.reset(FileStore.java:465) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.SessionState.reset(SessionState.java:382) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.resetState(Session.java:2503) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.disconnect(Session.java:1968) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.next(Session.java:1808) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:278) [quickfixj-core-1.6.2.jar:1.6.2]

      And from:
      16:18:28.788 [QFJ Message Processor] INFO quickfix.FileStore - initialize() called from.
      at quickfix.FileStore.initialize(FileStore.java:111) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.FileStore.reset(FileStore.java:465) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.SessionState.reset(SessionState.java:382) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.resetState(Session.java:2503) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.disconnect(Session.java:1968) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.next(Session.java:882) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.next(Session.java:1109) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:144) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:91) [quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:125) [quickfixj-core-1.6.2.jar:1.6.2]

      Out of file handles:
      ERROR [QFJ Timer] quickfix.SocketInitiator Error during timer processing
      quickfix.RuntimeError: java.io.FileNotFoundException: [/..../Session].header (Too many open files)
      at quickfix.SessionState.reset(SessionState.java:384) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.resetState(Session.java:2499) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.disconnect(Session.java:1967) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.Session.next(Session.java:1808) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:278) [quickfixj-core-1.6.2.jar:1.6.2]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_74]
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_74]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_74]
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_74]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_74]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_74]
      at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]
      Caused by: java.io.FileNotFoundException: [/.../Session].header (Too many open files)
      at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_74]
      at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_74]
      at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_74]
      at java.io.FileOutputStream.<init>(FileOutputStream.java:133) ~[?:1.8.0_74]
      at quickfix.FileStore.initializeMessageIndex(FileStore.java:198) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.FileStore.initializeCache(FileStore.java:121) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.FileStore.initialize(FileStore.java:116) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.FileStore.reset(FileStore.java:442) ~[quickfixj-core-1.6.2.jar:1.6.2]
      at quickfix.SessionState.reset(SessionState.java:382) ~[quickfixj-core-1.6.2.jar:1.6.2]
      ... 11 more

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chrjohn Christoph John
                Reporter:
                cfl001 Constantin Florescu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: