[QFJ-1000] can't pass target destination URL (SocketConnectHost) to proxy server Created: 09/Feb/21  Updated: 09/Feb/21

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.4
Fix Version/s: None

Type: Bug Priority: Blocker
Reporter: leaf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows



 Description   

i am using quickfixj to connect with target URL using proxy , the proxy protocol is socks ,
after checked with our proxy team , they said the destination URL (SocketConnectHost) is not passed to proxy , which means not send url to which proxy need to connect .

below is my quickfixj configure sample:

ProxyType=socks
ProxyHost=*************
ProxyPort=1080
ProxyVersion=5

SocketConnectHost=********
SocketConnectPort=******



 Comments   
Comment by Christoph John [ 09/Feb/21 ]

Did you try with a newer version? 1.6.4 is rather old and there have been proxy related fixes in the last versions.





[QFJ-977] ResendRequest message ( 35=2) received from Sender was ignored in QuickFIXJ Created: 21/May/19  Updated: 08/Jul/19

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Lantony Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, ResendRequest,, sequence,
Environment:

Linux



 Description   

Fix session was interrupted due to network issue... Both side FIX processes ( Client, Sender) did not know about the n/w issue. The Client side was continuously sending logon request to other side... Once the n/w issue was resolved the connectivity was established. However, the sequence was not in sync at this time . Hence , The sender sent resend request (35=2) immediately after successful logon response.
But this 35=2 request was not processed by quick fix engine at my side ( Client side). I can see the message in quickfix messages.log arrived from sender but not traceability on what had happened with this message.
Only what i can see , the Client side ( fix engine) sent 35=2 for which the sender has responded with sequencereset i.e., 35=4. I think , the sender on the other side responded well with the Admin messages sent by my quickFix engine.

the problem is why my fix engine (i.e., the client ) did not respond to 35=2 resent request given by sender ? . I am expecting my Fix engine should have responed with 35=4 like my Sender's engine . I am using QuickfixEngineJ 1.5.3

Some one kindly help please. The below the sequence of FIX messaged traced in messages.log/

Response from Sender:
8=FIXT.1.19=72 35=A 34=75098 49=456 52=20190516-23:30:20.382 56=123 98=0 108=30 1137=9 10=178

Resent request from sender ( problem where I did not respond for this message)
8=FIXT.1.19=66 35=2 34=75099 49=456 52=20190516-23:30:20.38256=123 7=74910 16=0 10=154

Resent request from Client (i.e., by my quick fix engine )
8=FIXT.1.19=66 35=2 34=74963 49=123 52=20190516-23:30:20.436 56=456 7=75019 16=0 10=154

reset request responded by sender:
8=FIXT.1.19=99 35=434=75019 43=Y 49=456 52=20190516-23:30:20.489 56=123 122=20190516-23:30:20.489 36=75100 123=Y 10=043



 Comments   
Comment by Lantony [ 22/May/19 ]

Small Correction , The environment is Windows , not Linux.

Comment by Christoph John [ 08/Jul/19 ]

Hi,
I cannot tell from the log file why this is not working. Have you tried with a newer version? Yours is rather old. But I guess the problem cannot be reproduced reliably?

Cheers,
Chris.





[QFJ-466] Initiator SocketConnectorIoProcessor-0.0 deadlocks with QFJ Message Processor at logout Created: 01/Sep/09  Updated: 29/Apr/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Steve Borrer Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Ubuntu Linux 2.6.24-23-server #1 SMP Thu Nov 27 18:45:02 UTC 2008 x86_64 GNU/Linux



 Description   

The FIX initiator is logged out at 21:30 UTC by the acceptor (itself built with QFJ). The initiator logs out. An exception is reported in the initiator. Shortly after, the initiator initiates a logon request which is declined as it is outside the time window of the acceptor. There is then a endless sequence of 'Pending connection not established after XXXX ms' messages. See my application log below:-

20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - Received logout request
20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - Sent logout response
20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - DisconnectingAug 28, 2009 9:30:00 PM quickfix.mina.AbstractIoHandler exceptionCaught
SEVERE: socket exception (localhost/127.0.0.1:9876): Connection reset by peer

20090828-21:30:00 20090828-09:30:00 DEBUG - loggingOutState:true
20090828-21:30:00 20090828-09:30:00 DEBUG - Session logged out: FIX.4.2:NONAME->SIMULA
Aug 28, 2009 9:30:01 PM quickfix.mina.initiator.InitiatorIoHandler sessionCreated
INFO: MINA session created: /127.0.0.1:47950
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Initiated logon request
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Received logout request
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Disconnecting
20090828-21:30:02 20090828-09:30:02 DEBUG - loggingOutState:true
20090828-21:30:02 20090828-09:30:02 DEBUG - Session logged out: FIX.4.2:NONAME->SIMULA
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Attempt to send while not connected (message stored until connected).
20090828-21:30:33 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 2000 ms.
20090828-21:30:36 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 5000 ms.
20090828-21:30:39 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 8001 ms.
20090828-21:30:42 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 11001 ms.
20090828-21:30:45 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 14002 ms.
20090828-21:30:48 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 17002 ms.
20090828-21:30:51 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 20003 ms.
20090828-21:30:54 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 23003 ms.
20090828-21:30:57 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 26004 ms.
... etc

A SIGQUIT to the initiator reveals a deadlock:-

20090901-08:07:25 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 38094032 ms.
20090901-08:07:28 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 38097032 ms.
2009-09-01 08:07:28
Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.0-b16 mixed mode):

"Checkpointer" daemon prio=10 tid=0x00007f15040fa000 nid=0x1f7f in Object.wait() [0x0000000040d6d000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:161)

  • locked <0x00007f150f850a80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"Cleaner-1" daemon prio=10 tid=0x00007f15040f9800 nid=0x1f7e in Object.wait() [0x0000000041293000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:161)

  • locked <0x00007f150f84ca80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"INCompressor" daemon prio=10 tid=0x00007f1504216000 nid=0x1f7d in Object.wait() [0x0000000042400000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163)

  • locked <0x00007f150f84ea80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"SocketConnectorIoProcessor-0.0" daemon prio=10 tid=0x0000000040789000 nid=0x1f7c waiting for monitor entry [0x0000000042804000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.sleepycat.je.Cursor.close(Cursor.java:242)

  • waiting to lock <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.close(Database.java:242)
  • locked <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at quickfix.SleepycatStore.reset(SleepycatStore.java:285)
    at quickfix.SessionState.reset(SessionState.java:368)
    at quickfix.Session.resetState(Session.java:1828)
    at quickfix.Session.disconnect(Session.java:1542)
    at quickfix.mina.AbstractIoHandler.sessionClosed(AbstractIoHandler.java:100)
    at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.sessionClosed(AbstractIoFilterChain.java:677)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:292)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.common.support.AbstractIoFilterChain$HeadFilter.sessionClosed(AbstractIoFilterChain.java:599)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.fireSessionClosed(AbstractIoFilterChain.java:313)
    at org.apache.mina.common.support.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:271)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.doRemove(SocketIoProcessor.java:225)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$700(SocketIoProcessor.java:44)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:567)
    at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Client:nt@tradera1" prio=10 tid=0x000000004060c000 nid=0x178f runnable [0x0000000042905000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

  • locked <0x00007f150ee02ab0> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
  • locked <0x00007f150ee02ab0> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"QFJ Message Processor" daemon prio=10 tid=0x00007f1504272000 nid=0x177b waiting for monitor entry [0x0000000042602000]
java.lang.Thread.State: BLOCKED (on object monitor)
at com.sleepycat.je.Database.removeCursor(Database.java:879)

  • waiting to lock <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at com.sleepycat.je.Cursor.close(Cursor.java:245)
  • locked <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.putInternal(Database.java:536)
    at com.sleepycat.je.Database.put(Database.java:479)
    at quickfix.SleepycatStore.set(SleepycatStore.java:301)
    at quickfix.SessionState.set(SessionState.java:299)
    at quickfix.Session.sendRaw(Session.java:1810)
    at quickfix.Session.generateLogout(Session.java:1050)
    at quickfix.Session.generateLogout(Session.java:1041)
    at quickfix.Session.nextLogout(Session.java:1024)
    at quickfix.Session.next(Session.java:790)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:107)
    at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:87)
    at java.lang.Thread.run(Thread.java:619)

"QFJ Timer" daemon prio=10 tid=0x00007f15042e1000 nid=0x177a in Object.wait() [0x0000000042501000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:443)
at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:364)
at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)

  • locked <0x00007f150ec65480> (a java.lang.Object)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:680)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:921)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:980)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:528)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Client:oms@tradera1" prio=10 tid=0x000000004018c000 nid=0x1773 runnable [0x0000000040b04000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

  • locked <0x00007f150ede9ec8> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
  • locked <0x00007f150ede9ec8> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Client:oms@tradera1" prio=10 tid=0x00000000405e8000 nid=0x1772 runnable [0x0000000040a03000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

  • locked <0x00007f150eddb7c8> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
  • locked <0x00007f150eddb7c8> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Client:oms@tradera1" prio=10 tid=0x0000000040477800 nid=0x1771 runnable [0x0000000040902000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)

  • locked <0x00007f150ec74260> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
  • locked <0x00007f150ec74260> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Interface" prio=10 tid=0x00007f15040a2000 nid=0x1770 runnable [0x0000000040c6c000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)

  • locked <0x00007f150ebcace0> (a java.net.SocksSocketImpl)
    at java.net.ServerSocket.implAccept(ServerSocket.java:453)
    at java.net.ServerSocket.accept(ServerSocket.java:421)
    at fixclient.OMS_Interface.run(OMS_Interface.java:93)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Sender" prio=10 tid=0x00007f1504059800 nid=0x1735 in Object.wait() [0x00000000422ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at fixclient.OMS_Sender.run(OMS_Sender.java:55)

  • locked <0x00007f150ebc01c0> (a java.util.LinkedList)

"MessageQueue" prio=10 tid=0x00007f1504058800 nid=0x1734 in Object.wait() [0x00000000421fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x00007f150ebc40d8> (a java.util.LinkedList)
    at java.lang.Object.wait(Object.java:485)
    at fixclient.OMS_Interface$MessageQueue.run(OMS_Interface.java:312)
  • locked <0x00007f150ebc40d8> (a java.util.LinkedList)

"Low Memory Detector" daemon prio=10 tid=0x00000000402f2800 nid=0x1726 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x00000000402ef800 nid=0x1724 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x00000000402eb000 nid=0x1721 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00000000402e9000 nid=0x171e waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00000000402c6000 nid=0x1714 in Object.wait() [0x0000000041ffc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)

  • locked <0x00007f150ebc2190> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x00000000402c4000 nid=0x170f in Object.wait() [0x0000000041efb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)

  • locked <0x00007f150ebc6168> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x0000000040112800 nid=0x16e5 in Object.wait() [0x00000000417f4000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x00007f150ebc4258> (a fixclient.FIX_Client)
    at java.lang.Object.wait(Object.java:485)
    at fixclient.FIX_Client.run(FIX_Client.java:218)
  • locked <0x00007f150ebc4258> (a fixclient.FIX_Client)
    at fixclient.FIX_Client.main(FIX_Client.java:325)

"VM Thread" prio=10 tid=0x00000000402bd800 nid=0x170a runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x000000004011c800 nid=0x16ea runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x000000004011e800 nid=0x16ee runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040120800 nid=0x16f2 runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040122000 nid=0x16f6 runnable

"GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000040124000 nid=0x16fa runnable

"GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000040126000 nid=0x16fe runnable

"GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000040127800 nid=0x1702 runnable

"GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000040129800 nid=0x1706 runnable

"VM Periodic Task Thread" prio=10 tid=0x00000000402f5000 nid=0x1728 waiting on condition

JNI global references: 1325

Found one Java-level deadlock:
=============================
"SocketConnectorIoProcessor-0.0":
waiting to lock monitor 0x000000004079f4d8 (object 0x00007f150f8529d8, a com.sleepycat.je.Cursor),
which is held by "QFJ Message Processor"
"QFJ Message Processor":
waiting to lock monitor 0x00000000407a0d20 (object 0x00007f150f852a18, a com.sleepycat.je.Database),
which is held by "SocketConnectorIoProcessor-0.0"

Java stack information for the threads listed above:
===================================================
"SocketConnectorIoProcessor-0.0":
at com.sleepycat.je.Cursor.close(Cursor.java:242)

  • waiting to lock <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.close(Database.java:242)
  • locked <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at quickfix.SleepycatStore.reset(SleepycatStore.java:285)
    at quickfix.SessionState.reset(SessionState.java:368)
    at quickfix.Session.resetState(Session.java:1828)
    at quickfix.Session.disconnect(Session.java:1542)
    at quickfix.mina.AbstractIoHandler.sessionClosed(AbstractIoHandler.java:100)
    at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.sessionClosed(AbstractIoFilterChain.java:677)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:292)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.common.support.AbstractIoFilterChain$HeadFilter.sessionClosed(AbstractIoFilterChain.java:599)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.fireSessionClosed(AbstractIoFilterChain.java:313)
    at org.apache.mina.common.support.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:271)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.doRemove(SocketIoProcessor.java:225)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$700(SocketIoProcessor.java:44)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:567)
    at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43)
    at java.lang.Thread.run(Thread.java:619)
    "QFJ Message Processor":
    at com.sleepycat.je.Database.removeCursor(Database.java:879)
  • waiting to lock <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at com.sleepycat.je.Cursor.close(Cursor.java:245)
  • locked <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.putInternal(Database.java:536)
    at com.sleepycat.je.Database.put(Database.java:479)
    at quickfix.SleepycatStore.set(SleepycatStore.java:301)
    at quickfix.SessionState.set(SessionState.java:299)
    at quickfix.Session.sendRaw(Session.java:1810)
    at quickfix.Session.generateLogout(Session.java:1050)
    at quickfix.Session.generateLogout(Session.java:1041)
    at quickfix.Session.nextLogout(Session.java:1024)
    at quickfix.Session.next(Session.java:790)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:107)
    at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:87)
    at java.lang.Thread.run(Thread.java:619)

Found 1 deadlock.

Heap
PSYoungGen total 10944K, used 3722K [0x00007f1519660000, 0x00007f151a3e0000, 0x00007f151ebb0000)
eden space 8448K, 36% used [0x00007f1519660000,0x00007f151995aaa8,0x00007f1519ea0000)
from space 2496K, 26% used [0x00007f151a170000,0x00007f151a218000,0x00007f151a3e0000)
to space 2688K, 0% used [0x00007f1519ea0000,0x00007f1519ea0000,0x00007f151a140000)
PSOldGen total 26368K, used 14066K [0x00007f150ebb0000, 0x00007f1510570000, 0x00007f1519660000)
object space 26368K, 53% used [0x00007f150ebb0000,0x00007f150f96c9f0,0x00007f1510570000)
PSPermGen total 86016K, used 66374K [0x00007f15097b0000, 0x00007f150ebb0000, 0x00007f150ebb0000)
object space 86016K, 77% used [0x00007f15097b0000,0x00007f150d881a38,0x00007f150ebb0000)



 Comments   
Comment by Eric Deshayes [ 29/Apr/11 ]

The problem is due to the usage of the SleepyCat database:

we're executing a Database.close() and Database.put() concurrently, and the javadoc for Database.close() points out that a database handle should not be closed while operations using that Database object are still running:
"The database handle should not be closed while any other handle that refers to it is not yet closed; for example, database handles should not be closed while cursor handles into the database remain open, or transactions that include operations on the database have not yet been committed or aborted."





[QFJ-966] Initiator Logout does not appear in QFJ log file on connection reset Created: 03/Dec/18  Updated: 03/Dec/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Daniel Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: fromAdmin, initiator, logout
Environment:

linux



 Description   

I think this is an old issue, but still seems to be a problem with the latest version of quickfixj 2.1.0.

After my initiator sends a Logon message, the acceptor immediately sends back a Logout Message followed by closing the connection. Looking at the tcpdump, it is clear why the connection is closed

x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> 0xe431), seq 1:104, ack 1, win 21, length 103
E.....@[email protected].(.7.>..+P....c..8=FIX.4.2.9=81.35=A.34=1161.49=XXX.52=20181130-15:08:36.259.56=**.98=0.108=10.554=**.10=026.
15:08:36.336321 IP (tos 0x60, ttl 48, id 0, offset 0, flags [DF], proto TCP (6), length 188)
x.x.x.x.x > x.x.x.x.56988: Flags [P.], cksum 0xdd5d (correct), seq 1:149, ack 104, win 4096, length 148
E`[email protected]...>..+(.8YP....]..8=FIX.4.2.9=0124.35=5.34=552397.49=XXX.52=20181130-15:08:36.298.56=XXX.789=469881.58=MsgSeqNum too low, expecting 469881 but received 1161.10=079.

The problem is that fromAdmin is never called before onLogout because of the connection reset following the logout message. IE: it is not in the QFJ log file but I can see it in the tcp dump.

The initiator stack trace (ie connection reset) looks like

"..."
"\tat quickfix.Session.disconnect(Session.java:2028)"
"\tat quickfix.mina.AbstractIoHandler.exceptionCaught(AbstractIoHandler.java:84)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.exceptionCaught(DefaultIoFilterChain.java:828)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)"
"\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:937)"
"\tat org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:102)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:590)"
"\tat org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:580)"
"\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:729)"
"\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:659)"
"\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:648)"
"\tat org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:68)"
"\tat org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1120)"
"\tat org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)"
"\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)"
"\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)"
"\tat java.base/java.lang.Thread.run(Thread.java:844)"

and all I receive on the initiator side is

"Initiated logon request"
"Disconnecting: Socket exception (/x.x.x.x:25265): java.io.IOException: Connection reset by peer"
"onLogout: FIX.4.2:XXX->XXX"

message from Christoph John

So this seems to be a special case that is not covered by the unit test.
The problem is that the session is disconnected the "hard way" through the IOException, not waiting for messages to be processed that might still be in flight. This might have to be changed in the AbstractIoHandler to behave similar to the sessionClosed() callback.

But this is only a first quick analysis. Of course this has to be made reproducable in a unit test and then fixed.






[QFJ-959] Fix session could stuck forever in acceptor mode. Created: 07/Nov/18  Updated: 16/Nov/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Dmitry Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Repetitive error: Multiple logons/connections for this session are not allowed
when client try to reconnect to server. (see server logs below)

Looks like session was not cleared correctly after exception.
Exception

Nov  6 06:47:32 aemulator: java.io.IOException: Broken pipe
Nov  6 06:47:32 aemulator: at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
Nov  6 06:47:32 aemulator: at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
Nov  6 06:47:32 aemulator: at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
Nov  6 06:47:32 aemulator: at sun.nio.ch.IOUtil.write(IOUtil.java:65)
Nov  6 06:47:32 aemulator: at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
Nov  6 06:47:32 aemulator: at org.apache.mina.transport.socket.nio.NioProcessor.write(NioProcessor.java:384)
Nov  6 06:47:32 aemulator: at org.apache.mina.transport.socket.nio.NioProcessor.write(NioProcessor.java:47)
Nov  6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.writeBuffer(AbstractPollingIoProcessor.java:1107)
Nov  6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.flushNow(AbstractPollingIoProcessor.java:994)
Nov  6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.flush(AbstractPollingIoProcessor.java:921)
Nov  6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:688)
Nov  6 06:47:32 aemulator: at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
Nov  6 06:47:32 aemulator: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
Nov  6 06:47:32 aemulator: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Nov  6 06:47:32 aemulator: at java.lang.Thread.run(Thread.java:748)

Server logs.

2018-11-07 13:06:56.731 +0000 ERROR [NioProcessor-471] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:06:57.736 +0000 ERROR [NioProcessor-472] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:06:58.740 +0000 ERROR [NioProcessor-473] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:06:59.780 +0000 ERROR [NioProcessor-474] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:07:01.446 +0000 ERROR [NioProcessor-475] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:07:03.446 +0000 ERROR [NioProcessor-476] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:07:05.446 +0000 ERROR [NioProcessor-477] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed
2018-11-07 13:07:07.446 +0000 ERROR [NioProcessor-478] quickfixj.errorEventFIX.4.4:emulator->TRADE:  - Multiple logons/connections for this session are not allowed


 Comments   
Comment by Christoph John [ 07/Nov/18 ]

Hi,

is the pasted stack trace complete? I can not gather much from it. There should also be log entries from QFJ along with the IOException. Can you post them?

I have never seen this behaviour before. Is it reproducible? Did it stop at some time or did it keep running like that?
When an exception occurs, the exceptionCaught() callback from the AbstractIoHandler should get notified and close the session.
Where are you running the QFJ session? Is it a plain Java process or are you running it inside an application server or Spring application?

Thanks,
Chris.

Comment by Christoph John [ 07/Nov/18 ]

And of course as always: would be better to first ask on the mailing list. We can clarify there if it really is a bug.

Comment by Dmitry [ 08/Nov/18 ]

Hi,

is the pasted stack trace complete? I can not gather much from it. There should also be log entries from QFJ along with the IOException. Can you post them?

We do not see another exceptions in log.

I have never seen this behaviour before. Is it reproducible? Did it stop at some time or did it keep running like that?

When an exception occurs, the exceptionCaught() callback from the AbstractIoHandler should get notified and close the session.

Where are you running the QFJ session? Is it a plain Java process or are you running it inside an application server or Spring application?

  • Reproducible from time to time.
  • This is Java plain application which use Spring Core inside.

I believe we found the way how to achieve this behavior. It is definitely not an expected working behavior, but as a result, quickfix could be stuck in some wrong state forever.

From time to time we have an issue with our HyperV cloud. Virtual machine is frozen for ~30-40 seconds. (it is testing environment)
During this time quickfix client tries to reconnect and send logon message.
When VM becomes alive, application throws an exception for TCP session and then try to handle new connection from client and process several logon messages for the same quickfix session.

Probably qfSession.hasResponder() keeps wrong state forever due to exception in Session.disconnect

  public void disconnect(String reason, boolean logError) throws IOException {
        try {
            final boolean logonReceived = state.isLogonReceived();
            final boolean logonSent = state.isLogonSent();

            synchronized (responderLock) {
                if (!hasResponder()) {
                    if (!ENCOUNTERED_END_OF_STREAM.equals(reason)) {
                        getLog().onEvent("Already disconnected: " + reason);
                    }
                    return;
                }
                final String msg = "Disconnecting: " + reason;
                if (logError) {
                    getLog().onErrorEvent(msg);
                } else {
                    getLog().onEvent(msg);
                }
                responder.disconnect();   // Maybe exception is here 
                setResponder(null);
            }


Comment by Christoph John [ 09/Nov/18 ]

Hi,

OK, if the whole VM froze for 30 seconds all sort of things can happen.

You are right: given the log message "Multiple logons/connections for this session are not allowed" it looks like the responder was not set to null.
But good that you can reproduce it. So you could checkout the code from github, tentatively wrap parts of the disconnect() method in a try block with cleanup of the responder in a finally-block and try to reproduce again.
But I would actually expect to see QFJ classes appear in the exception stack trace if something in that method failed. That is also why I was asking if this is a plain Java process. I don't know much about Spring but assume that some parts might run differently than in a "normal" Java process. E.g. sometimes Exceptions are swallowed if thrown in threads that are running as part of a thread pool.

When you say that the Exception might occur in or after responder.disconnect() is called then you should have something in your log file like "Disconnecting: " because that happens some lines above. Do you see something like that in your log file?
Could you maybe post the contents of your event log (including exceptions) around the freeze?

Thanks,
Chris.

Comment by Dmitry [ 16/Nov/18 ]

Hi,

I am sorry for late response.
We transferred our test environment from VM to hardware server.
So, logs were lost and currently we do not have similar issues.

Dmitry.

Comment by Christoph John [ 16/Nov/18 ]

OK, feel free to update the issue when you encounter problems again.





[QFJ-810] Messages get lost during logon process with sequence gap Created: 06/Oct/14  Updated: 16/Oct/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Heribert Steuer Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: condition, gap, logon, race

Issue Links:
Relates
is related to QFJ-804 Race condition between sessionClosed ... Closed

 Description   

When a sequence gap exists, all subsequent messages sent by the counterparty are lost when the logon message contains a out-of-sync sequence number.

Scenario: Counterparty A is initiating a logon, counterparty B responds to the logon and immediately sends 10 more messages. A now detects the sequence problem in the login message and closes the session. The 10 messages are lost (meaning they are logged as received but never appear in the callback)

The problem seems to be that the verify() function detects the gap in the logon messages, creates a logout and a disconnect. This leads to the fact that the ThreadPerSessionEventHandlingStrategy.run method is left and its local queue gets destroyed - still containing the messages that have been received after the logon.

Whatever the correct behaviour would be - the current one is a problem because of the messages are received but never processed. It might be better to leave ThreadPerSessionEventHandlingStrategy.run() only if the queue is empty.



 Comments   
Comment by Christoph John [ 08/Oct/14 ]

Just for clarification: the Logon of counterparty B has a sequence number which is too low? In my opinion the messages following the Logon should not be processed since their sequence number was too low. But maybe I did not understand fully.

Comment by Heribert Steuer [ 09/Oct/14 ]

In any case, just dropping messages should never happen. Probably a session level reject should be sent because business level messages appear without the session being completely established. But receiving and simply dropping does not feel good at all because none of the counterparties get a feedback on the problem.

Please share your thoughts.

Comment by Christoph John [ 10/Oct/14 ]

I see a problem with sending Reject messages if the Session is not fully established.
Let me ask again: was the sequence number too low? I guess it must have been too low if the connection has been closed immediately by A. If the sequence number is too low it is considered a serious problem and the connection has to be dropped (preferrably by sending a Logout message if possible). This would require manual intervention in any case.
Maybe the processing has to be changed in a way that the connection is closed immediately and no more messages are accepted until the session has been completely established.
What do you think?

Comment by Heribert Steuer [ 10/Oct/14 ]

Correct, the SeqNum was too low. In fact we had the issue in production where a session went out of sync and we got a periodic Logon/Logout war. This means that both "A" sent a logon, "B" accepts the logon immediately followed by a resend request. "A" then sends a logout because the sequence number of the logon "A" sent earlier did not match. The problem in total seems to be that there is no real handshake in FIX. So you never really know when a session is really established e.g. like in TCP you have SYN, SYN ACK, ACK. In FIX you would - using TCP as an example - have only SYN and SYN ACK. When the responder replies to a logon message, he never knows if the initiator will accept his logon message. Therefore he does not know when he can start to send data. This is a design flaw in FIX, it would be best the have the initiator have something like "established" returned to the acceptor after his logon.

Nevertheless, the problem remains that the acceptor sends data immediately after sending his logon without knowing if he is allowed to do so or if the logon gets rejected somehow. I do not think that closing the connection is a solution. While you are processing the logon in quickfix, Mina would probably already read data from the socket or traffic will arrive in the network stacks buffer etc., so simply closing the connection would not be a real solution. It always ends up in a race condition. In my optinion, the reject would be okay because from the perspective of the acceptor, the session is established. He received a logon and he replied to it. Therefore quickfix is able to respond to it - and it does today by sending a logout. Therefore a solution might be to check if the buffer (ThreadPerSessionEventHandlingStrategy) is empty. If so, send a logout (as it does now). If the buffer is not empty, process the messages and send a session level reject in return. As the buffer is empty, send a logout and close the socket.

Unfortunately I do not really have a better idea how to solve it, as mentioned earlier its more or less a flaw in the design of the protocol. How does the above idea sound to you?

Comment by Christoph John [ 11/Oct/14 ]

Just another question: this issue basically sounds a little like QFJ-790. Do you agree?

However, your solution sounds feasible. Will take a deeper look at it.

As a workaround for the time being: many people implement the handshake as follows: after Logon immediately send a TestRequest. As soon as you receive the Heartbeat message with your TestReqID you can be pretty sure that the session is established. Of course, this is no 100% guarantee but works quite well in most cases.





[QFJ-752] Instrument component dont show the instrumentParties repeating group Created: 30/Jul/13  Updated: 12/Mar/20

Status: Open
Project: QuickFIX/J
Component/s: Engine, Message Generation
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christian Avalos Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ


 Description   

the noInstrumentParties(1018) component is in the componentFields int array, and not in componentGroup, so, when it added a instrumentParties ,the generated message has the field 1018, but not the other possibles fields of instrumentParties (1019, 1050, 1051, 1052)



 Comments   
Comment by Christian Avalos [ 30/Jul/13 ]

this issue is fixed adding the noIntrumentParties fields into componentGroups attribute (and removing from componentFields int[] )

public class Instrument extends quickfix.MessageComponent {
static final long serialVersionUID = 20050617;
public static final String MSGTYPE = "";
private int[] componentFields =

{ 55, 65, 48, 22, 454, 460, 461, 167, 762, 200, 541, 224, 225, 239, 226, 227, 228, 255, 543, 470, 471, 472, 240, 202, 947, 206, 231, 223, 207, 106, 348, 349, 107, 350, 351, 691, 667, 875, 876, 864, 873, 874, 965, 966, 1049, 967, 968, 969, 970, 971, 996, 997, 1079, }

;
private int[] componentGroups =

{1018, }

;

public Instrument()

{ super(); }

......
}

Comment by Jörg Thönnes [ 31/Jul/13 ]

This message and field classes are generated automatically from the data dictionary so this is most probably a code generation issue.

Comment by Christoph John [ 31/Jul/13 ]

... or in the data dictionary.

Comment by Marcin L [ 12/Mar/20 ]

I was wondering if this is really a bug. Clearly the definition in FIX5* looks like this

<component name="Instrument">
[...]
<component name="InstrumentParties" required="N"/>
</component>

<component name="InstrumentParties">
<group name="NoInstrumentParties" required="N"> // 1018
<field name="InstrumentPartyID" required="N"/> // 1019
<field name="InstrumentPartyIDSource" required="N"/> // 1050
<field name="InstrumentPartyRole" required="N"/> // 1051
<component name="InstrumentPtysSubGrp"/>
</group>
</component>

<component name="InstrumentPtysSubGrp">
<group name="NoInstrumentPartySubIDs" required="N"> // 1052
<field name="InstrumentPartySubID" required="N"/> // 1053
<field name="InstrumentPartySubIDType" required="N"/> // 1054
</group>
</component>

Component classes have their own dedicated classes generated so to access the group we need to go via component abstraction.

public class Instrument extends quickfix.MessageComponent {
private int[] componentFields =

{ [...], 1018, [...] }

;
private int[] componentGroups = { };
}

public class InstrumentParties extends quickfix.MessageComponent {
private int[] componentFields = { };
private int[] componentGroups =

{ 1018, }

;
}

public class InstrumentPtysSubGrp extends quickfix.MessageComponent {
private int[] componentFields = { };
private int[] componentGroups =

{ 1052, }

;
}

This is not a single case. This is basically a case for every component containing another component with a repeating group e.g. in FIX44 UnderlyingInstrument -> UnderlyingStipulations

What kind of errors currently generated classes could produce with this behaviour?





[QFJ-749] Syntax error in SessionSettings.java/line 572 leading exception from java.util.regex Created: 17/Jun/13  Updated: 08/Jul/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Pavel Sagulenko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment:

Eclipse 22 on Debian GNU/Linux 7.0



 Description   

There is a syntax error in SessionSettings.java class, line 572.
original line: private final Pattern variablePattern = Pattern.compile("\\$

{(.+?)}

");
Should be: private final Pattern variablePattern = Pattern.compile("\\$

{(.+?)\\}

");

(with double slash before closing parenthesis).
This causes the failure in creating a SessionSettings instance.

I tried to recompile myself the corrected version, but I got the following exception:

java.lang.NoClassDefFoundError: quickfix.SessionSettings

Probably, you can suggest the possible solution.
Thank you!



 Comments   
Comment by Christoph John [ 08/Jul/13 ]

Sorry, what do you mean by: "This causes the failure in creating a SessionSettings instance"??
Are you able to supply a test case which shows the failure?





[QFJ-734] Properties for configuring Proxool vs. not incremented outgoing sequence number Created: 13/Mar/13  Updated: 13/Mar/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Krzysztof Szalast Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

All



 Description   

I want to remind about comment in quickfix.JdbcUtil in method "static synchronized DataSource getDataSource(String jdbcDriver, String connectionURL, String user, String password, boolean cache)":
// TODO JDBC Make these configurable
setMaximumActiveTime(ds, 5000);
ds.setMaximumConnectionLifetime(28800000);
ds.setMaximumConnectionCount(10);
ds.setSimultaneousBuildThrottle(10);

If "messages" table are locked longer than 5 seconds, connection is aborted by Proxool:
[13.03.2013|12:42:07:875] [HouseKeeper] [DEBUG] [proxool.quickfixj-1] [org.logicalcobwebs.proxool.ConnectionPool] [removeProxyConnection] [441] : 000026 (01/02/00) - #0001 removed because it has been active for too long.
[13.03.2013|12:42:07:875] [HouseKeeper] [WARN ] [proxool.quickfixj-1] [org.logicalcobwebs.proxool.HouseKeeper] [sweep] [149] : #0001 was active for 98688 milliseconds and has been removed automaticaly. The Thread responsible was named 'QFJ Timer', but the last SQL it performed is unknown because the trace property is not enabled.
[13.03.2013|12:42:07:906] [QFJ Timer] [ERROR] [quickfixj.errorEvent] [quickfix.SLF4JLog] [logError] [147] : Error Reading/Writing in MessageStore
java.io.IOException: Couldn't perform the operation prepareStatement: You can't perform any operations on this connection. It has been automatically closed by Proxool for some reason (see logs).
at quickfix.JdbcStore.set(JdbcStore.java:252)
at quickfix.SessionState.set(SessionState.java:308)
at quickfix.Session.sendRaw(Session.java:2313)
at quickfix.Session.generateHeartbeat(Session.java:1861)
at quickfix.Session.next(Session.java:1834)
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:283)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Couldn't perform the operation prepareStatement: You can't perform any operations on this connection. It has been automatically closed by Proxool for some reason (see logs).
at org.logicalcobwebs.proxool.WrappedConnection.invoke(WrappedConnection.java:207)
at org.logicalcobwebs.proxool.WrappedConnection.intercept(WrappedConnection.java:87)
at $java.sql.Wrapper$$EnhancerByProxool$$a5ac9e8b.prepareStatement(<generated>)
at quickfix.JdbcStore.set(JdbcStore.java:245)
... 14 more

and outgoing sequence number is not incremented! Next outgoing message has the same sequence number (sic!).

Scenario:
1. Send NOS message.
2. Lock "messages" table (on Postgres: LOCK TABLE messages IN ACCESS EXCLUSIVE MODE)
3. Send NOS message - message will be sent, but stacktrace from above will be happen. For example message has t34=666
4. Send NOS message - message will be sent with t34=666



 Comments   
Comment by Jörg Thönnes [ 13/Mar/13 ]

Good find! I think in that case the message should not be sent.





[QFJ-681] During load test, some clients are able to connect only after repeated attempts Created: 22/May/12  Updated: 22/May/12

Status: Open
Project: QuickFIX/J
Component/s: Engine, Networking
Affects Version/s: 1.5.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Viswanath Palutla Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, logon
Environment:

1. on my PC, i am running 'Executor' from quickfixj examples
PC config: windows7, i7, 2.7 GHz, 32-bit, 4GB RAM

2. one linux box, where i am starting 25-30 clients to put load on the 'Executor'
linux config: Intel(R) Xeon(TM) CPU 3.20GHz, 2GB RAM


Attachments: Zip Archive load-test-executor.zip    

 Description   

I am using Executor service in the example, because it is similar to my FIX Gateway, in terms of using AcceptorTemplate

1. I have a simple client that connects to Executor service and puts a MarketDataRequest for every 10 milliseconds, total 100 market data requests.
note: 'UnsupportedMessageType' errors are expected for all MarketDataRequests

2. two clients are started for every second. total 30 clients are started

Issues:
-> connection is not established for some clients for some period
From client logs:
(Pending connection not established after 2146 ms.)
(Pending connection not established after 2850 ms.)
(Pending connection not established after 9795 ms.)

-> server is not responding to logon requests promptly for some clients
From client logs:
(Initiated logon request)
(Disconnecting: Timed out waiting for logon response)
(Pending connection not established after 4648 ms.)
(Pending connection not established after 21948 ms.)
(Initiated logon request)
(Received logon)



 Comments   
Comment by Viswanath Palutla [ 22/May/12 ]

attaching batch scripts and class files to load Executor





[QFJ-554] SleepycatStore can enter infinite loop retrieving messages for resend. Created: 26/Aug/10  Updated: 26/Aug/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: James Olsen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

SleepycatStore does not check returned OperationStatus from call to com.sleepycat.je.Cursor.getNext(...):

public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException {
...
if (retVal == OperationStatus.NOTFOUND)

{ log.debug(sequenceKey + "/" + messageBytes + " not matched in database " + messageDatabase.getDatabaseName()); return; } else {
Integer sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
while (sequenceNumber.intValue() <= endSequence) {
messages.add(new String(messageBytes.getData(), charsetEncoding));
if (log.isDebugEnabled()) { log.debug("Found record " + sequenceNumber + "=>" + new String(messageBytes.getData(), charsetEncoding) + " for search key/data: " + sequenceKey + "=>" + messageBytes); }
cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);
sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
}
}
...
}

Should be:

public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException {
...
if (retVal == OperationStatus.NOTFOUND) { log.debug(sequenceKey + "/" + messageBytes + " not matched in database " + messageDatabase.getDatabaseName()); return; }

else {
Integer sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
while (retVal == OperationStatus.SUCCESS && sequenceNumber.intValue() <= endSequence) {
messages.add(new String(messageBytes.getData(), charsetEncoding));
if (log.isDebugEnabled())

{ log.debug("Found record " + sequenceNumber + "=>" + new String(messageBytes.getData(), charsetEncoding) + " for search key/data: " + sequenceKey + "=>" + messageBytes); }

retVal = cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);
sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
}
}
...
}



 Comments   
Comment by James Olsen [ 26/Aug/10 ]

For clarity, the changed lines are:

while (retVal == OperationStatus.SUCCESS && sequenceNumber.intValue() <= endSequence) {
...
}
retVal = cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);





[QFJ-552] Message Stores expected to be thread safe but are not Created: 05/Aug/10  Updated: 21/Apr/17

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.5.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Carmelo Piccione Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux + JDBC + SQL Server 2005


Attachments: Java Source File SynchronizedMessageStore.java     Java Source File SynchronizedMessageStoreFactory.java    
Issue Links:
Relates
is related to QFJ-762 Message stores can become corrupted o... Open
is related to QFJ-923 FileStore is leaking file handles Closed

 Description   

SessionState.java starts with the following bit of code:

/**

  • Used by the session communications code. Not intended to be used by
  • applications.
    *
  • All dynamic data is protected by the session's intrinsic lock. The
  • log and message store implementation must be thread safe.
    */
    public final class SessionState {
    private final Object lock;
    private final Log log;

// MessageStore implementation must be thread safe
private final MessageStore messageStore;
...
}

Yet, looking at the JdbcStore.java and MemoryStore.java implementations, they are in fact not thread safe as designed (may also be true for FileStore.java, but haven't checked).

The memory store uses a regular java.util.HashMap for all its operations, which is clearly not thread safe. I have never actually seen this cause a problem in practice, however.

The jdbc store does not synchronize its use of the jdbc connection, which can lead to race conditions inside the close() method of the jtds prepared statement, as shown in the stack trace below (specifically, the jtds close() method has a boolean to determine if its connection is already closed in which case it will not doubly close it despite any subsequent calls. This value is set once the close routine is complete but occurs after the connection object has already been assigned a null value- this allows another thread to potentially access the null connection object before the boolean guard has been set). Note that this bug occurs about once a week in a production environment (usually during startup), although its frequency is probably higher than average due to many quickfixj applications accessing the same database concurrently.

Example 1:

java.lang.NullPointerException
at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:868)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.close(JtdsPreparedStatement.java:485)
at org.logicalcobwebs.proxool.AbstractProxyStatement.close(AbstractProxyStatement.java:115)
at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:93)
at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)
at quickfix.JdbcUtil.close(JdbcUtil.java:143)
at quickfix.JdbcStore.storeSequenceNumbers(JdbcStore.java:288)
at quickfix.JdbcStore.setNextTargetMsgSeqNum(JdbcStore.java:272)
at quickfix.JdbcStore.incrNextTargetMsgSeqNum(JdbcStore.java:173)
at quickfix.SessionState.incrNextTargetMsgSeqNum(SessionState.java:359)
at quickfix.Session.nextLogon(Session.java:1535)
at quickfix.Session.next(Session.java:720)
at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:106)
at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:86)
at java.lang.Thread.run(Thread.java:619)

Example 2:

java.lang.NullPointerException
at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:868)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.close(JtdsPreparedStatement.java:485)
at org.logicalcobwebs.proxool.AbstractProxyStatement.close(AbstractProxyStatement.java:115)
at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:93)
at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)
at quickfix.JdbcUtil.close(JdbcUtil.java:143)
at quickfix.JdbcStore.set(JdbcStore.java:259)
at quickfix.SessionState.set(SessionState.java:299)
at quickfix.Session.sendRaw(Session.java:1736)
at quickfix.Session.generateLogon(Session.java:1430)
at quickfix.Session.next(Session.java:1357)
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:248)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)



 Comments   
Comment by Carmelo Piccione [ 17/Aug/10 ]

A simple but thread safe message store implementation. The constructor takes a MessageStore instance as input and delegates all method calls to it through an object lock. This is a work around which allows safe usage of the original QFJ message stores such as JdbcStore, FileStore, and MemoryStore.





[QFJ-551] Sender comp id doesnt not get reflected while creating a session if given at runtime Created: 04/Aug/10  Updated: 04/Aug/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: hitesh Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Desc:
i have a quickfix config (for initiator), which doesn't have sender comp id. from my application i get the sender comp id at runtime, which i then set it session settings object. i then create SocketInitiator, and start the initiator. in the onCreate() method, i get sessionID without the sender comp id. am i trying which is not possible? please let me know if i am doing something wrong.

Code:

public boolean initialize(String senderCompID){
SessionSettings settings = null;
try

{ settings = new SessionSettings(settingsFileName); }

catch (ConfigError e)

{ System.err.println("Could not find FIX setting file."); e.printStackTrace(); System.exit(-1); }

String fixVersion = null;
String sndrCompId = senderCompID;
settings.setString(SessionSettings.SENDERCOMPID, senderCompID); // sender compid added in the default session. also tried adding the same in NONE default session in the below lines of code.

String tgtCompId = null;
SessionID sessID;
Iterator<SessionID> ite = settings.sectionIterator();
while(ite.hasNext()) {
sessID = ite.next();
try

{ fixVersion = settings.getString(sessID, SessionSettings.BEGINSTRING); tgtCompId = settings.getString(sessID, SessionSettings.TARGETCOMPID); //settings.setString(sessID, SessionSettings.SENDERCOMPID, senderCompID); // tried adding in one of the sessions. }

catch (Exception e)

{ System.err.println("Could not get [BEGINSTRING] or [TARGETCOMPID] " + "from session id [" + sessID.toString() + "]-[" + e.toString() + "]"); }

}

String fileStorePath;
String fileLogPath;
try

{ fileStorePath = settings.getString("FileStorePath"); }

catch (Exception e)

{ fileStorePath = null; System.err.println("Could not get [FileStorePath] [" + e.toString() + "]"); }

try {
fileLogPath = settings.getString("FileLogPath");
if(fileStorePath == null)

{ fileStorePath = fileLogPath; System.out.println("[FileStorePath] set to [FileLogPath] - [" + fileStorePath + "]"); }

} catch (Exception e)

{ fileLogPath = null; System.err.println("Could not get [FileLogPath] [" + e.toString() + "]"); }

MessageStoreFactory storeFactory = new FileStoreFactory(settings);
LogFactory logFactory = new FileLogFactory(settings);
MessageFactory messageFactory = new MessageFactory();
try

{ initiator = new SocketInitiator(this, storeFactory, settings, logFactory, messageFactory); }

catch (ConfigError e)

{ System.err.println("FIXClient Could not initiate socket!"); e.printStackTrace(); System.exit(-1); }

String fixedString = fixVersion + "" + sndrCompId + "" + tgtCompId; // if i check my settings object i have sender comp id correctly inserted

// further code

initiator.start();
}






[QFJ-584] Repeated test request with dynamic acceptor Created: 03/Apr/11  Updated: 05/Apr/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Other Priority: Major
Reporter: Information Management Experts Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows XP



 Description   

Dear All,

Our application is Client-Server System. Multiple clients(initiators) connect to the server (dynamic acceptor) to exchange fix messages.
We don't have physical configuration file; we create it at runtime. Our server(as initiator) also connects to other external server(second party).
Our clients can connect to the server through internal or external network and up to 10 concurrent clients may connect to the server.When we was using threads to send and receive messages, the client was sending test request and sometimes login request(when the server didn't respond to the client test request). after we removed the threads, the problem became less but still occurring.
Our time zone is Asia/Amman (GMT+2), but we still using default UTC.Below you will find our configuration file

  1. Acceptor Configuration

[DEFAULT]

StartTime=05:00:00

EndTime=05:00:00

FileStorePath=Settings/FIXStorage/AcceptorStore

FileLogPath=Settings/FIXStorage/AcceptorStore

DataDictionary=data/tools/IMEXQuickFIXDialect42.xml

ValidateUserDefinedFields=Y

CheckLatency=N

ConnectionType=acceptor

SocketAcceptPort=2025

[SESSION]

BeginString=FIX.4.2

SenderCompID=FIX

TargetCompID=*

AcceptorTemplate=Y

  • Initiator Configuration

[DEFAULT]

StartTime=05:00:00

EndTime=05:00:00

FileStorePath=Settings/FIXStorage/InitiatorStore

FileLogPath=Settings/FIXStorage/InitiatorStore

DataDictionary=data/tools/IMEXQuickFIXDialect42.xml

ValidateUserDefinedFields=Y

CheckLatency=N

ConnectionType=initiator

ReconnectInterval=5

HeartBtInt=5

SocketConnectHost=123.123.123.123

SocketConnectPort=2025

[SESSION]

BeginString=FIX.4.2

SenderCompID= – GENERATED AT RUNTIME

TargetCompID=FIX

Thanks in advanced for your help ...



 Comments   
Comment by Laurent Danesi [ 05/Apr/11 ]

Hi,

I saw your post but I'm not sure to well understand.
What do you mean by "When we was using threads to send and receive messages"? is it ThreadedSocketAcceptor?

Regards,

Laurent DANESI
Chief Architect
smartTrade Technologies

Comment by Information Management Experts [ 05/Apr/11 ]

Dear Laurent,

First of all, thanks for your reply.

No, we didn't use ThreadedSocketAcceptor. We were creating new threads in the implementation of Application.fromApp method and when we were calling Session.sendToTarget(message, sessionID);

Also we were sending huge no of orders at the same time(up to 1000) from the external server, which means when each order arrives to our server, our server will start new thread to handle the message, then a new thread will be created to send the message again for each client.





[QFJ-569] QuickFix/J is not scalable due to overly long duration lock on sequence number in Session.sendRaw Created: 10/Dec/10  Updated: 10/Dec/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Leon Chadwick Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

I have been writing a capacity test to measure the performance of our FIX engine (not quickfix based), however I can barely get my machine to use any CPU during the test despite my using a large number of threads to generate orders and respond with Acks.
Sadly I will have to now drop usage of quickfix/j as this is a showstopper for a high performance system.

After profiling, I tracked down that all these threads are generally single threaded through quickfix.Session.sendRaw() which is locking to hold onto a sequence number. A scalable solution should keep locking to a minimum duration, i.e. do as much message checking and string conversion as possible, then fill in the sequence number as a short last step whilst inside the lock.

A StringBuilder could be used to hold the toString()'d message whilst not holding the lock and leave sufficient capacity at the head of the array to shift some of the header content left/right to accomodate the sequence number field whilst during the locked region.



 Comments   
Comment by Steve Bate [ 10/Dec/10 ]

Are you trying to send all these orders through one session? If so, the incoming messages must be processed sequentially by the session protocol engine. However, that doesn't mean your application interface implementation must be single threaded. The application implementation can hand off the message processing to a worker thread pool, for example. The downside is that the FIX engine won't be able to rollback the sequence numbers if an exception occurs in the application code.

As the comments state in sendRaw(), the lock is held until the message is processed and the application callback has completed. This is necessary because any exception thrown during this time will cause the sequence number to be rolled back. In other words, the sequence number can't be incremented until these operations complete successfully.

The message throughput depends on many factors. Some of the biggest factors include the number of sessions, the implementations of the message store and message log and the response time of the application implementation. For example, any logging to the disk in the application callback could have a significant impact on message throughput and CPU utilization.

I don't know if any of this applies to your situation since you didn't include many details. However, I know of other organizations that have reported throughput in the 10's of thousands of messages/second.





[QFJ-385] If one Session reset failed, it will cause all existed session failed Created: 17/Dec/08  Updated: 17/Dec/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Mike Gu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows XP



 Description   

If one SessionState reset() failed, it will throw a RuntimeError and cause all existed session failed. It's unacceptable.
public void reset() {
try

{ messageStore.reset(); }

catch (IOException e)

{ throw new RuntimeError(e); }

}






[QFJ-992] quickfixj 2.1.1 has dependency convergence error for org.apache.maven:maven-artifact:3.5.0 Created: 06/Mar/20  Updated: 09/Mar/20

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.1
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrew Peter Marlow Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ


 Description   

the maven plugin enforcer shows that quickfixj has a dependency convergence error for org.apache.maven:maven-artifact:3.5.0. Here is the output:

Dependency convergence error for org.apache.maven:maven-artifact:3.5.0 paths to dependency are:
+-project-that-uses-quickfixj
+-org.quickfixj:quickfixj-all:2.1.1
+-org.quickfixj:quickfixj-codegenerator:2.1.1
+-org.apache.maven:maven-plugin-api:3.5.0
+-org.apache.maven:maven-artifact:3.5.0
and
+-project-that-uses-quickfixj
+-org.quickfixj:quickfixj-all:2.1.1
+-org.quickfixj:quickfixj-codegenerator:2.1.1
+-org.apache.maven:maven-project:2.2.1
+-org.apache.maven:maven-artifact-manager:2.2.1
+-org.apache.maven:maven-artifact:2.2.1
and
+-project-that-uses-quickfixj
+-org.quickfixj:quickfixj-all:2.1.1
+-org.quickfixj:quickfixj-codegenerator:2.1.1
+-org.apache.maven:maven-project:2.2.1
+-org.apache.maven:maven-artifact:2.2.1



 Comments   
Comment by Christoph John [ 06/Mar/20 ]

Thanks for the report. Under which cicrumstances does this cause an issue?

Comment by Andrew Peter Marlow [ 07/Mar/20 ]

The maven enforcer plugin is being used to report on inconsistent component versions. By default the plugin treats all such inconsistencies as errors, so by default such inconsistencies would make the build fail. This behaviour is configurable so it is possible just to get a report of the issues without making the build fail. But I want such inconsistencies to be flagged as serious issues because some of these inconsistencies involve components that show up on Black Duck security scans. The software I am working on is subject to a corporate decree that the software gets through Black Duck without raising any flags. This job is made harder when the number of flags raised is larger due to the same component appearing multiple times at different versions due to these dependency convergence errors.

My aim is to report any such dependency convergence errors that I find via enforcer. If they are then all fixed (there are only a few) then we could set the flag such that any such errors are fatal, so as to detect any regressions as soon as they occur. Without such fixes I will have to write a script that processes the enforcer output and checks any issues against a list of exceptions. This quickfixj issue would have to be such as exception. It just makes ultimate the job of dealing with Black Duck scans harder.

Comment by Christoph John [ 09/Mar/20 ]

Understood. If you would like you could also open a pull request on the quickfixj project on github. https://github.com/quickfix-j/quickfixj
I am also in favour of making the build fail on enforcer errors.





[QFJ-947] Logout message with additional field is not logged Created: 26/Mar/18  Updated: 27/Mar/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.4
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Nikolai Kulakov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux



 Description   

I came across a situation where a Logout messages that have some additional field are not logged anywhere, so it seems that the connection is breaking before the Logout message receiving. Only from our counterparty I found out that they are sending a Logout messages indeed, and my own network traffic analyzing confirmed it.
Logged messages:

8=FIX.4.2^A9=80^A35=A^A34=1^A49=fxw2^A52=20180326-14:16:20.879^A56=JLQD^A98=0^A108=30^A554=password^A10=215^A
8=FIX.4.2^A9=80^A35=A^A34=2^A49=fxw2^A52=20180326-14:16:23.840^A56=JLQD^A98=0^A108=30^A554=password^A10=207^A
...

Logged events:

<2018-03-26 10:16:19.751 -0400> <Session FIX.4.2:fxw2->JLQD:MARKET_DATA schedule is weekly, SUN 21:30:00-UTC - SUN 21:00:00-UTC (weekly, SUN 17:30:00-EDT - SUN 17:00:00-EDT)>
<2018-03-26 10:16:19.752 -0400> <Created session: FIX.4.2:fxw2->JLQD:MARKET_DATA>
<2018-03-26 10:16:19.833 -0400> <Configured socket addresses for session: [/some_ip:port]>
<2018-03-26 10:16:19.863 -0400> <MINA session created: local=/172.20.30.34:33369, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port>
<2018-03-26 10:16:20.882 -0400> <Initiated logon request>
<2018-03-26 10:16:20.885 -0400> <Disconnecting: Socket exception (/some_ip:port): java.io.IOException: Connection reset by peer>
<2018-03-26 10:16:22.888 -0400> <MINA session created: local=/172.20.30.34:33372, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port>
<2018-03-26 10:16:23.842 -0400> <Initiated logon request>
<2018-03-26 10:16:23.845 -0400> <Disconnecting: Socket exception (/some_ip:port): java.io.IOException: Connection reset by peer>
...

I read a pure network traffic and had seen that immediately after out Logon we actually receive the following Logout messages:

8=FIX.4.2^A9=0109^A35=5^A34=88^A49=JLQD^A52=20180326-14:16:20.886^A56=fxw2^A789=51^A58=MsgSeqNum too low, expecting 51 but received 1^A10=075^A
8=FIX.4.2^A9=0109^A35=5^A34=89^A49=JLQD^A52=20180326-14:16:23.846^A56=fxw2^A789=51^A58=MsgSeqNum too low, expecting 51 but received 2^A10=076^A
...

I think, this incoming Logout messages must be logged as well as our outcoming Logon messages, but this does not happen. I set logging level to the minimal possible values, but this did not help: there are no this Logout messages anywhere.
The additional tag 789 was added to FIX dictionary, so I think it should be processed correctly. But at any case, whether this Logout message valid or not, it should be presented in logs.
Thanks!



 Comments   
Comment by Nikolai Kulakov [ 26/Mar/18 ]

Of course, it would be great to see this Logout messages not only in the logs, but and in the quickfix.Application#fromAdmin as it usually happens - it allows us to handle this Logout message and set the expected sequence number (I already did this with some other counterparties).

Comment by Christoph John [ 26/Mar/18 ]

I think the problem is that the session is disconnected at 2018-03-26 10:16:20.885 (see event log) but the Logout message is coming in at 20180326-14:16:20.886 (SendingTime on message in your network log). This means the session is disconnected 1ms before the counterparty sends the Logout message (given that both your and the counterparty's clock is in sync).

This can't be a general problem because this kind of disconnection (message seqnum too low) happens quite often with some counterparties and the messages show up in the log files. I don't know of any occurrences with current QuickFIX/J versions.

So unless we can reproduce this in a unit test we cannot do anything about it.

Sorry, but I can only repeat my statement from the other issues: please please please write to the mailing list before opening possible "bug" tickets. That way we can clarify if there really is a bug before issues are opened which in the end is saving your and our time.

Thanks,
Chris.

Comment by Nikolai Kulakov [ 27/Mar/18 ]

Please, take a look to this network dump of the incoming messages (on the our side):

10:16:19.846538 IP some_ip.port > our_ip.port: Flags [S.], seq 28365822, ack 2573233766, win 32000, options [mss 1380,nop,wscale 8], length 0
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  0030 0000 4000 3406 90c8 cfef 1bda ac14  [email protected].........
        0x0020:  1e22 6204 8259 01b0 d3fe 9960 6e66 7012  ."b..Y.....`nfp.
        0x0030:  7d00 8f83 0000 0204 0564 0103 0308       }........d....
10:16:20.885507 IP some_ip.port > our_ip.port: Flags [P.], seq 1:134, ack 103, win 4096, length 133
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  00ad 0000 4000 3406 904b cfef 1bda ac14  [email protected]......
        0x0020:  1e22 6204 8259 01b0 d3ff 9960 6ecc 5018  ."b..Y.....`n.P.
        0x0030:  1000 216b 0000 383d 4649 582e 342e 3201  ..!k..8=FIX.4.2.
        0x0040:  393d 3031 3039 0133 353d 3501 3334 3d38  9=0109.35=5.34=8
        0x0050:  3801 3439 3d4a 4c51 4401 3532 3d32 3031  8.49=JLQD.52=201
        0x0060:  3830 3332 362d 3134 3a31 363a 3230 2e38  80326-14:16:20.8
        0x0070:  3836 0135 363d 6678 7732 0137 3839 3d35  86.56=fxw2.789=5
        0x0080:  3101 3538 3d4d 7367 5365 714e 756d 2074  1.58=MsgSeqNum.t
        0x0090:  6f6f 206c 6f77 2c20 6578 7065 6374 696e  oo.low,.expectin
        0x00a0:  6720 3531 2062 7574 2072 6563 6569 7665  g.51.but.receive
        0x00b0:  6420 3101 3130 3d30 3735 01              d.1.10=075.
10:16:20.885543 IP some_ip.port > our_ip.port: Flags [R.], seq 134, ack 103, win 0, length 0
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  0028 0000 4000 3406 90d0 cfef 1bda ac14  .([email protected].........
        0x0020:  1e22 6204 8259 01b0 d484 9960 6ecc 5014  ."b..Y.....`n.P.
        0x0030:  0000 3711 0000 0000 0000 0000            ..7.........
10:16:22.887575 IP some_ip.port > our_ip.33372: Flags [S.], seq 4175399090, ack 3053759955, win 32000, options [mss 1380,nop,wscale 8], length 0
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  0030 0000 4000 3406 90c8 cfef 1bda ac14  [email protected].........
        0x0020:  1e22 6204 825c f8df 88b2 b604 add3 7012  ."b..\........p.
        0x0030:  7d00 878b 0000 0204 0564 0103 0308       }........d....
10:16:23.845393 IP some_ip.port > our_ip.33372: Flags [P.], seq 1:134, ack 103, win 4096, length 133
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  00ad 0000 4000 3406 904b cfef 1bda ac14  [email protected]......
        0x0020:  1e22 6204 825c f8df 88b3 b604 ae39 5018  ."b..\.......9P.
        0x0030:  1000 1b6f 0000 383d 4649 582e 342e 3201  ...o..8=FIX.4.2.
        0x0040:  393d 3031 3039 0133 353d 3501 3334 3d38  9=0109.35=5.34=8
        0x0050:  3901 3439 3d4a 4c51 4401 3532 3d32 3031  9.49=JLQD.52=201
        0x0060:  3830 3332 362d 3134 3a31 363a 3233 2e38  80326-14:16:23.8
        0x0070:  3436 0135 363d 6678 7732 0137 3839 3d35  46.56=fxw2.789=5
        0x0080:  3101 3538 3d4d 7367 5365 714e 756d 2074  1.58=MsgSeqNum.t
        0x0090:  6f6f 206c 6f77 2c20 6578 7065 6374 696e  oo.low,.expectin
        0x00a0:  6720 3531 2062 7574 2072 6563 6569 7665  g.51.but.receive
        0x00b0:  6420 3201 3130 3d30 3736 01              d.2.10=076.
10:16:23.845427 IP some_ip.port > our_ip.33372: Flags [R.], seq 134, ack 103, win 0, length 0
        0x0000:  000c 2909 731b 001f 9edf 3ebf 0800 4500  ..).s.....>...E.
        0x0010:  0028 0000 4000 3406 90d0 cfef 1bda ac14  .([email protected].........
        0x0020:  1e22 6204 825c f8df 8938 b604 ae39 5014  ."b..\...8...9P.
        0x0030:  0000 2f19 0000 0000 0000 0000            ../.........
...

As I see, there are constantly repeating 3 records:

  1. Probably, a connect message.
  2. Then, about one second later, the Logout message.
  3. Then, immediately, one more message, probably disconnect.

I think that the time between our and the counterparty sides is not so good synchronized to rely on this (in fact, we have some problems with time synchronization on the used machine).
I guess, the problem may be in the too little delay between receiving the Logout and the disconnect messages. Also, it can be in the using of non-standard tag 789. Of course, it can be in something else, but it exists - that is the reason because I wrote this in the issues tracker, not to bother you, of course.

Thanks and my apologize if I'm using not the best procedure of issues notifying,
Nikolai.

Comment by Christoph John [ 27/Mar/18 ]

Hi Nikolai,
there is no need to apologize. I just wanted to stress the fact that it is hard to fix issues that cannot be reproduced reliably. We would need some kind of reproducable unit test to look deeper into this issue. I must admit that I never heard of this problem before but of course this does not mean that it is not possible to happen. May be related to a couple of things.

Thanks,
Chris.





[QFJ-946] Improve misleading error message during Logon Created: 21/Mar/18  Updated: 26/Mar/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.4
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Nikolai Kulakov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Linux



 Description   

It seems that the network connection between my company and our partner is very fast and the QuickFix/J engine suppose that their answering Logon message is coming before we send our Logon message:

8=FIX.4.2^A9=86^A35=A^A34=1^A49=fxw2^A52=20180321-14:09:01.447^A56=JLQD^A98=0^A108=30^A141=Y^A554=password^A10=253^A
8=FIX.4.2^A9=0065^A35=A^A34=317^A49=JLQD^A52=20180321-14:09:01.453^A56=fxw2^A98=0^A108=30^A10=119^A
8=FIX.4.2^A9=86^A35=A^A34=1^A49=fxw2^A52=20180321-14:09:04.413^A56=JLQD^A98=0^A108=30^A141=Y^A554=password^A10=249^A
8=FIX.4.2^A9=0065^A35=A^A34=318^A49=JLQD^A52=20180321-14:09:04.417^A56=fxw2^A98=0^A108=30^A10=123^A
8=FIX.4.2^A9=86^A35=A^A34=1^A49=fxw2^A52=20180321-14:09:07.413^A56=JLQD^A98=0^A108=30^A141=Y^A554=password^A10=252^A
8=FIX.4.2^A9=0065^A35=A^A34=319^A49=JLQD^A52=20180321-14:09:07.416^A56=fxw2^A98=0^A108=30^A10=126^A
etc

<2018-03-21 10:09:00.329 0400> <Session FIX.4.2:fxw2>JLQD:MARKET_DATA schedule is weekly, SUN 21:30:00-UTC - SUN 21:00:00-UTC (weekly, SUN 17:30:00-EDT - SUN 17:00:00-EDT)>
<2018-03-21 10:09:00.330 0400> <Created session: FIX.4.2:fxw2>JLQD:MARKET_DATA>
<2018-03-21 10:09:00.409 -0400> <Configured socket addresses for session: [/some_ip:port]>
<2018-03-21 10:09:00.444 -0400> <MINA session created: local=/172.20.30.34:57198, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port>
<2018-03-21 10:09:01.453 -0400> <Initiated logon request>
<2018-03-21 10:09:01.462 -0400> <Received logon>
<2018-03-21 10:09:01.462 -0400> <Disconnecting: Received logon response before sending request>
<2018-03-21 10:09:03.461 -0400> <MINA session created: local=/172.20.30.34:57200, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port>
<2018-03-21 10:09:04.417 -0400> <Initiated logon request>
<2018-03-21 10:09:04.421 -0400> <Received logon>
<2018-03-21 10:09:04.421 -0400> <Disconnecting: Received logon response before sending request>
<2018-03-21 10:09:06.468 -0400> <MINA session created: local=/172.20.30.34:57202, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port>
<2018-03-21 10:09:07.415 -0400> <Initiated logon request>
<2018-03-21 10:09:07.419 -0400> <Received logon>
<2018-03-21 10:09:07.419 -0400> <Disconnecting: Received logon response before sending request>
etc.

So, I do not see any way to establish the connection.



 Comments   
Comment by Nikolai Kulakov [ 21/Mar/18 ]

After further investigation of the problem, I guess that the problem is not in the speed of the connection, but in the fact that we set ResetSeqNumFlag=Y in the our Logon message but in the answering Logon message it is not presented (so, the priority of the problem can be lowered).

The code that is generating this error is in the method quickfix.Session#nextLogon:

        // Check for proper sequence reset response
        if (state.isResetSent() && !state.isResetReceived()) {
            disconnect("Received logon response before sending request", true);
        }

It can be a problem indeed, but the disconnect message seems totally misleading for me.
Also, I do not see any benefits to break down the connection in this case (we sent ResetSeqNumFlag but did not received it in the answer).

Comment by Christoph John [ 21/Mar/18 ]

I don't know but what sense does it make to reset the sequence numbers only on one side? So I guess the check might have a point. Unless I'm missing something.

Comment by Nikolai Kulakov [ 21/Mar/18 ]

Just a real situation: our partner is ready to reset the sequence number on the one side only. It seems that using ResetSeqNumFlag for this purpose it is not nice by FIX standards, but also it is a fact that this situation is not related with the "Received logon response before sending request".
Also, it is not clear for me why dropping down the connection is better than working.

Comment by Christoph John [ 21/Mar/18 ]

I think it does not make sense to open a bug ticket for every counterparty that does not conform to normal FIX standard.
If the counterparty wanted that you reset your sequence numbers I guess they should be sending you tag 141 in their Logon. At least IMHO.
Why are you sending 141? Does the counterparty require you to do so?

IMHO the best would be to direct such questions to the mailing list. Please post follow up questions and your answers there.
https://sourceforge.net/projects/quickfixj/lists/quickfixj-users

Thanks

Comment by Nikolai Kulakov [ 21/Mar/18 ]

John, I agree that it is no sence to open a bug ticket for a counetrparty that does not conform regular FIX standard (of course, I'll discuss this situation with them directly). But the ticket is about another thing and I'm really sorry if I'm describing it not clear.
The problem is that the engine is disconnecting with "Received logon response before sending request" when there is no such situation (logon response is coming after sending request).
Thank you for the patience.





[QFJ-918] Initiator failover funtion Created: 03/Apr/17  Updated: 03/Apr/17

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.3
Fix Version/s: None

Type: Other Priority: Default
Reporter: Cheng Guo Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment:

Linux



 Description   

Hi,
I just learn the QuickFIXJ, and thinking if QuickFIXJ could support the below failover/ HA scenario, or if you could give me some suggestion for the custom dev?
We act as Initiator to connect to the external acceptor, as the Initiator, we want to deploy to two servers, one as the primary which is active, the other is the backup, and non-active.
Primary will be communicating with external acceptor, meanwhile, by HA module, the secondary sync with the primary at the seq number, msg, etc.. Once primary is down, secondary will take over the primary role, and connect to the external.
Checked the document, seems QuickFIX support the acceptor failover, but dont have the initiator case?
could you help let me know how address this? thanks.

by the way, i also developing to send the FIX msg to IBM MQ, could you suggest if QuickFIXJ has the persistence module/ api to store the un-delivery msg if MQ down, and then send out once MQ is up?



 Comments   
Comment by Christoph John [ 03/Apr/17 ]

You'll need to implement the HA functionality by yourself. You could use a database, synced file system or clustered in-memory cache to store the session information. But the whole failover decision has to be implemented by your application.

Since JIRA is the bug tracker you could ask on the mailing list whether anyone has already implemented something similar: https://lists.sourceforge.net/lists/listinfo/quickfixj-users
If you need production or implementation support please check: http://www.quickfixj.org/support/

Thanks,
Chris.





[QFJ-930] FIXSettings not set for new DataDictionary objects created by DefaultDataDictionaryProvider Created: 18/Jul/17  Updated: 29/Jan/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Sachin Agrawal Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File quickfix_930.rar    

 Description   

While DefaultSessionsFactory.processFixtDataDictionaries adds ApplicationDictionary to dataDictionaryProvider it uses beginStringQualifier only.

While method of MessageUtils
(Message parse(Session session, String messageString) throws InvalidMessage)

method used session,messageString both to get the ApplVerId. this results in mismatched key inside DefaultDataDictionaryProvider and DefaultDataDictionaryPovider need to create a new DataDictionary but FIX sesion settings are not set here at all, which are set by DefaultSessionsFactory while executing processFixtDataDictionaries inside createDataDictionary.

Please let me know in case this is intentional.



 Comments   
Comment by Christoph John [ 18/Jul/17 ]

I am sorry, but I cannot follow your description. Could you maybe describe the problem with some code examples or a unit test?
Thanks,
Chris.

Comment by Sachin Agrawal [ 19/Jul/17 ]

Thanks Chris for looking into,i now try to be more specific.

Please look into DeafultSessionFactory.processFixtDataDictionaries, we find following piece of code, please note here that toApplVerID is used to create the instance of ApplVerID before the applicationDictionary is added to the datadictionaryProvider.

final DataDictionary dd = createDataDictionary(sessionID, settings, key,
                            beginStringQualifier);
dataDictionaryProvider.addApplicationDictionary(MessageUtils.toApplVerID(beginStringQualifier), dd);

and createDataDictionary method reads the settings

private DataDictionary createDataDictionary(SessionID sessionID, SessionSettings settings,
            String settingsKey, String beginString) throws ConfigError, FieldConvertError {
        final String path = getDictionaryPath(sessionID, settings, settingsKey, beginString);
        final DataDictionary dataDictionary = getDataDictionary(path);

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER)) {
            dataDictionary.setCheckFieldsOutOfOrder(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER));
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES)) {
            dataDictionary.setCheckFieldsHaveValues(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES));
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS)) {        	
            boolean chkUnorderedGrpFields = settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS);
            settings.getLogger().error("Msg SttString "+Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS +" value "+ chkUnorderedGrpFields +" dataDictionary " +dataDictionary.toString() +" beginString "+beginString);
			dataDictionary.setCheckUnorderedGroupFields(chkUnorderedGrpFields);
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_USER_DEFINED_FIELDS)) {
            dataDictionary.setCheckUserDefinedFields(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_USER_DEFINED_FIELDS));
        }

        if (settings.isSetting(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS)) {
            dataDictionary.setAllowUnknownMessageFields(settings.getBool(sessionID,
                    Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS));
        }

        return dataDictionary;
    }

Please note that DefaultDataDictionaryProvider uses ApplVerID as the key of the map.

While there is a method called MessageUtils.parse(Session session, String messageString), which uses getApplVerID(session, messageString) to create the ApplVerId and uses the same to fetch the applicationDictionary from dataDictionaryProvider.

MessageUtils.parse(Session session, String messageString)
.....
final String beginString = getStringField(messageString, BeginString.FIELD);
		final String msgType = getMessageType(messageString);

		ApplVerID applVerID;

		if (FixVersions.BEGINSTRING_FIXT11.equals(beginString))
		{
			applVerID = getApplVerID(session, messageString);
		}
		else
		{
			applVerID = toApplVerID(beginString);
		}
		final MessageFactory messageFactory = session.getMessageFactory();

		final DataDictionaryProvider ddProvider = session.getDataDictionaryProvider();
		final DataDictionary sessionDataDictionary = ddProvider == null ? null : ddProvider.getSessionDataDictionary(beginString);
		final DataDictionary applicationDataDictionary = ddProvider == null ? null : ddProvider.getApplicationDataDictionary(applVerID);
....

So when an application message is received a new DataDictionary is created by DefaultDataDictionaryProvider, look at method

 public DataDictionary getApplicationDataDictionary(ApplVerID applVerID) 

so the problem is that this new DataDictionary created by DefaultdataDictionaryProvider does not copy the FIX configurations i.e. settings. Hence even though the application dev has set the settings they are not picked while parsing messages.

Please ask further clarifications.

Thanks,
Sachin

Comment by Christoph John [ 19/Jul/17 ]

Hi,

I have changed https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/DefaultSessionFactoryTest.java (method testFixtDataDictionaryConfiguration) slightly to set "AllowUnknownMessageFields" to true.

 @Test
    public void testFixtDataDictionaryConfiguration() throws Exception {
        SessionID sessionID = new SessionID(FixVersions.BEGINSTRING_FIXT11, "SENDER", "TARGET");
        setUpDefaultSettings(sessionID);
        settings.setBool(sessionID, Session.SETTING_USE_DATA_DICTIONARY, true);
        settings.setString(sessionID, Session.SETTING_TRANSPORT_DATA_DICTIONARY, "FIXT11.xml");
        settings.setString(sessionID, Session.SETTING_DEFAULT_APPL_VER_ID, "FIX.4.2");
        settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY, "FIX42.xml");
        settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY + "." + FixVersions.BEGINSTRING_FIX40, "FIX40.xml");
        settings.setString(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS, "Y");   // added

        Session session = factory.create(sessionID, settings);

        DataDictionaryProvider provider = session.getDataDictionaryProvider();
        assertThat(provider.getSessionDataDictionary(sessionID.getBeginString()),
                is(notNullValue()));
        final DataDictionary applicationDataDictionary = provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX42));

        assertThat(applicationDataDictionary,
                is(notNullValue()));
        assertThat(provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX40)),
                is(notNullValue()));
        
        System.err.println("XXXX " + provider.getSessionDataDictionary(sessionID.getBeginString()).isAllowUnknownMessageFields() );
        System.err.println("XXXX " + applicationDataDictionary.isAllowUnknownMessageFields() );
        
        quickfix.fixt11.Logon logon = new quickfix.fixt11.Logon(new EncryptMethod(EncryptMethod.NONE_OTHER), new HeartBtInt(30),
                new DefaultApplVerID(ApplVerID.FIX42));
        MessageUtils.parse(session, logon.toString());

This gives the following as output:

XXXX true
XXXX true

So the setting is picked up by both dictionaries.

Then I stepped into MessageUtils.parse(session, logon.toString()) with the debugger and could also see that the setting is true on the applicationDataDictionary.

So maybe I am taking a different code path than you? BTW I tested it on master, i.e. with version 1.7.0-SNAPSHOT.

Cheers,
Chris.

Comment by Sachin Agrawal [ 20/Jul/17 ]

Hi Chris,

I understand in your use case a new dictionary is never created, but there are some configurations in which a scenario occurs which creates a new dictionary by DefaultDataDictionaryProvider, then the settings will not be copied.
so to solve the problem we have 2 options

1) I correct the Fix configurations so that a new dictionary is never created.
2) Somehow set the session settings on each datadictionary created by DefaultDataDictionaryProvider

I see you are suggesting option 1.

--Regards,
Sachin

Comment by Christoph John [ 20/Jul/17 ]

Hi,

when there are such configurations then please give an example configuration. Otherwise I don't know how to reproduce the problem.

Thanks,
Chris.

Comment by Sachin Agrawal [ 29/Jan/18 ]

Attached is the zip src files with change

Add fix for correctly reading the settings in all the cases, i.e. whenever DataDictionary object is created from SessionFactory or MessageUtils.parse
    a) Update interface DataDictionaryProvider.java add DataDictionary getApplicationDataDictionary(SessionID sessionId, ApplVerID applVerID);
    b) Update DefaultDataDictionaryProvider
        1) add field SessionSettings
        2) add another constructor with SessionSettings, update old constructors to initialize field settings
        3) add implementation of new method getApplicationDataDictionary(SessionID sessionId, ApplVerID applVerID); which also readSettings\
        4) add public static method fillSettings
        5) add local method readSettings which calls static method fillsettings which is extracted from DefaultSessionFactory
    c) Update DefaultSessionFactory
        1) Update method create use new Constructor added for DefaultDataDictionaryProvider, pass settings.
        2) Update method createDataDictionary remove filling of settings locally and call DefaultDataDictionaryProvider.fillSettings instead
    d) Update MessageUtils, by updating method parse(Session session, String messageString) throws InvalidMessage so that appDataDictionary is fetched from ddProvider by also passing SessionID
    e) Update Session. next(Message message, boolean isProcessingQueuedMessages) throws FieldNotFound, RejectLogon, IncorrectDataFormat,
            IncorrectTagValue, UnsupportedMessageType, IOException, InvalidMessage, so that getApplicationDataDictionary is called with sessionID param as well.

Comment by Sachin Agrawal [ 29/Jan/18 ]

only improvement the changes i uploaded have is that whereever the DataDictionary object is created, the existing settings of  current session are always applied on DataDictionary created.

Comment by Christoph John [ 29/Jan/18 ]

Thanks for the file. Are you maybe able to generate a patch or even better open a pull request on github?
Thanks,
Chris.





[QFJ-880] qfj doesn't send the next batch when ResendRequestChunkSize > 0 Created: 26/Feb/16  Updated: 10/Oct/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.0
Fix Version/s: None

Type: Other Priority: Default
Reporter: Xiaojun Zhang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: CME, resend
Environment:

FIX4.2



 Description   

Quickfixj seems to send the 1st resend batch only after receiving a seq reset.

i.e.
ResendRequestChunkSize=10

15:28:37.142 [QFJ Message Processor] - fromAdmin PNPBBBN: 8=FIX.4.29=10735=134=44749=CME50=G52=20160225-20:28:37.22356=PNPBBBN57=DROPCOPY143=US,NY369=45621112=IDiky5miuo10=025

15:28:37.142 [QFJ Message Processor] - toAdmin PNPBBBN: 8=FIX.4.29=10735=034=4562249=PNPBBBN50=DropCopy52=20160225-20:28:37.14256=CME57=G142=US,NY143=CME112=IDiky5miuo10=004

I manually reduced the incoming seq no to 428
15:29:07.969 [Thread-30] - command: "fix PNPBBBN 428"

15:29:08.120 [QFJ Message Processor] - toAdmin PNPBBBN: 8=FIX.4.29=10535=234=4562449=PNPBBBN50=DropCopy52=20160225-20:29:08.12056=CME57=G142=US,NY143=CME7=42816=43710=188

15:29:08.148 [QFJ Message Processor] - fromAdmin PNPBBBN: 8=FIX.4.29=13635=434=42843=Y49=CME50=G52=20160225-20:29:08.25556=PNPBBBN57=DROPCOPY122=20160225-20:29:08.255143=US,NY369=4562436=449123=Y10=001

I expect quickfix to send the next batch (438-447) but it sent a normal heartbeat instead.

15:29:38.704 [QFJ Timer] - toAdmin PNPBBBN: 8=FIX.4.29=9235=034=4562549=PNPBBBN50=DropCopy52=20160225-20:29:38.70456=CME57=G142=US,NY143=CME10=069



 Comments   
Comment by Christoph John [ 26/Feb/16 ]

Sorry, this is a little hard to follow. I don't know about the command "fix PNPBBBN". Is this some custom stuff that you are executing? Actually, I would expect QFJ to send a Logout if it received too low a sequence number. The last log line is half an hour later than the log line before. What happened in between?

Actually, there are some unit tests around the chunked resend requests behaviour. But of course, this does not mean that there could be no bugs inside that portion of the code. But I cannot really follow your example. Do you have a unit test or some more concise steps which can be followed?

Comment by Xiaojun Zhang [ 26/Feb/16 ]

Sorry the log is a bit hard to read.
1. "fix PNPBBBN" is an admin command I created which calls session.setNextTargetMsgSeqNum() so I can manipulate the seq no to trigger the scenario.
2. The log started from 15:28:37 to 15:29:38. You can ignore the UTC time in the FIX message.
3. A simpler example is
1) both sides seq no started from 1.
2) for some reasons client didn't receive some messages
3) the next message client received was seq no 20
4) chunk size was set to 10. so client sent a resent request being=2 end =11
5) client received seq reset with GapFill=Y
6) client should send the next resend request begin=2 end=20 but it didn't

Hope this helps.

Comment by Xiaojun Zhang [ 26/Feb/16 ]

Sorry there was a typo in 6) client should send the next resend request begin=12 end=20 but it didn't

Comment by Christoph John [ 26/Feb/16 ]

If you look at the incoming sequence reset you see the field 36/NewSeqNo set to 449. So that is why there are no more messages resent. This should actually also be a message in your event log. Something along the lines of: "received sequence reset to 449."

Comment by Xiaojun Zhang [ 26/Feb/16 ]

Hmm i am reading the FIX doc.
The message in all situations specifies NewSeqNo <36> to reset as thevalue of the next sequence numberimmediately following the messages and/or sequence numbers being skipped.
It seems <36> just indicates the next seq no I will receive. Does it mean the whole gap has been filled? In order to send the next resend request what <36> should be?

Actually I was connecting to CME drop copy and encountered this issue.

Comment by Christoph John [ 26/Feb/16 ]

If you send a resend request and the counterparty sends SequenceReset with NewSeqNo = 449 then QFJ will continue with that incoming seqnum. That means that the gap has been filled up to that seqnum, yes. Probably there were only session messages in between, or messages that CME does not want to resend.

Comment by Christoph John [ 26/Feb/16 ]

NB: I think this belongs more onto the mailing list or with CME support than here. It's no bug after all.

Comment by Xiaojun Zhang [ 27/Feb/16 ]

Sorry I still don't quite understand the logic here.
Let's say the seq gap is 1-20 and chunk size is set to 10. Application messages are in 11-20 but NOT in 1-10. After QFJ sends the 1st resend request from 1 to 10 what NewSeqNo should be in SequenceReset?
I don't think it will be 11 because seq no shouldn't decrease.
If it is 21 QFJ will continue with 21 and lose all messages from 11 to 20.

If my understanding is not correct can you please give me an example how chunked resend should work? Appreciate your help.

Comment by Xiaojun Zhang [ 01/Mar/16 ]

If you don't think it is a bug I will add code in my Application class to override the logic. I just want to point out that current logic does NOT work for CME iLink and Drop Copy 4.0 which place a 2500 limit on resend request. QFJ would lose all messages after the first batch. It makes more sense to assume the corresponding chunk gap has been filled based on the sequence reset, not the whole gap.

Comment by Christoph John [ 01/Mar/16 ]

Sorry, I totally forgot about this one. What logic do you want to override? Actually, you do not need to override any of the session logic.

It seems <36> just indicates the next seq no I will receive. Does it mean the whole gap has been filled? In order to send the next resend request what <36> should be?

The NewSeqNo on a sequence reset is the next-to-be sequence number of the other communication side.

Let's say the seq gap is 1-20 and chunk size is set to 10. Application messages are in 11-20 but NOT in 1-10. After QFJ sends the 1st resend request from 1 to 10 what NewSeqNo should be in SequenceReset?

NewSeqNo should be 11.

I don't think it will be 11 because seq no shouldn't decrease.
If it is 21 QFJ will continue with 21 and lose all messages from 11 to 20.

Why does the seq no decrease? It is as follows: you connect to the counterparty. Your expected target seq no is 1. The other side comes in with 20. Now you send a resend request from 1-10.
You receive a SeqReset with NewSeqNo 11. Now your expected target seq no is 11. This is an increment to 1 and not a decrement.
You issue another ResendRequest for 11-20 and receive the app messages, each will increment your expected target seq no by 1.

I know that there are several users which use QFJ to connect to CME. The last issue around chunked resend requests I remember was QFJ-751 which was fixed in QF/J 1.6.0.

Please write a mail to the mailing list if you have further questions. https://lists.sourceforge.net/lists/listinfo/quickfixj-users
Or please attach a failing unit test so that I can reproduce the error.

Thanks.

Comment by John [ 07/Apr/16 ]

I can confirm that I am seeing the same behavior when connecting to the CME's Drop Copy 4.0. It appears this issue arises due to the way in which the CME increments tag 36 of SequenceReset messages and the way QuickFIX/J handles the sending of ResendRequest messages.

The following code is an excerpt from the Session class in the quickfix package which I believe is where the logic breaks down. It appears that QuickFIX/J will attempt to send chunks of ResendRequests depending upon when it receives incoming SequenceReset messages.The code below requires the value of the newSequence variable to be less than the EndSeqNo value defined in the range of messages that the client is supposed to fetch. This case fails and some ResendRequest messages will not be sent. Would it be safe to remove this logic from the if statement allowing subsequent RequestsRequests to be sent to the counterparty despite the fact that the newSequence value will be past the range of messages defined in the range variable? If so, the code would also need to be modified to not leverage the value of the newSequence variable when sending the ResendRequest to define the range of sequence numbers to send in the subsequent ResendRequest.

newSequence < range.getEndSeqNo()

private void nextSequenceReset(Message sequenceReset) throws IOException, RejectLogon,
FieldNotFound, IncorrectDataFormat, IncorrectTagValue, UnsupportedMessageType {
boolean isGapFill = false;
if (sequenceReset.isSetField(GapFillFlag.FIELD))

{ isGapFill = sequenceReset.getBoolean(GapFillFlag.FIELD) && validateSequenceNumbers; }

if (!verify(sequenceReset, isGapFill, isGapFill))

{ return; }

if (validateSequenceNumbers && sequenceReset.isSetField(NewSeqNo.FIELD)) {
final int newSequence = sequenceReset.getInt(NewSeqNo.FIELD);

getLog().onEvent(
"Received SequenceReset FROM: " + getExpectedTargetNum() + " TO: "
+ newSequence);
if (newSequence > getExpectedTargetNum()) {
state.setNextTargetMsgSeqNum(newSequence);
final ResendRange range = state.getResendRange();
if (range.isChunkedResendRequest()) {
if (newSequence >= range.getCurrentEndSeqNo()
&& newSequence < range.getEndSeqNo())

{ // If new seq no is beyond the range of the current chunk // and if we are not done with all resend chunks, // we send out a ResendRequest at once. // Alternatively, we could also wait for the next incoming message // which would trigger another resend. final String beginString = sequenceReset.getHeader().getString( BeginString.FIELD); sendResendRequest(beginString, range.getEndSeqNo() + 1, newSequence + 1, range.getEndSeqNo()); }

}
// QFJ-728: newSequence will be the seqnum of the next message so we
// delete all older messages from the queue since they are effectively skipped.
state.dequeueMessagesUpTo(newSequence);
} else if (newSequence < getExpectedTargetNum()) {

getLog().onErrorEvent(
"Invalid SequenceReset: newSequence=" + newSequence + " < expected="
+ getExpectedTargetNum());
if (resetOrDisconnectIfRequired(sequenceReset))

{ return; }

generateReject(sequenceReset, SessionRejectReason.VALUE_IS_INCORRECT,
NewSeqNo.FIELD);
}
}
}

Comment by Xiaojun Zhang [ 07/Apr/16 ]

John, I received your email. Do you have a non-personal email address? My corporate email doesn't allow sending to gmail, hotmail, etc.

Comment by Christoph John [ 08/Apr/16 ]

I very briefly read through http://www.cmegroup.com/confluence/display/EPICSANDBOX/Drop+Copy+Session+Layer+-+Resend+Request and I do not see anything that is contradictory to how QFJ interprets the NewSeqNo tag. However, it seems that CME is implementing something very custom. A test case would definitely help here.
One thing I noticed though, is that they mention "duplicate resend requests". Maybe you can try if the configuration SendRedundantResendRequests=Y helps in that case?

Comment by John [ 09/Apr/16 ]

Chrstoph,

What format would you prefer to see a test case in? I could provide the FIX messaging log so that it might be easier to see how the CME is increasing the sequence numbers in the SequenceReset messages they send in the middle of sending ResendRequest chunks past the end of the original ResendRequest range such that QuickFIX/J defines after logging on which causes some of the ResendRequest chunks to not be sent to the exchange.

As I mentioned in my earlier post, I believe the issue is in the code snippet I posted above where upon receiving SequenceReset messages QuickFIX/J will determine if a new ResendRequest chunk message needs to get sent out based on the following criteria in the nextSequenceReset method of the Session class by checking if newSequence < range.getEndSeqNo(). Since this fails, some ResendRequests fail to ever get sent out.

Say the defined ResendRequest range is 1-10000.
1) I send a ResendRequest for messages 1-2500. I receive these messages and then receive a SequenceReset to 2520 for example.
2) I then send a ResendRequest for messages 2501-5000. I receive these messages and then receive a SequenceReset to 5050.
3) I send a ResendRequest for messages 5001-7500. I receive these messages and then receive a SequenceReset of 11000.
At this point the newSequence < range.getEndSeqNo() test fails and QuickFIX/J never sends the final ResendRequest for messages 7501-10000. The FIX connection continues to receive real-time trades etc as per normal.

I would assume this to be custom logic that the CME is implementing on their end. I think the difference comes down to the fact that QuickFIX/J appears to rely more on the newSequence number that is received on an incoming SequenceReset message to dictate whether further ResendRequest chunks will be sent out and the CME may send out a SequenceReset message taking you past the originally defined EndSeqNo range.

I am not a FIX expert so I can't say if this solution would work but it seems that if the logic for sending chunks of ResendRequest messages is changed such that each of the ResendRequest chunk messages is defined immediately after the need for a ResendRequest arises and put into a queue (or some other object) QuickFIX/J could then pop the each ResendRequest message off the queue once it receives all of the messages for a given ResendRequest from the exchange and continue onto the next one and decouple itself in that way from the newSequenceNumber. This might ensure that any of the chunks of ResendRequest messages that need to get sent out don't get skipped. This might have negative implications as it might not be the standard logic for communicating via FIX however so it might not be a feasible solution however.

It would be really great if this functionality could be modified in QuickFIX/J as this is an extremely useful engine however for users attempting to use QuickFIX/J with CME's Drop Copy 4 this case will cause problems in a recovery scenario.

Comment by Christoph John [ 29/Apr/16 ]

Hi John,

yes, a message log would be a good starting point. From this I should hopefully be able to construct an automated test case.

Thanks, Chris.

Comment by Christoph John [ 26/Aug/16 ]

Hi John, Xiaojun Zhang: is there still the need to support this? Do you have any message logs?
Thanks, Chris.

Comment by John [ 01/Sep/16 ]

Hi Christoph John,

I can confirm that there is a need to support this functionality. To be clear, I was experiencing this issue when working with the 1.6.0 release of QuickFIX/J and have not tested with more recent releases. I don't have any message logs available currently aside from what Xiaojun posted earlier in this issue.

Thanks

Comment by Christoph John [ 01/Sep/16 ]

Hi John, OK, if this functionality was implemented could you test it against CME to verify if it works? Do they have some sort of certification test?

Comment by Dmitry Razumov [ 10/Oct/18 ]

Hey John
I know it's been two years, but did you pass the certification in the end?





[QFJ-889] Leading zeros in int and float fields Created: 23/Apr/16  Updated: 28/Apr/16

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Roman Liverovskiy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment:

Fedora 23 x86_64, Windows 7 64 bit SP1



 Description   

I use quickfixj-examples-executor-1.6.0.jar as FIX 5.0 server.

Server does not respond on any client messages if there are leading zeros in int or float fields in client message, for example if there are leading zeros in sequnce number in message header (in login message or in heart beat message).
Server application outputs information about client conection on socket, but there is no any response on FIX message sent from client to server (there is no even an error message).
Leading zeros are permitted by FIX 5.0 protocol specification.



 Comments   
Comment by Christoph John [ 28/Apr/16 ]

Could you please attach some logs? Thanks





[QFJ-867] Thread handling after exiting QuickFIX Created: 06/Nov/15  Updated: 16/Nov/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.1
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Martin Vrábel Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment:

Apache Tomcat 8 on Windows 10, Spring 4 web application



 Description   

Hi,

I stumbling over this thing over and over. When I stop my application I always get error messages in output about spawning a new thread during execution of application but failing to remove it when application stops:

```
06-Nov-2015 12:29:26.653 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@682d21a6]) and a value of type [quickfix.field.converter.UtcTimestampConverter] (value [quickfix.field.converter.UtcTimestampConverter@7f03e077]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
06-Nov-2015 12:29:26.654 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@682d21a6]) and a value of type [quickfix.field.converter.UtcTimestampConverter] (value [quickfix.field.converter.UtcTimestampConverter@6b6c6237]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
06-Nov-2015 12:29:26.654 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@682d21a6]) and a value of type [quickfix.field.converter.UtcTimestampConverter] (value [quickfix.field.converter.UtcTimestampConverter@2aca975d]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
06-Nov-2015 12:29:26.654 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@682d21a6]) and a value of type [quickfix.field.converter.UtcTimestampConverter] (value [quickfix.field.converter.UtcTimestampConverter@2bebb3b3]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
06-Nov-2015 12:29:26.655 SEVERE [localhost-startStop-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [ROOT] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@682d21a6]) and a value of type [quickfix.field.converter.UtcTimestampConverter] (value [quickfix.field.converter.UtcTimestampConverter@4b8bfee1]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
```

It has something to do with `UtcTimestampConverter` but I have no idea why is it happening. I correctly shutdown all threads I manually start.



 Comments   
Comment by Christoph John [ 16/Nov/15 ]

Hmm, looks more like a false positive to me. But to be honest I never tried running QFJ in a managed environment or appserver.
http://stackoverflow.com/questions/28105803/tomcat8-memory-leak says that the ThreadLocals need to be removed but I think this is not done because it is expected that QFJ is ran in a stand-alone environment.

Comment by Martin Vrábel [ 16/Nov/15 ]

Would it be possible to call a method or something from outside of quickfix to manually shutdown all spawned threads when exiting my webapp?

Comment by Christoph John [ 16/Nov/15 ]

From what I understand from the output it does not have to do anything with threads but ThreadLocal variables that are still held. But AFAIK there is no built-in mechanism to clean them up.





[QFJ-863] In the quickfix version that we use (1.5.3b), an invalid checksum error does not result in a session reject Created: 12/Oct/15  Updated: 12/Oct/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Saurabh Desai Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The scenario:

The initiator parses the following message with quickfix.MessageUtils.parse(), and sends it to the acceptor.

8=FIX.4.4|9=238|35=AE|31=1.35|32=1000|48=EURUSD|22=6|55=EURUSD|64=20131106|194=1.35|15011=1.33|15012=1.33|461=RCSXXX|487=2|571=1105200000001234|829=0|572=M1006050412587|552=1|54=1|37=SYS1:4545556002|11=SYS2:1006050412587|526=SSP:BDID001|453=2|448=a123456|447=D|452=3|802=2|523=5730607|803=10|523=5730606|803=15|1=123456-789|

It is not a valid FIX message, as we have 453<NoPartyIDs>=2, but there is only one group in the message. We would expect the acceptor to return a validation error, but instead the following happens:

1) An error message is logged:

2013-12-07 15:47:30,919 [SocketAcceptorIoProcessor-0.0] ERROR [] q.mina.acceptor.AcceptorIoHandler - Invalid message: Expected CheckSum=29, Received CheckSum=28 in 8=FIX.4.4?9=367?35=AE?34=82?49=SIDE1?52=20131207-14:47:30.618?56=SIDE2?22=6?31=1.35?32=1000?48=EURUSD?55=EURUSD?64=20131106?194=1.35?461=RCSXXX?487=2?571=1105200000001234?572=M1006050412587?829=0?15011=1.33?15012=1.33?552=1?54=1?37=SYS1:4545556002?11=SYS2:1006050412587?526=SSP:BDID001?453=2?448=a123456?447=D?452=3?802=2?523=5730607?803=10?523=5730606?803=15?1=123456-789?10=028?

2) The acceptor does not send any response and does not increase the message sequence count, either.

The suspect:

The acceptor validates the fix message with the checksum(String s) method of Message class. It calculates 29. But the checksum calculated on the initiator side is 28. The initiator calculates the checksum with a different method, which is called by the toString() method of Message class. This is the method which does the calculation (in FieldMap class) - my comments added:

int calculateTotal() {

int result = 0;
// JK: fields variable contains tags which are not group tags
for (final Field<?> field2 : fields.values()) {
final Field<?> field = field2;
if (field.getField() == CheckSum.FIELD || isGroupField(field.getField()))

{ continue; }

result += field.getTotal();
}

// JK: groups variable contains the group tags, but only the tag number (the key is Integer, not Field - why?)
final Iterator<Map.Entry<Integer, List<Group>>> iterator = groups.entrySet().iterator();
while (iterator.hasNext()) {
final Map.Entry<Integer, List<Group>> entry = iterator.next();
final List<Group> groupList = entry.getValue();
if (!groupList.isEmpty()) {
final IntField groupField = new IntField((entry.getKey()).intValue());
// JK: at this point we have 453=0, as this is a new field, no value yet
groupField.setValue(groupList.size());
// JK: now it is 453=1, because we have only one group
result += groupField.getTotal();
for (int i = 0; i < groupList.size(); i++)

{ final Group group = groupList.get(i); result += group.calculateTotal(); }

}
}

return result;
}

So the checksum will be the checksum of a different message actually, where 453=1.






[QFJ-852] Missing synchronized blocks discovered in code review. Created: 01/Jul/15  Updated: 04/Jul/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Nathan Tippy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: sequnece

Attachments: File quickfixjQFJ-852.patch    

 Description   

In the class Session in the following method

private boolean verify(Message msg, boolean checkTooHigh, boolean checkTooLow)

we find that synchronized (state.getLock()) is used to ensure atomic updates to the multiple fields in the state object. If this is needed then we must also check to ensure partial or dirty reads are avoided. This would happen any place that we call for more than one field of the state and assume the fields remain consistent with one another until we make a second or third request for other fields.

Directly after this same block the usage of isChunkedResendRequest, getCurrentEndSeqNo and getEndSeqNo run the risk of partial (dirty) reads because they are out side the sync.

This same problem also exists here where it may be the cause of other more serious issues.
private void nextSequenceReset(Message sequenceReset)

And to a a lesser degree in
private void doTargetTooHigh(Message msg)



 Comments   
Comment by Nathan Tippy [ 02/Jul/15 ]

For immediate use I apply this patch file to be safe. However this should be re-visited with a more elegant fix.

Comment by Christoph John [ 03/Jul/15 ]

Thanks for the patch!
Edit: did you encounter dirty read problems?

Comment by Nathan Tippy [ 04/Jul/15 ]

No I have not seen any problems but after discovery in code review I was uncomfortable running it without a fix.

I have not seen any negative consequences to applying the patch. So it was a defensive move.





[QFJ-847]  Transport Data Dict vs App Data Dict - settings not passed to App Data Dict Created: 19/May/15  Updated: 19/May/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Chris Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

N/a its in the java code



 Description   

The Quickfixj settings are not being passed to the ApplicationDataDictionary.

This seems to be because the transport dictionary is created via
DefaultSessionFactory.createDataDictionary.

Whereas the app DataDictionary is created in Default Data Dictionary
Provider, which does not seem to know about settings and so cannot pass them on.



 Comments   
Comment by Chris [ 19/May/15 ]

Not sure how best to address this - should the application DataDictionary be created when the Transport one is?

Or should the settings be made available to the DefaultDataDictionaryProvider class?





[QFJ-843] Some times QuickFix Message received on Fix session is processed first by Application thread and later by socket connection Processor Thread causing sequence error by later processing Created: 04/May/15  Updated: 07/May/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: anurag jain Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: sequnece, session


 Description   

Message on Fix session received by both Application thread (ProcessQ) and socket connection Processor Thread.
In below example Our Application Thread ProcessQ got the message with 34=158757 and later quickfix thread ScoketConnectionProcessor-0.0 Thread process the same message this happen only 4 time in a whole day.
But Due to this when quickfix's socketConnectionThread thread process the message and give it to QFJMessageProcessor thread to process further it encounters sequence rest error as a result session logged out triggers. In all the other scenarios ScoketConnectionProcessor process the message first and delegate the responsibility to QFJ Message Processor thread and later to application Thread i.e. ProcessQ

Please FInd application logs as below ..
2015-04-28 14:16:03,856 [ProcessQ] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A
2015-04-28 14:16:03,856 [SocketConnectorIoProcessor-0.0] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1
^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A



 Comments   
Comment by anurag jain [ 04/May/15 ]

Please note that ProcessQ Thread is application thread which read objects from application Queue and not the the Quick Fix internal queue .

Comment by Christoph John [ 04/May/15 ]

When this happens, do you have to restart your application or does the problem correct itself? Are you starting/stopping your Initiator/Acceptor at certain times? This sounds a little like QFJ-825.
Would it be possible to create a stack dump with the program "jstack" once the error appears?

Comment by anurag jain [ 04/May/15 ]

Sorry rephrasing the above JIRA:

  • We noticed 2 threads logging the string --> "quickfixj.msg.incoming" at the same time. One of the threads to log this was SocketConnectorIoProcessor-0.0, Surprisingly the second thread which logs this is the ProcesQ thread
  • The ProcessQ thread is our own application thread so we are not sure how this could happen?
  • This does not seem to be a logging issue, we are using log4j2 as our logger. I say that because of the 2 consecutive messages received one after the other, Quickfix/J complains when the second message comes in:
    2015-04-28 14:16:03,857 [QFJ Message Processor] INFO  quickfixj.msg.outgoing - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=141^A35=5^A34=11955^A49=MDAQ_DEV2_SANDPIT_MD^A52=20150428-06:16:03.857^A56=ANZD_SANDPIT_MD^A58=MsgSeqNum too low, expecting 158758 but received 158757^A10=030^A
    

All the log files below:

2015-04-28 14:16:03,856 [ProcessQ] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A


2015-04-28 14:16:03,856 [SocketConnectorIoProcessor-0.0] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1
^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A

2015-04-28 14:16:03,857 [QFJ Message Processor] INFO  quickfixj.msg.outgoing - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=141^A35=5^A34=11955^A49=MDAQ_DEV2_SANDPIT_MD^A52=20150428-06:16:03.857^A56=ANZD_SANDPIT_MD^A58=MsgSeqNum too low, expecting 158758 but received 158757^A10=030^A

Could you please tell us:

  • If this is a bug in 1.5.3 version of Quickfix/J?
  • We only notice this happening 4 times a day during peak load, 2000 market data messages a second
  • We did not seem to encounter this in 1.5.2
Comment by Christoph John [ 05/May/15 ]

Could you please answer my questions from the last comment?
As already said, the only thing I could imagine is that you are hitting QFJ-825. But to be sure I would need a stack dump.
Are you able to try without log4j2? It's probably not the problem, but just to be sure... Are you using asynchronous logging?

Comment by anurag jain [ 06/May/15 ]

Please find my answer here:

  • No we are not stopping are starting Initiator/Acceptor at certain times.
  • Also When this happened, we didn't restart the application as mention above due to this we get the sequence error as a result of that log-out has sent to target session after a while it connect back again as part of login retry attempts.
  • This is not happening daily basis, we are trying to reproduce the problem at our end if succeeds we will share the JStack's stack dump with you. However No luck so far.
  • We are using synchronous logging also this is unlikely to have log4j2 issue as its result in sequence error.
Comment by Christoph John [ 06/May/15 ]

OK, without the stack dump I cannot really tell anything. Just let me know once you have it.
More questions: are you using a ThreadedSocketAcceptor/Initiator? Or the default SocketAcceptor/Initiator? is there only one session that you have configured? Could you please post your session config?

And I guess you are certain that the counterparty did not just send the same message twice?

Comment by anurag jain [ 07/May/15 ]

1) We are using default Socket Initiator.
2) Have just one session .
3) Session config is as here
DataDictionary=config/FIX42.xml
ResetOnLogon=Y
ScreenLogShowOutgoing=N
FileLogHeartbeats=N
ResetOnDisconnect=Y
ScreenLogEvents=N
ConnectionType=initiator
SocketKeyStore=config/SummitInvt.pem
TargetCompID=ANZD_SANDPIT_MD
SenderCompID=MDAQ_DEV2_SANDPIT_MD
SocketConnectHost=1.x.x.x
ReconnectInterval=10
ScreenLogShowIncoming=N
SocketConnectPort=xxxx
SocketUseSSL=Y
FileStorePath=var/fix/fxlp
PersistMessages=N
EndTime=17:00:00
SocketKeyStorePassword=*******
TimeZone=America/New_York
StartTime=17:10:00
HeartBtInt=20
UseDataDictionary=Y
ResetOnLogout=Y
Username=TODO
e-mail=
SessionID= FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD

4)and yes there is no replay of same message from counter party .

Comment by Christoph John [ 07/May/15 ]

Hmm, now that I see that you are using SSL I am asking myself if it could be the same problem as in QFJ-745. However, there is not really much information to find the problem and the issue creator did not come back with answers.





[QFJ-793] When RuntimeException is thrown within a call-back and DisconnectOnError=Y the session does not get disconnected automatically Created: 12/Jun/14  Updated: 23/Aug/16

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File SessionTest.java.add     File SessionTest.java.add.fixed    

 Description   

Hi,

When a RuntimeException is thrown within a call-back and the "DisconnectOnError" parameter is set to "Y" the session does not get disconnected automatically unless the "RejectMessageOnUnhandledException" parameter is also set to "Y". This dependency is not documented, but even if it was, it wouldn’t make too much sense.

It seems to be logical to understand an occurrence of a RuntimeException within a call-back as an error condition on its own, which should be sufficient to trigger the disconnection logic dictated by the "DisconnectOnError" parameter (regardless of any other settings).

Currently, if RejectMessageOnUnhandledException=N, the engine simply ignores the message for which the exception was thrown even though DisconnectOnError=Y (although it reports the exception as an “error” in the log).

Test code attached.

Please note that probably all call-backs exhibit the same behaviour, not just the fromApp() call-back used in the TestApp class. It is also possible that the “ResetOnError” parameter is treated in a similar manner.

Regards,
Andrzej Hajderek



 Comments   
Comment by Andrzej Hajderek [ 13/Jun/14 ]

Attached a fixed version of the test.





[QFJ-798] Refactor Session construction in test code Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Currently to create some of the Session related test cases it is necessary to use constructs like this:

Session session = new Session(new UnitTestApplication(), new MemoryStoreFactory(),
sessionID, null, null, new ScreenLogFactory(true, true, true),
new DefaultMessageFactory(), isInitiator ? 30 : 0, false, 30, true, resetOnLogon,
false, false, false, false, false, true, false, 1.5, null, validateSequenceNumbers,
new int[]

{ 5 }

, false, disconnectOnError, false, true, false, true, false, null,
true, 0, false, false);

This kind of code turns from an asset into a liability pretty quickly, so the sooner it is rationalised the better. The Session constructor itself isn't very pretty either with its 30+ parameters.






[QFJ-797] Add a clean and safe mechanism to disconnect a session from within a call-back Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Hi,

There is currently no clean and safe mechanism to completely stop and disconnect a session from within a call-back (e.g. when processing an incoming message inside fromApp()) with the effect that the session is immediately disconnected, no reject messages are sent back to the counter-party and no change to the session store is made.

Such a mechanism is useful for implementing failure handling scenarios, e.g. when a target database becomes suddenly unavailable, or the application needs to be shutdown in emergency. In such cases it's best to close the TCP socket immediately and keep the session state unchanged.

One could try to use something like SocketInitiatior.stop(true), but it's not generic (does not work for acceptors) and it results in a call to Session.disconnect(). Then, if Session.disconnect() is called from fromApp(), we get an immediate call to the onLogout() callback - a call is made from within the fromApp() call-back into the onLogout() call-back. This isn't a particularly safe scenario, especially when in most cases the onLogout() call is made by from QuickFIX/J threads.

Another solution is to throw a RuntimeException within the call-back method, but it's not reliable right now (QFJ-793, QFJ-795) and makes QuickFIX/J send an error message and the stack trace into the log.

I propose to add a special a subclass of RuntimeException, e.g. DisconnectSessionExcepion, which when thrown from a call-back method would notify QuickFIX/J that the session should be disconnected immediately. In such case only an informational message would be send to the log instead of error with a stack trace.

Regards,
Andrzej Hajderek






[QFJ-796] There is no mechanism to clear or update the data dictionary cache Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

A data dictionary file loaded into memory is cached without any option to reload the file after its contents has been updated. Worse - it's not even possible to clear the cache. This means that in order to use an updated data dictionary it is necessary to restart the application.

See: quickfix.DefaultSessionFactory.dictionaryCache






[QFJ-794] Misleading message logged when RuntimeException thrown within a call-back: "Rejecting message: ..." Created: 12/Jun/14  Updated: 23/Aug/16

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

This is in relation to QFJ-793.

When both settings RejectMessageOnUnhandledException and DisconnectOnError are set to Y and a RuntimeException is thrown within a call-back, QuickFIX/J will write to the log output messages like below:

<20140612-16:25:06, FIX.4.4:SENDER->TARGET, error> (Rejecting message: java.lang.RuntimeException: ... (with more details and the message)

2014-06-12 18:25:06 quickfix.Session disconnect

INFO: [FIX.4.4:SENDER->TARGET] Disconnecting: Auto disconnect

This is misleading because nothing is actually rejected (no reject message is sent back to the sender). The session is simply disconnected. I believe a slightly different wording should be used for this scenario, e.g "Failed to process message" instead of "Rejecting message".






[QFJ-801] Validation fail on ClassCastException if using XML Created: 14/Jul/14  Updated: 14/Jul/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Benoit Xhenseval Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: validation
Environment:

Mac OSX


Attachments: Java Source File IOIValidation.java    

 Description   

It appears that the Message validation fails on a ClassCastException if the message contains the XmlDataLen tag.

Example:
https://gist.github.com/benoitx/e7da15dba9da133ac7b9

It seems that the validation code only expects StringField but the XmlDataLen is an IntField.

The output is:

Testing WITHOUT XML
8=FIXT.1.19=7935=649=Sender56=Target15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=079
Testing WITH XML
8=FIXT.1.19=10335=649=Sender56=Target212=12213=<a>Hello</a>15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=086
Exception in thread "main" java.lang.ClassCastException: quickfix.field.XmlDataLen cannot be cast to quickfix.StringField
	at quickfix.DataDictionary.iterate(DataDictionary.java:668)
	at quickfix.DataDictionary.validate(DataDictionary.java:653)
	at quickfix.DataDictionary.validate(DataDictionary.java:624)
	at quickfix.DataDictionary.validate(DataDictionary.java:606)
	at fixfun.IOIValidation.validateIoi(IOIValidation.java:45)
	at fixfun.IOIValidation.validateIoiWithXml(IOIValidation.java:37)
	at fixfun.IOIValidation.main(IOIValidation.java:78)


 Comments   
Comment by Benoit Xhenseval [ 14/Jul/14 ]

test case.

Comment by Christoph John [ 14/Jul/14 ]

It turned out that the setField( int, Field<?> ) method should rather not be used with non-String tags. Rather use setField( Field ), e.g.:
ioi.getHeader().setField(new XmlData(xml));
ioi.getHeader().setField(new XmlDataLen(xml.length()));

Of course, there should be means to prevent your original error. Either setField( int, Field<?> ) should convert the values (as setField( Field ) does) or the validate-method should not assume that all fields are Strings.





[QFJ-743] JdbcLog doesn't take into consideration the sessionID to verify if a parameter is set like JdbcMessage does Created: 13/May/13  Updated: 06/Nov/15

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Leandro Garcia Herrera Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: JdbcLog, QuickfixJ, jdbc
Environment:

Windows 7, using JBoss and camel-quickfix


Attachments: Java Source File JdbcLog.java     Java Source File TemaJdbcLog.java    

 Description   

I'm trying to configure JdbcLogFactory dynamically passing a SessionSettings object as argument to constructor. All my configurations are under an especific session ID and I have no configurations under default session.

Looking at the source code of JdbcLog I was able to confirm that only the default session configurations are taken into consideration instead of checking first in session ID configurations informed via constructor, wich I was expecting.

The JdbcStore class does this verification by session ID and I think JdbcLog should work in the same way.



 Comments   
Comment by Christoph John [ 09/Jan/14 ]

Would it be OK for you to attach the patch to this ticket? I do not have the necessary rights to grant you SVN access but could integrate the patch shortly.

Comment by Leandro Garcia Herrera [ 06/Nov/15 ]

Hi, Christoph. Look if it helps. I've attached the patch as you requested.

Best regards.

Comment by Leandro Garcia Herrera [ 06/Nov/15 ]

bug fix





[QFJ-733] Provide customisable hooks into the core QuickFIX/J workflow Created: 11/Mar/13  Updated: 21/Mar/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Ryan Lea Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, modules, performance


 Description   

Background:
I have been using QuickFIX for some time and found it a great library. As my usage has increased, I have become more aware that it's not necessarily GC-friendly. As such I'm in the process of extending the Message object to become more GC-friendly, but really a step away from the message type-safety that is currently present within QuickFIX/J.

In order to achieve my goal, I have made (what I consider) minor modifications to the core codebase to provide locatable service lookups where need be,

Improvement:
It would be great if QuickFIX/J was more customisable in the services that were able to be overriden/extended or generally customised. I've currently created a (very simple - based on http://martinfowler.com/articles/injection.html) ServiceLocator that prepares services with defaults (current 1.5.3 behaviour) that can be overriden with custom services on startup (if required).

The services that I have currently customised are:

  1. Message validation with the Data Dictionary
  2. Message creation from String (the FIXMessageDecoderFactory)
  3. Handling of FIX Messages within the abstract IoHandler (as it depends upon MINA passing it a String)

This approach seemed less intrusive than a DI framework, and perhaps a simpler fit into QuickFIX core. It provides the benefits of maintaining a library that can be used out of the box, whilst providing the capability of customisation where required or necessary.

I'd be mostly interested to know whether this (or something like it) would be considered as part of the roadmap with QuickFIX/J and how or whether I would be able to contribute.



 Comments   
Comment by Jörg Thönnes [ 15/Mar/13 ]

Dear Ryan,

we appreciate your idea and would like to consider it.
It is true the QF/J is quite monolithic and would greatly benefit from being split into more components.
E.g.

  • separate IO layer in order to allow to replace MINA by Netty, etc (QFJ-598)

But currently there is no roadmap what should be done.

But it would be good if you could start a page in QF/J Confluence to outline your thoughts
and perhaps attached some code.

Thanks, Jörg

Comment by Ryan Lea [ 21/Mar/13 ]

Hi Jorg,

Thanks for your feedback. I will certainly attach a patch once I've extracted out the pieces I need to.

Following that, I will look at creating a Confluence page as well (quite a few of the Confluence pages seem quite old as well I've noticed). Do you have a preference of which hierarchy you would like it placed in?

Ryan





[QFJ-729] Allow MessageCrackers to handle polymorphic Message types Created: 07/Feb/13  Updated: 07/Feb/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: S Raf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The MessageCracker expects an invoker to be registered for the specific actual Message type being processed. If an invoker exists for a supertype of the received Message, the invoker won't be called.

For those extending MessageCracker, the onMessage(quickfix.Message message, SessionID sessionID) method can be overridden to allow implementers to handle the most general quickfix.Message type (but not, for example, only quickfix.fix44.Message messages).

However, for those providing a delegate object to the MessageCracker, there is no option to provide a handler for a more general Message type to provide custom fallback logic.

The crack(..) method should look for invokers registered for the received Message's superclass(es), as well as its actual type, before resorting to the fallback onMessage call in MessageCracker.






[QFJ-762] Message stores can become corrupted on acceptor/initiator shutdown Created: 09/Dec/13  Updated: 21/Apr/17

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Alexey Ermakov Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None

Issue Links:
Relates
relates to QFJ-552 Message Stores expected to be thread ... Open
is related to QFJ-923 FileStore is leaking file handles Closed

 Description   

Since MessageStore implementations aren't thread safe, all operations on them should be performed from the session thread only. However, all 4 acceptor and initiator implementations perform shutdown (and thus call Session.unregisterSessions) in the calling thread. Since that calls MessageStore.close(), it can easily lead to store corruption if, for example, FileStore.close() gets called while the session thread is in the middle of persisting a Logout response.



 Comments   
Comment by Constantin Florescu [ 20/Apr/17 ]

The same issue could be reproduced when the logon request is timing out.

There are 2 threads trying to do the same thing:
16:18:28.763 [QFJ Timer] INFO quickfix.FileStore - initialize() called from.
at quickfix.FileStore.initialize(FileStore.java:111) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.FileStore.reset(FileStore.java:465) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.SessionState.reset(SessionState.java:382) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.resetState(Session.java:2503) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.disconnect(Session.java:1968) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.next(Session.java:1808) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:278) [quickfixj-core-1.6.2.jar:1.6.2]

And from:
16:18:28.788 [QFJ Message Processor] INFO quickfix.FileStore - initialize() called from.
at quickfix.FileStore.initialize(FileStore.java:111) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.FileStore.reset(FileStore.java:465) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.SessionState.reset(SessionState.java:382) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.resetState(Session.java:2503) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.disconnect(Session.java:1968) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.next(Session.java:882) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.next(Session.java:1109) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:144) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:91) [quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:125) [quickfixj-core-1.6.2.jar:1.6.2]

This causes unexpected behaviour and leaks File Descriptors.
Example:
ERROR [QFJ Timer] quickfix.SocketInitiator Error during timer processing
quickfix.RuntimeError: java.io.FileNotFoundException: [/..../Session].header (Too many open files)
at quickfix.SessionState.reset(SessionState.java:384) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.resetState(Session.java:2499) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.disconnect(Session.java:1967) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.Session.next(Session.java:1808) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:278) [quickfixj-core-1.6.2.jar:1.6.2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_74]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_74]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_74]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_74]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_74]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_74]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]
Caused by: java.io.FileNotFoundException: [/.../Session].header (Too many open files)
at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_74]
at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_74]
at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_74]
at java.io.FileOutputStream.<init>(FileOutputStream.java:133) ~[?:1.8.0_74]
at quickfix.FileStore.initializeMessageIndex(FileStore.java:198) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.FileStore.initializeCache(FileStore.java:121) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.FileStore.initialize(FileStore.java:116) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.FileStore.reset(FileStore.java:442) ~[quickfixj-core-1.6.2.jar:1.6.2]
at quickfix.SessionState.reset(SessionState.java:382) ~[quickfixj-core-1.6.2.jar:1.6.2]
... 11 more





[QFJ-780] QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect Created: 01/May/14  Updated: 23/Aug/16

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Tarun Bahadur Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect

After a resend request is received from server subsequent messages should be ignored unless SequenceReset is received or messages replayed. In below test case the logout message is sent with sequence number 6, this should actually be ignored by the server and the test case should fail. This would require fixing both QuickFixJ behavior and the test case(probably logout should have MsgSeqNum=1).

from fix42/1a_ValidLogonMsgSeqNumTooHigh.def
...
E8=FIX.4.29=5835=234=249=ISLD52=00000000-00:00:00.00056=TW7=116=410=0
I8=FIX.4.235=534=649=TW52=<TIME>56=ISLD
...



 Comments   
Comment by Christoph John [ 05/May/14 ]

This is related to the QF/J-specific behaviour to always accept a Logout message regardless of the sequence number. But it probably would not hurt to change this in order to be more in line with the FIX spec.





[QFJ-659] Sequence number issues when acceptor sends a message immediately following logon response Created: 09/Jan/12  Updated: 09/Jan/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0, 1.5.2
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrew Richards Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: logon, sequnece


 Description   

I have an initiator connection to a foreign acceptor which has sequence number confusion on logon as the foreign acceptor sends a TestRequest immediately after the logon response.

It looks like the following is happening: Quickfix/J accepts both (Logon & TestRequest) messages. The and proceeds to check the sequence number of the Logon to make sure the session is valid. It does this via Session.isTargetTooHigh() which looks at the last message in the message store which is now 1 higher than is should be because the TestRequest has been received and stored. The "MsgSeqNum too high" is issued and the engine tries to recover. In some most circumstances, but not all it (with the engine in question) it can recover however it's not ideal.

The effect can be simulated (QuickFIX/J -> QuickFIX/J) by using and acceptor with the toAdmin below.

public void toAdmin(Message message, SessionID sessionID)
{
print("toAdmin", message, sessionID);

try {
Header header = message.getHeader();
if ( header.getField( new MsgType() ).valueEquals( MsgType.LOGON ) )

{ quickfix.fix44.TestRequest tst = new quickfix.fix44.TestRequest(new TestReqID("TEST")); Session.sendToTarget(tst, sessionID); }

}
catch (Exception e) {}
}

09-Jan-2012 08:49:19 INFO : Initialize
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 SEVERE : FIX.4.4:CLIENT->SERVER: MsgSeqNum too high, expecting 2 but received 3: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Enqueued at pos 3: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 INFO : [toAdmin](FIX.4.4:CLIENT->SERVER)[2](MsgType:ResendRequest (2))
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=6435=234=249=CLIENT52=20120109-08:49:50.01956=SERVER7=216=010=238
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Sent ResendRequest FROM: 2 TO: 2
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=9335=434=243=Y49=SERVER52=20120109-08:49:50.02556=CLIENT122=20120109-08:49:5036=4123=Y10=182
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: ResendRequest for messages FROM 2 TO 2 has been satisfied.
09-Jan-2012 08:49:50 INFO : [fromAdmin](FIX.4.4:CLIENT->SERVER)[2](MsgType:SequenceReset (4))
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Received SequenceReset FROM: 2 TO: 4
09-Jan-2012 08:50:20 INFO : [toAdmin](FIX.4.4:CLIENT->SERVER)[3](MsgType:Heartbeat (0))
09-Jan-2012 08:50:20 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=349=CLIENT52=20120109-08:50:20.21556=SERVER10=100
09-Jan-2012 08:50:20 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=449=SERVER52=20120109-08:50:20.21856=CLIENT10=104
09-Jan-2012 08:50:20 INFO : [fromAdmin](FIX.4.4:CLIENT->SERVER)[4](MsgType:Heartbeat (0))






[QFJ-657] TestRequest should be sent before a Logout is sent. (Initiator) Created: 05/Dec/11  Updated: 06/Dec/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0, 1.5.1
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Ken Douglas Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Logout, testRequest
Environment:

Windows 7 Ultimate, Java 6 u29
QuickFix J 1.5.1



 Description   

TestRequest should be sent before a Logout is sent (Initiator).
You may not agree this is a bug by the way but perhaps a feature not currently supported but the FIX Protocol recommends the following:-

[1] a TestRequest is sent and a hearbeat is expected to be sent in return
[2] NB No logout is sent until a heartbeat (with the TestRequest ID) is received.
[3] Once receive a logout is sent and waitfor Logout is carried out.
[4] then disconnect is done.

I have seen a question relating to this on the forum but this goes way back
http://quickfix-j.364392.n2.nabble.com/Log-file-rotation-td665743.html

Currently the way the reset() on the Session is done it does not allow you to fire a TestRequest off and wait for the hearbeat.

Any idea guys?

Thanks



 Comments   
Comment by Christoph John [ 06/Dec/11 ]

I think this would be a good improvement. BTW, the link to the forum thread is wrong. It leads to a discussion about log rotation.





[QFJ-610] java.io.IOException when using FileStore Created: 17/Jun/11  Updated: 17/Jun/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Daniel Clusin Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: QuickfixJ
Environment:

uname -a: Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:56:28 EST 20 06 x86_64 x86_64 x86_64 GNU/Linux
Java VM: BEA JRockit(R) R27.6.0-50_o-100423-1.5.0_15-20080626-2104-linux-x86_64,BEA Systems, Inc.



 Description   

java.io.IOException: Bad file descriptor
at java.io.RandomAccessFile.seek(Native Method)
at quickfix.FileStore.storeSequenceNumbers(FileStore.java:403)
at quickfix.FileStore.incrNextSenderMsgSeqNum(FileStore.java:288)
at quickfix.SessionState.incrNextSenderMsgSeqNum(SessionState.java:350)
at quickfix.Session.sendRaw(Session.java:2177)
at quickfix.Session.generateHeartbeat(Session.java:1786)
at quickfix.Session.next(Session.java:1759)
at quickfix.Session.next(Session.java:1032)
at quickfix.mina.ThreadPerSessionEventHandlingStrategy$MessageDispatchingThread.run(ThreadPerSessionEventHandlingStrategy.java:119)

This occurred during a period of intermittent network disconnects. I did not see any forcible closes of the network sockets being done. Another time this occured is during a shutdown, after we called Initiator/Acceptor.stop

FIX.4.2:XXXX->XXXX: Error during message processing

quickfix.RuntimeError: java.io.IOException: Bad file descriptor
at quickfix.SessionState.reset(SessionState.java:373)
at quickfix.Session.resetState(Session.java:2193)
at quickfix.Session.nextLogout(Session.java:1279)
at quickfix.Session.next(Session.java:941)
at quickfix.mina.ThreadPerSessionEventHandlingStrategy$MessageDispatchingThread.run(ThreadPerSessionEventHandlingStrategy.java:119)

Caused by: java.io.IOException: Bad file descriptor
at java.io.RandomAccessFile.length(Native Method)
at quickfix.FileStore.initializeCache(FileStore.java:119)
at quickfix.FileStore.initialize(FileStore.java:111)
at quickfix.FileStore.reset(FileStore.java:436)
at quickfix.SessionState.reset(SessionState.java:370)
... 4 more






[QFJ-619] Late Resend Request causes sequence number mayhem : what happened? Created: 18/Jul/11  Updated: 18/Jul/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Other Priority: Default
Reporter: Nicolas Chaufette Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File trace.txt    

 Description   

Hello everyone,

A client seems to have a problem that I cannot reproduce. Basically, the third party asks for a resend request of a message sent 7 minutes earlier. During those 7 mintures, heartbeats have been exchanged and the sequence increased. When the Resend Request is received, this happens (look at the sequence numbers).
Is it a known bug in QuickFIX/J 1.4.0 ? fixed in 1.5.0 ?

Thanks, best regards,

Nicolas

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:30:40.704" level="Enabled" thread="QFJ Timer">
<fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=260|49=NOYOBU|52=20110705-11:30:40.704|56=NOYOBU|10=215|]]></fix:message>
</fix:trace>

[... nothing of importance, just heartbeats in and out for 7 minutes...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.695" level="Enabled" thread="QFJ Timer">
<fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=267|49=NOYOBU|52=20110705-11:37:46.695|56=NOYOBU|10=244|]]></fix:message>
</fix:trace>

[... here seqnum is 267, right? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.788" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
<fix:message length="88" way="incoming"><![CDATA[8=FIX.4.4|9=0064|35=2|49=NOYOBU|56=NOYOBU|34=269|52=20110705-11:37:46|7=260|16=0|10=102|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.788" level="Enabled" thread="QFJ Message Processor">
<fix:event length="45"><![CDATA[Received ResendRequest FROM: 260 TO: infinity]]></fix:event>
</fix:trace>

[... counterparty asks for 260, sent 7 minutes earlier ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.185" level="Enabled" thread="QFJ Message Processor">
<fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=260|43=Y|49=NOYOBU|52=20110705-11:37:47.185|56=NOYOBU|122=20110705-11:37:47|36=261|123=Y|10=150|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
<fix:event length="26"><![CDATA[Sent SequenceReset TO: 261]]></fix:event>
</fix:trace>

[... sending 260, reset to 261 ?? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
<fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=260|43=Y|49=NOYOBU|52=20110705-11:37:47.186|56=NOYOBU|122=20110705-11:37:47|36=267|123=Y|10=157|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
<fix:event length="26"><![CDATA[Sent SequenceReset TO: 267]]></fix:event>
</fix:trace>

[... sending 260 AGAIN, reset to 267 ????? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:46.819" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
<fix:message length="77" way="incoming"><![CDATA[8=FIX.4.4|9=0053|35=0|49=NOYOBU|56=NOYOBU|34=270|52=20110705-11:38:46|10=121|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.699" level="Enabled" thread="QFJ Timer">
<fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=268|49=NOYOBU|52=20110705-11:38:47.698|56=NOYOBU|10=250|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.831" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
<fix:message length="88" way="incoming"><![CDATA[8=FIX.4.4|9=0064|35=2|49=NOYOBU|56=NOYOBU|34=271|52=20110705-11:38:47|7=261|16=0|10=098|]]></fix:message>
</fix:trace>

[... and it goes on and on and on and on...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.831" level="Enabled" thread="QFJ Message Processor">
<fix:event length="45"><![CDATA[Received ResendRequest FROM: 261 TO: infinity]]></fix:event>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.187" level="Enabled" thread="QFJ Message Processor">
<fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=261|43=Y|49=NOYOBU|52=20110705-11:38:48.187|56=NOYOBU|122=20110705-11:38:48|36=262|123=Y|10=158|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
<fix:event length="26"><![CDATA[Sent SequenceReset TO: 262]]></fix:event>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
<fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=261|43=Y|49=NOYOBU|52=20110705-11:38:48.188|56=NOYOBU|122=20110705-11:38:48|36=268|123=Y|10=165|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
<fix:event length="26"><![CDATA[Sent SequenceReset TO: 268]]></fix:event>
</fix:trace>






[QFJ-623] How do you change CompIDs, IP and port of a session at runtime? Created: 26/Jul/11  Updated: 19/Oct/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Other Priority: Default
Reporter: Andrew Niland Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment:

java 1.6 on Windows



 Description   

How do you change CompIDs, IP and port of a session at runtime?

I would like to repoint the initiator engine to a different FIX server while the java app is still running.



 Comments   
Comment by Grant Birchmeier [ 26/Jul/11 ]

Did you try posting your question to the mailing list? This doesn't look like it should be a bug report.
http://quickfixj.org/support/

Comment by Laurent Danesi [ 19/Oct/11 ]

CompIDs are part of the SessionID, if you change them it is not the same Session anymore.
About the IP/port, you can specify multiple ip/port for the session and switch to them.

Regards,

Laurent DANESI





[QFJ-558] Can no longer send messages to initiators that have not yet been started Created: 13/Sep/10  Updated: 16/Sep/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File diff.txt     Text File diff.txt    

 Description   

We have some system states where we disable network connection but still may have messages that we wish to deliver to the gateway, i.e., message store, for future delivery to the counterparty when we re-enable the network connection. Our technique is to create a SocketAcceptor or SocketInitiator, but not to start it. Since we upgraded from QF/J 1.2.1 to QF/J 1.4.0, this is no longer working for initiators.

Traced this back to QFJ-354, revision number 874. QFJ-354 allows a SocketInitiator to be stopped and restarted. (Previously it could be stopped, but the only way to restart it was to recreate it.) As part of this change, they moved where the sessions are created out of the constructor. (Session creation is one way a session can be registered.) QFJ-354 was part of the 1.4.0 release. There was no similar change for acceptors.

I don't see any reason why the session creation cannot be moved back into the constructor. It does not interfere with the intent of QFJ-354.
Patch attached.

While in the neighbourhood, noticed the following two lines in the AbstractSocketInitiator constructor:

ByteBuffer.setAllocator(new SimpleByteBufferAllocator());
ByteBuffer.setUseDirectBuffers(false);

This should be some sort of static initialization. e.g., for applications that use Mina for their own purposes as well as QF/J, they may need control over the buffer type and allocator, and shouldn't have to worry about QF/J changing it on the fly.



 Comments   
Comment by Rhys Yarranton [ 16/Sep/10 ]

Updated diff includes mod to ThreadedSocketInitiator.java.





[QFJ-555] Problems with heartbeating in one session can obstruct heartbeating on other sessions Created: 01/Sep/10  Updated: 07/Jan/20

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to QFJ-291 Problems with initiator reconnect wil... Closed

 Description   

SessionConnector.scheduledExecutorService is a single-threaded executor, and is used by all sessions for heartbeating. If there is a problem in one session, it can impact other sessions.

This is really QFJ-291 again. QFJ-291 was stated in terms of initiator reconnects, and the fix to it targeted that specific scenario. However, we have just encountered this problem in production involving only heartbeats. Heartbeating seems innocuous, but in fact has most of the same vulnerabilities as sending any other message (logging, toAdmin etc.).



 Comments   
Comment by Christoph John [ 07/Jan/20 ]

This will be improved with https://github.com/quickfix-j/quickfixj/issues/254.





[QFJ-531] To set the number of logon attempts Created: 23/Jun/10  Updated: 23/Jun/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: Future Releases
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Pranab Kumar Naik Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment:

Linux



 Description   

I was trying to set up the number of failed logon attempts after which the logon session would be created with a different host. The configuration does match with the interval between reconnects, but I thought it would be useful if the number of reconnect attempts may also be configurable from the config file. Or is there an option that I maybe missing ?






[QFJ-583] Session.forceStoreResync doesn't check persistMessages Created: 31/Mar/11  Updated: 31/Mar/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Dmitri Lenna Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows & Linux.



 Description   

We set the "ForceResync" to true and "PersistMessages" to false. Attempting connect to a target with a different sequence number, we get a logout message with the expected sequence number and the following is called:

(in Session.java):
private void forceStoreResync(int nextTargetMsgSeqNum, int nextSenderMsgSeqNum )

in that method, a heartbeat message is created for each missing sequence number and it's stored in the Message Store without checking if persisteMessage is true.

Is there a reason for this or this a bug? If it's a bug, I'd suggest this fix:

Line 1601:

  • if (actualSenderMsgSeqNum < nextSenderMsgSeqNum) {
    + if (actualSenderMsgSeqNum < nextSenderMsgSeqNum && persistMessages ) {

Thank you.






[QFJ-529] RTT benchmarking Created: 05/Jun/10  Updated: 05/Jun/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Wayne Zhu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

windows 7 professional 64 bits, multiple core Intel



 Description   

I have tested with a simple server and client setup using a single session in the same machine(tcp socket on local host). The client sent in new orders and got execution reports back. If the orders were sent at a high data rate for a period of 100-200ms, the round trip time for a new order and the corresponding report was not good.

My understanding is that there are two different threads in sending and receiving messages. What could be the reason causing QuickFix engine to slow down processing the incoming messages? Since there is no network overhead, it's most likely to be caused by the decoder and the way FIX message is constructed.






[QFJ-521] Support use of partial session ID for routing purposes Created: 10/May/10  Updated: 09/Jun/11

Status: Reopened
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.1, 1.3.2, 1.3.3, 1.4.0
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Rhys Yarranton Assignee: Grant Birchmeier
Resolution: Unresolved Votes: 1
Labels: None

Attachments: Text File diff3.txt    

 Description   

QFJ-272/1.3.1 added several fields to SessionID (SenderSubID, TargetSubID, SenderLocationID). These fields will be populated in MessageUtils.getReverseSessionID(...) if they are set in the Login message. They then form part of the session look-up in AbstractSocketAcceptor.StaticAcceptorSessionProvider.

In 1.2.1 and below, MessageUtils.getReverseSessionID included only the comp IDs, not the sub-IDs and SenderLocationID.

This breaks us in two ways. One is that we have been configuring our sessions by the comp IDs, which works well with the 1.2.1 code but mismatches with the 1.3.1+ code even if both sides get all their IDs right.

The other, and more important, is that several of our customer gateways will vary the SenderSubID and/or TargetSubID. For example, the FIX spec describes SenderSubID as being useful to distinguish between traders, and that messages not intended for a particular trader may optionally be flagged as ADMIN. It seems to me that the changes in QFJ-272 have resulted in a restrictions/obstacles beyond what was really intended by the spec, and certainly beyond what our customers will handle.

Fortunately there is a mechanism to provide your own AcceptorSessionProvider, which can be used to work around this issue. That said, the default behaviour is not ideal. Attached is a patch to replace StaticAcceptorSessionProvider with a new class, DefaultAcceptorProvider, which first tries a "strict" 1.3.1-style lookup, and if it fails, tries a 1.2.1-style lookup based on the Comp IDs only.

If the maintainers have an objection to the patch, I would suggest at the least that something like DefaultAcceptorProvider (plus instructions) be included in the distribution for those who cannot use StaticAcceptorSessionProvider.



 Comments   
Comment by Steve Bate [ 10/May/10 ]

The specifications include subID and locationID in information such as DeliverTo and OnBehalfOf, which are used for session routing. A route endpoint in this case is identified by CompID/SubID/LocationID in the originator sessionID and in the third-party routing cases. As you mentioned, I have seen the specification describe senderSubID as being useful to distinquish between traders, but this appears to me to still be in the context of message routing to a specific trader (or trading desk) within a broader organization (represented by the compID). Every counterparty I've worked with that used subIDs have used them for session identification and routing.

That said, from what I've seen on the FPL forums there was confusion in FIX 4.2 and before about the use of session ID fields for non-routing purposes. In FIX 4.3 and later, the Parties component is recommended for identifying traders outside the context of message routing.

I'm not the only one making the decision, but I'm open to to a feature that optionally ignores parts of the session ID for routing purposes.

Comment by Grant Birchmeier [ 19/Jul/10 ]

I am having similar issues as Rhys.

While reading the 4.4 spec to puzzle out my issues, I could not find where it says that SubID and LocationID should be used for anything substantial, let alone for message routing or session identification. Like Rhys, I'm not sure QFJ-272 is a correct interpretation. Such a change is not present in the latest QF/C++.

For now I'm integrating Rhys' patch into our local QF/J build, and being able to ignore the QFJ-272 additions is quite important for my company's usage.

Comment by Grant Birchmeier [ 19/Jul/10 ]

To integrate this patch into r958, DefaultAcceptorSessionProvider::getSession() needs a slight update to match an updated AcceptorSessionProvider interface:

  • public Session getSession(SessionID sessionID)
    + public Session getSession(SessionID sessionID, SessionConnector ignored)
Comment by Grant Birchmeier [ 12/Apr/11 ]

Committed to 1015

Comment by Jörg Thönnes [ 09/Jun/11 ]

Re-opening the issue to document the changes introduced
by this ticket.

IMHO, the handling of sub IDs and location IDs in QF/J deserves an extra section in the manual. There was some
ongoing discussion in the mailings lists and FPL forums
and it also not clear to me how QF/J handles this now.





[QFJ-564] Operating on (modifying) repeating groups Created: 10/Nov/10  Updated: 10/Nov/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Addy Bhardwaj Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

N/A



 Description   

Currently if you want to add a value in a repeating group, there isn't a straightforward way of doing so. For instance,

//Just for illustration, if noSides group is read from TCR
TradeCaptureReport.NoSides noSidesGroup = new TradeCaptureReport.NoSides();
tradeCaptureReport.getGroup(1, noSidesGroup);

//add account information in TCR
noSidesGroup.set(new Account("accountA"));

This action doesn't modify the original TCR message as the group returned by getGroup() method is a copy of the actual group object.

Is there a reason for this behaviour? Also, I noticed that when we add a typed group object it is copied into a Group object and then stored in the underlying TreeMap of groups. The only reason I could find is that String to Object parsing doesn't create concrete group objects hence copy is required but I hoping there is another reason that I have overlooked.

ps: I have modified source code locally to improve this behaviour by

  • Removing the copy behaviour all together so concrete groups objects are kept in the message object
  • parsing string to QUICKFIXJ message object handles creation of concrete Groups as well.
    I am more than happy to provide a patch if it this functionality is desirable. It should surely help optimise the code by reducing copies and object creation.





[QFJ-400] Make scheduledExecutorService non-static Created: 03/Feb/09  Updated: 16/Feb/09

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Mihai Sardarescu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

Hello all,

In order to start our FIX connector, we have a "start" call from our middleware (there could be more than one FIX connectors started at different times on the same JVM). On this call, a logger is paased to the connector and all the log of the FIX connector should be dumped to this logger. The log of the QuickfixJ session is easily routed. However, the log messages from Mina and of the QuickfixJ initiators are more difficult to route. We use InheritableThreadLocal to pass the logger to all threads started by our FIX connector.
We encountered a problem due to the fact that the QuickfixJ SessionConnector uses a static executor service for executing SessionTimerTask tasks. Could you please change this to non-static?
For more details, please refer to thread: http://sourceforge.net/mailarchive/message.php?msg_name=1714.65.220.39.123.1208198497.squirrel%40webmail3.hrnoc.net

We expect our application to go in production very soon (at most 2 month for now).

Thank you,
Mihai



 Comments   
Comment by Steve Bate [ 05/Feb/09 ]

Please remind me why you weren't able to write a SLF4J logger implementation that delegates to the thread local logger provided by the middleware product. All logging from all threads goes through SLF4J so it seems that this would route those messages through the logger you provided.

Comment by Mihai Sardarescu [ 05/Feb/09 ]

Hi Steve,

In order to start the FIX connector, I get a start call from the middleware. On this call, a logger is passed to my code and I shall redirect all log message written by quickfixj and mina to this logger.

To do that I have used an InheritableThreadLocal that holds the middleware logger object in order to pass it to all the created by quickfixj and mina code. The loggers created by the slf4j are just proxies to the current thread middleware logger. I can thus redirect the quickfixj and mina log messages to the middleware log.

This technique does not work in case a static executor (or a static thread) is used. For example:
1. The FIX connector is started using the middleware logger LOGGER1. When loading of the quickfixj library, the static executor is initialized and the the logger attached to it is LOGGER1. All log messages written on this logger will be directed to the LOGGER1.
2. A second instance of the FIXConnector is started in the same JVM using LOGGER2. As the static executor was already initialized, the log written by quickfixj on the static executor will be also routed to LOGGER1 (and not to LOGGER2 as expected). This leads to potentials problems - what if the first instance of the FIXConnector is stopped and the LOGGER1 is closed by the middleware (etc.)

Also, I do not see the benefit of using a static scheduledExecutorService for all the initiators (which also uses a daemon worker thread). Wouldn't it be better to use a non-static executor that can be shutdown once the initiator is stopped?

Regards,
Mihai

Comment by Steve Bate [ 06/Feb/09 ]

Hello Mihai,

As we discussed before, the purpose of the static (singleton) executor is to minimize the number of threads used by QFJ for it's own purposes. The executor is shared among all initiators and acceptors.

I'm still not understanding why using a custom SLF4J implementation didn't work for you. If the custom logger implementation delegates to the /current/ middleware logger then it doesn't matter when the logger client actually retrieved the SLF4J logger instance. Actually, you could do this even without SLF4J by just implementing the QFJ Log interface. Your custom Log would delegate to the middleware logger. The middleware callback would modify the logger delegate in your Log implementation and everything that uses the Log implementation would automatically be using the new middleware logger. If the middleware logger is thread-local then that could also be supported with this approach.

I'm probably missing something, but this is my understanding at this point based on the information I have. Feel free to clarify any key issues I'm not seeing. I want to understand why this can't be done in the way I describe before changing the QFJ internals to support it.

Thanks,

Steve

Comment by Mihai Sardarescu [ 16/Feb/09 ]

Hi Steve,

The middleware provides a different logger object for each FIX connector. As I have one and only one QFJ session per connector, I have implemented a special QFJ logger that routes the log of one QFJ session to the middleware logger - this works fine and the log messages are routed to the correct logger.
However, I am missing the mina log and the low-level log of QFJ (the log that is written using slf4j). In fact, each FIX connector has its own initiator - the initiator is started upon the initialization of the connector and is stopped once the connector is destroyed. The problem is that once an initiator writes log messages from the static thread, there is no way to find out the which connector this initiator corresponds in order to route the log message to the correct logger.

To make it more clear: let's suppose I have two concurrent FIX connectors running in the same JVM having:

  • LOGGER1, LOGGER2 - the middleware loggers
  • INITIATOR1, INITIATOR2 - the initiators.
    INITIATOR1 logs something using the slf4j logger: log.debug("MESSAGE1") from the static thread THREAD1.
    INITIATOR2 logs something using the slf4j logger: log.debug("MESSAGE2") from the static thread THREAD1.
    There is no way to decide if the MESSAGE1 should be routed to LOGGER1 or LOGGER2 (the same applies for MESSAGE2). If the thread was non-static, then using inheritable thread locals could allow me to route the message to the right logger.

Thanks,
Mihai





[QFJ-405] Fields missing on resends Created: 26/Feb/09  Updated: 26/Feb/09

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.1
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

In some circumstances fields may be omitted from re-sent messages.

We tracked this down to the following line in Session.parseMessage(String):
msg.fromString(messageData, dataDictionary, false);
In our case there was a repeating group used in outbound messages only that was not present in the data dictionary. Those fields silently disappeared.

So, a workaround is to edit the data dictionary. But given the silent misbehaviour, I would rate this a bug.

Probably could fix this by creating a subclass of Message that treats the body as an immutable string --> overrides to parseBody, calculateString, calculateLength and calculateTotal. Hopefully there is a more elegant solution.






[QFJ-373] Add a new validation configruation to bypass the data type check Created: 20/Nov/08  Updated: 26/Nov/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Alvin Wang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Steve Bate [ 26/Nov/08 ]

Can you be more specific about what you want in this feature? Thanks.

Comment by Alvin Wang [ 26/Nov/08 ]

For example, if there is a configuration called ValidateDateType=N in the configuration file, my QFJ server will not reject a message in which there is "867=100,0" rather than "867=100.0" because I do not care about tag 867.

thanks.





[QFJ-369] Send too many logout messages when continuely received messages which has BadTime or doBadCompID Created: 06/Nov/08  Updated: 06/Nov/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Mike Gu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Windows XP



 Description   

Send too many logout messages when continuely received messages which has BadTime or doBadCompID.
Session.java
private void doBadCompID(Message msg) throws IOException, FieldNotFound

{ generateReject(msg, SessionRejectReason.COMPID_PROBLEM, 0); generateLogout("Bad CompID"); }

private void doBadTime(Message msg) throws IOException, FieldNotFound

{ generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0); generateLogout(); }

Before Send a logout message, should check the session statues. isLogoutSent()

private void doBadCompID(Message msg) throws IOException, FieldNotFound

{ generateReject(msg, SessionRejectReason.COMPID_PROBLEM, 0); if (!state.isLogoutSent()) generateLogout("Bad CompID"); }

private void doBadTime(Message msg) throws IOException, FieldNotFound

{ generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0); if (!state.isLogoutSent()) generateLogout(); }




[QFJ-394] SocketInitiator.stop(false) always waits sessions logoutTimeout before exit Created: 15/Jan/09  Updated: 03/Feb/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Alexey Utkin Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

SocketInitiator.stop(false) doesn't check sessions state. It always waits session logoutTimeout

It seems a bug in SessionConnection.waitForLogout()



 Comments   
Comment by Peter Franzen [ 03/Feb/10 ]

As far as I can tell, the problem lies in SessionConnection.waitForLogout().

If a session hasn't received its logout response when this method is entered
waitForLogout() keeps it in the local set of logged on sessions until the session's
logout timeout expires.

I noticed this problem when moving from 1.2.1 to 1.3.x, it seems to have been
introduced in version 1.3.

The solution would be to check if the session has been logged out each time
the sessions in the local logged on set are checked.

This could be achieved by changing the inner while loop in waitForLogout() to

while (sessionItr.hasNext()) {
Session session = sessionItr.next();
if (elapsed >= session.getLogoutTimeout() * 1000L) {
try

{ session.disconnect(); }

catch (IOException e)

{ log.error(e.getMessage(), e); }

}
if (!session.isLoggedOn())
sessionItr.remove();
}





[QFJ-968] SessionState messageQueue causing out of memory exception Created: 06/Jan/19  Updated: 11/Jan/19

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.0.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Ryan Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment:

Java 8



 Description   

The issues is based on my observation and investigation. Please correct me if I misunderstood it, thanks!

Scenario:

  • We use SocketInitiator to connect to vendor server and consume data.
  • ResendRequestChunkSize is set to 5000.
    Now assume our process is down for several hours, and 10 million messages are queued up on the vendor side. Once we get connected again, we will send about 2000 resend requests with batch size of 5000 to catch up to the latest message. In between of every two batches, vendor sends latest messages to us, which have much higher sequence number, and are added to the messageQueue in SessionState. They will not be processed until all the 10 million messages have been received by us, which means the memory will keep expanding during our "catching up" time. And this will eventually cause out of memory issue.

Proposed Solution:

Not limiting messageQueue size could be risky. My understanding is that messageQueue should only store temporary out of order messages, and if there are say 100,000 messages in it, there must be something wrong. Therefore options should be provided to discard newer messages instead of keep eating up the memory.

My proposal is to:

  1. Change LinkedHashMap to TreeMap. Even it may slow things down a little bit, O(lg n) should work fine given the size of messageQueue is normally very small. Even when it grows up, sequence number ordered entries should be beneficial.
  2. Update enqueue method to limit the size of messageQueue. If the queue is full, then add the new message and remove the entry with highest sequence number.
  3. Update dequeueMessagesUpTo method.

If you think this could work, I would be happy to submit a PR, thanks!



 Comments   
Comment by Christoph John [ 08/Jan/19 ]

I am not sure I understand fully: when you want to limit the queue size and discard messages if the queue is full, then why do you do a resend request for that many messages anyway? You could then just not do any resends (e.g. reset to seqnum 1)?

Apart from that: if you are transmitting millions of messages in a few hours then I am not sure if FIX is the right choice of protocol. I assume that some kind of volatile data is transmitted (e.g. market data) which in turn leads to the question why you need resends at all if the data is outdated in the meantime anyway.

Or did I misunderstand something?

Thanks,
Chris.

Comment by Ryan [ 09/Jan/19 ]

Hi Chris - Thank you for the reply! Let me try to give more details.

1. The reason we want to persist these messages is for future analysis. Normally the system works perfectly fine, but we want to handle the rarely happened situation where there is an issue and the system is down in the middle of the day. When that happens, we do need to send resend requests to catch up the outdated data.

2. My understanding is that there are two queues involved:

  • eventQueue in Session which is a bounded blocking queue
  • messageQueue in SessionState which is a map and not bounded

Every message has to go through the eventQueue, but only messages with higher seqNum will be put into messageQueue. Let's assume the batch size is 20 and consider the following event sequence:

  • initiator -> acceptor: expect 501, but seeing 10000, request resending of 501-520
  • acceptor -> initiator: resend 501-520, done
  • acceptor -> initiator: new messages 10001, 10002, 10003
  • initiator -> acceptor: request resending of 521-540
  • acceptor -> initiator: resend 521-540, done
  • acceptor -> initiator: new messages 10004, 10005
    ...

In the given sequence, messages 501-540 will be handled correctly, but 10001-10005 will be added to messageQueue and will not be touched until all messages 501-10000 have been requested and processed.

My proposal is to limit the size of the messageQueue (the map) to prevent the unbounded growing. For example if we set the limit to 4 messages, then the message 10005 and later messages will not be added to the messageQueue. When the initiator finishes handling messages 501-10004, it will expect 10005, but since it's discarded, initiator will try to request messages from 10005 to 10024 and move on.

This scenario will happen only when both of the following conditions are satisfied:

  • resend batch size is set.
  • new messages are coming in when the acceptor resends the messages.

Hope this better explained the scenario.

Thanks,
Ryan

Comment by Ryan [ 09/Jan/19 ]

BTW this is not a blocking issue for us, just want to report what we've observed during the development.

Comment by Christoph John [ 11/Jan/19 ]

Hi Ryan,

after thinking a little more about it I think it should work the way you said, i.e. if we limited the size of the queued messages QFJ will request them anyway as soon as it has dequeued all messages.

I have the following questions:
1. Why should this only be active when resend batch size is set? IMHO it could also happen that new messages are interleaved with resent messages. But of course that depends on how the counterparty has implemented their resend processing.
2. Why do you want to change the LinkedHashMap to TreeMap? Just because of the size limitation? Wouldn't a simple check if the allowed size has been reached (and then not queueing the message) be sufficient?
3. In the description you said "If the queue is full, then add the new message and remove the entry with highest sequence number." Why? I thought the idea was to not add the message?

Thanks,
Chris.

Comment by Christoph John [ 11/Jan/19 ]

Of course it will lead to massive number of messages since messages will be re-requested multiple times if the counterparty already sent them and we re-request them because we did not put them to the queue.

Comment by Ryan [ 11/Jan/19 ]

Hi Chris,

Let me pull the example sequence from above and try to explain my understanding:

  • initiator -> acceptor: expect 501, but seeing 10000, request resending of 501-520
  • acceptor -> initiator: resend 501-520, done
  • acceptor -> initiator: new messages 10001, 10002, 10003
  • initiator -> acceptor: request resending of 521-540
  • acceptor -> initiator: resend 521-540, done
  • acceptor -> initiator: new messages 10004, 10005
  • ...
    1. Totally agree that the limit can and should be applied for both batch and non-batch. Normally non-batch request won't cause this problem because of the acceptor logic. Let's take the same example, instead of using a batch of 20, if we send the resend request as 501-infinity, then messages 501-10000 will be added to the acceptor's send buffer. Now if new messages 10001-10005 appears on the acceptor side, they will only be appended to the end of the buffer, which means initiator will not see them until 501-10000 are received and processed. But if the acceptor takes another strategy of handling resend request, then non-batch may hit the same memory expanding problem. So I think you are right that we should limit the size of messageQueue in all cases.
    2 & 3. Generally speaking, we care more about the message sequence (MsgSeqNum) but not the insertion sequence. More specifically, let's assume the messageQueue is full with messages 10001-10004, and there is a temporary out of order message 542 coming in. Since we care more about 542 but not 10004 at this time, evicting 10004 would be the only option if we want to keep 542 and not expanding the queue size. By this way the TreeMap messageQueue will still work even after it's full.
    4. If the size limit is to be added, it should definitely be configurable so that user can choose between memory expanding and massive resend requests.

Thanks,
Ryan





[QFJ-974] Session+FileStore reset fails indefinitely when stream closed Created: 23/Apr/19  Updated: 06/Jun/21

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.1
Fix Version/s: None

Type: Bug Priority: Default
Reporter: amichair Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   

If the FileStore stream is closed for any unexpected reason, the reset will fail indefinitely even if the original cause is fixed (or was just a transient error), since it will continue to try and flush/closed the failed stream, instead of truly resetting and opening a new one.

In our case the error was triggered by the disk being full (space was then freed up), but any reason for closing the stream could get the store stuck in this unrecoverable state. Perhaps the store reset mechanism should be made more robust so that it really tries to reset even in this unexpected state rather than require the application to be restarted manually.

quickfix.RuntimeError: java.io.IOException: Stream Closed
at quickfix.SessionState.reset(SessionState.java:384)
at quickfix.Session.resetState(Session.java:2624)
at quickfix.Session.nextLogon(Session.java:2116)
at quickfix.Session.next(Session.java:1026)
at quickfix.Session.next(Session.java:1204)
at quickfix.mina.ThreadPerSessionEventHandlingStrategy$MessageDispatchingThread.doRun(ThreadPerSessionEventHandlingStrategy.java:222)
at quickfix.mina.ThreadPerSessionEventHandlingStrategy$ThreadAdapter.run(ThreadPerSessionEventHandlingStrategy.java:146)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Stream Closed
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at quickfix.FileStore.close(FileStore.java:218)
at quickfix.FileStore.close(FileStore.java:209)
at quickfix.FileStore.closeAndDeleteFiles(FileStore.java:223)
at quickfix.FileStore.initialize(FileStore.java:101)
at quickfix.FileStore.reset(FileStore.java:405)
at quickfix.SessionState.reset(SessionState.java:382)
... 7 more Suppressed: java.io.IOException: Stream Closed
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at java.io.FilterOutputStream.close(FilterOutputStream.java:159)
... 13 more



 Comments   
Comment by Sergei [ 06/Jun/21 ]

13:35:17.885 QFJ Timer ERROR quickfix.SocketInitiator.run:356 - Error during timer processing
quickfix.RuntimeError: java.io.IOException: Stream Closed
at quickfix.SessionState.reset(SessionState.java:384)
at quickfix.Session.resetState(Session.java:2624)
at quickfix.Session.generateLogon(Session.java:2003)
at quickfix.Session.next(Session.java:1918)
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:350)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Stream Closed
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at quickfix.FileStore.close(FileStore.java:218)
at quickfix.FileStore.close(FileStore.java:209)
at quickfix.FileStore.closeAndDeleteFiles(FileStore.java:223)
at quickfix.FileStore.initialize(FileStore.java:101)
at quickfix.FileStore.reset(FileStore.java:405)
at quickfix.SessionState.reset(SessionState.java:382)
... 11 common frames omitted
Suppressed: java.io.IOException: Stream Closed
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.FilterOutputStream.close(FilterOutputStream.java:158)
at java.io.FilterOutputStream.close(FilterOutputStream.java:159)
... 17 common frames omitted

Same issue

What file might be the problem? - java.io.FileOutputStream.writeBytes





[QFJ-969] Thread Safety FileStore.set Created: 28/Jan/19  Updated: 28/Jan/19

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.0
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Yanick Salzmann Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Message, QuickfixJ


 Description   

When seeking for optimizations for file sync I have come across the FileStore.set method and was wondering about its thread safety. I labelled this issue as improvement, because I am not sure if it is maybe already been taken care of somewhere else, however lets look at this method:

https://github.com/quickfix-j/quickfixj/blob/045be5c09f872b1bf7b1a8ed2904efd0030c59bd/quickfixj-core/src/main/java/quickfix/FileStore.java#L364

As you can see, at the very beginning of the function the offset inside the RandomAccessFile is taken as the message offset. The actual writing of the information happens after many more instructions, one of them even including a synchronization on a file descriptor.

According to my understanding the offset obtained on line 365 could easily be outdated by the time the message is written at line 377. Of course it is possible, that synchronization is supposed to be happening outside of the set method, however I do not see any contract enforcing this in MessageStore.set.

Should this method not be thread safe to avoid potential data inconsitencies?



 Comments   
Comment by Christoph John [ 28/Jan/19 ]

Hi Yanick Salzmann,

as far as I am aware the MessageStore.set() method is only accessed from Session.sendRaw() where it is guarded by a lock. But yeah, in theory it could happen that it is accessed concurrently. It seems like this could happen especially on shutdown. See QFJ-762.
There is another issue QFJ-552 which provides a wrapper around a MessageStore that one could use.

Cheers,
Chris.





[QFJ-646] MiscFeeType field is handled incorrectly Created: 27/Oct/11  Updated: 17/Nov/11

Status: Open
Project: QuickFIX/J
Component/s: Engine, Metadata/Specs
Affects Version/s: 1.5.0, 1.5.1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: John Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ


 Description   

The "MiscFeeType" extends StringField and the constants into it are "String"s. The specification however (and the dictionary) indicates that this field is a "char".
One of our clients sends this as a "String" (value "12"), but we reject the message because it is more than one "char" long. If he sent it as a "char", we would not reject, but it would not match the constants in the Class file.

Workarround, either:

  • Change the dictionary to have this field be a "STRING" (used this one since our client sends it as a String).
  • Have the "MiscFeeType" class extends "CharField" and change the constants in the Class file, and embed it into your own code to replace the one from the library (haven't tested that though).


 Comments   
Comment by Laurent Danesi [ 15/Nov/11 ]

Hi John,

Which FIX version are you using?

In fact, in the spec, this field was ever a char and recently changed to String with FIX5.0.
So the FIX4X.xml dictionaries are declaring it as a CharField and FIX50.xml is specifying it as a StringField accordingly with the spec.

Now, of course, if you need to downgrade FIX5 feature code to a FIX4X session, you need a custom dictionary but it is not a bug, isnt it?
What do you think?

Regards,

Laurent

Comment by John [ 15/Nov/11 ]

Hi,
I'm using the 4.x version. The problem is not to have "String" or "char", I do not care.
The problem is that if we receive a client's message as a char (which is the standard), it would not match the constants in the java source file

Comment by Laurent Danesi [ 15/Nov/11 ]

Hi,

In fact, we generate fields for each dictionary but only once for a field (no override) and we generate FIX50 before FIX4X so MiscFeeType is a StringField.

But I understand that if a client sends a String and MiscFeeType is a CharField, QFJ will reject but the current code should work as the parser will set the char as a String.anyway and we can match constants with equals method or I'm missing something perhaps?

Can you provide me a simple test to well understand and fix it please?

Laurent

Comment by John [ 17/Nov/11 ]
  • If the user sends as a String, he will send "12" => error on our side because it is more than two characters.
  • If the user sends as a character (character \0x000C), then it will be OK, BUT it will not match the constants declared in the java file "12" (this is a guess, maybe your code changes character '\0x000C' in String "12").




[QFJ-543] "connector" parameter in interface AcceptorSessionProvider.getSession() is undocumented and unclear Created: 19/Jul/10  Updated: 19/Jul/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Grant Birchmeier Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

See paste below from AcceptorSessionProvider. The parameter connector is absent from the javadoc, and it's not clear what it's for. As it's an interface, the parameter's purpose should be documented.

From the two instantiations of this interface (DynamicAcceptorSessionProvider and AbstractSocketAcceptor::StaticAcceptorSessionProvider), it appears this parameter is only used for dynamic sessions and is otherwise ignored.

/**

  • Return a session for this sessionID. The session might be
  • created dynamically.
  • @param sessionID
  • @return the associated session or null if no session can be
  • associated with the given ID.
    */
    Session getSession(SessionID sessionID, SessionConnector connector);


 Comments   
Comment by Grant Birchmeier [ 19/Jul/10 ]

Paste above is from r959 (the latest at time of bug creation).





[QFJ-663] FieldMap's "getGroup()", "replaceGroup()", and possibly others... should start with 0 (not 1) Created: 23/Jan/12  Updated: 23/Jan/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: John Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   

The first index for anything in the IT world is "0", not "1". Most certainly any programmer in this world expects the index "0" to be the first one.
An implementation must add 1 to each index, and the FIX engine then decrements it (will not kill the CPU but it is at least some work that could be avoided).

My solution would be to deprecate the methods and have new replacing methods (remove the deprecated methods and rename the new ones in a later version).






Generated at Fri Mar 29 13:01:51 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.