[QFJ-1000] can't pass target destination URL (SocketConnectHost) to proxy server Created: 09/Feb/21 Updated: 09/Feb/21 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.4 |
Fix Version/s: | None |
Type: | Bug | Priority: | Blocker |
Reporter: | leaf | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Windows |
Description |
i am using quickfixj to connect with target URL using proxy , the proxy protocol is socks , below is my quickfixj configure sample: ProxyType=socks SocketConnectHost=******** |
Comments |
Comment by Christoph John [ 09/Feb/21 ] |
Did you try with a newer version? 1.6.4 is rather old and there have been proxy related fixes in the last versions. |
[QFJ-977] ResendRequest message ( 35=2) received from Sender was ignored in QuickFIXJ Created: 21/May/19 Updated: 08/Jul/19 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Critical |
Reporter: | Lantony | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ, ResendRequest,, sequence, | ||
Environment: |
Linux |
Description |
Fix session was interrupted due to network issue... Both side FIX processes ( Client, Sender) did not know about the n/w issue. The Client side was continuously sending logon request to other side... Once the n/w issue was resolved the connectivity was established. However, the sequence was not in sync at this time . Hence , The sender sent resend request (35=2) immediately after successful logon response. the problem is why my fix engine (i.e., the client ) did not respond to 35=2 resent request given by sender ? . I am expecting my Fix engine should have responed with 35=4 like my Sender's engine . I am using QuickfixEngineJ 1.5.3 Some one kindly help please. The below the sequence of FIX messaged traced in messages.log/ Response from Sender: Resent request from sender ( problem where I did not respond for this message) Resent request from Client (i.e., by my quick fix engine ) reset request responded by sender: |
Comments |
Comment by Lantony [ 22/May/19 ] |
Small Correction , The environment is Windows , not Linux. |
Comment by Christoph John [ 08/Jul/19 ] |
Hi, Cheers, |
[QFJ-466] Initiator SocketConnectorIoProcessor-0.0 deadlocks with QFJ Message Processor at logout Created: 01/Sep/09 Updated: 29/Apr/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Critical |
Reporter: | Steve Borrer | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Ubuntu Linux 2.6.24-23-server #1 SMP Thu Nov 27 18:45:02 UTC 2008 x86_64 GNU/Linux |
Description |
The FIX initiator is logged out at 21:30 UTC by the acceptor (itself built with QFJ). The initiator logs out. An exception is reported in the initiator. Shortly after, the initiator initiates a logon request which is declined as it is outside the time window of the acceptor. There is then a endless sequence of 'Pending connection not established after XXXX ms' messages. See my application log below:- 20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - Received logout request 20090828-21:30:00 20090828-09:30:00 DEBUG - loggingOutState:true A SIGQUIT to the initiator reveals a deadlock:- 20090901-08:07:25 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 38094032 ms. "Checkpointer" daemon prio=10 tid=0x00007f15040fa000 nid=0x1f7f in Object.wait() [0x0000000040d6d000]
"Cleaner-1" daemon prio=10 tid=0x00007f15040f9800 nid=0x1f7e in Object.wait() [0x0000000041293000]
"INCompressor" daemon prio=10 tid=0x00007f1504216000 nid=0x1f7d in Object.wait() [0x0000000042400000]
"SocketConnectorIoProcessor-0.0" daemon prio=10 tid=0x0000000040789000 nid=0x1f7c waiting for monitor entry [0x0000000042804000]
"OMS_Client:nt@tradera1" prio=10 tid=0x000000004060c000 nid=0x178f runnable [0x0000000042905000]
"QFJ Message Processor" daemon prio=10 tid=0x00007f1504272000 nid=0x177b waiting for monitor entry [0x0000000042602000]
"QFJ Timer" daemon prio=10 tid=0x00007f15042e1000 nid=0x177a in Object.wait() [0x0000000042501000]
"OMS_Client:oms@tradera1" prio=10 tid=0x000000004018c000 nid=0x1773 runnable [0x0000000040b04000]
"OMS_Client:oms@tradera1" prio=10 tid=0x00000000405e8000 nid=0x1772 runnable [0x0000000040a03000]
"OMS_Client:oms@tradera1" prio=10 tid=0x0000000040477800 nid=0x1771 runnable [0x0000000040902000]
"OMS_Interface" prio=10 tid=0x00007f15040a2000 nid=0x1770 runnable [0x0000000040c6c000]
"OMS_Sender" prio=10 tid=0x00007f1504059800 nid=0x1735 in Object.wait() [0x00000000422ff000]
"MessageQueue" prio=10 tid=0x00007f1504058800 nid=0x1734 in Object.wait() [0x00000000421fe000]
"Low Memory Detector" daemon prio=10 tid=0x00000000402f2800 nid=0x1726 runnable [0x0000000000000000] "CompilerThread1" daemon prio=10 tid=0x00000000402ef800 nid=0x1724 waiting on condition [0x0000000000000000] "CompilerThread0" daemon prio=10 tid=0x00000000402eb000 nid=0x1721 waiting on condition [0x0000000000000000] "Signal Dispatcher" daemon prio=10 tid=0x00000000402e9000 nid=0x171e waiting on condition [0x0000000000000000] "Finalizer" daemon prio=10 tid=0x00000000402c6000 nid=0x1714 in Object.wait() [0x0000000041ffc000]
"Reference Handler" daemon prio=10 tid=0x00000000402c4000 nid=0x170f in Object.wait() [0x0000000041efb000]
"main" prio=10 tid=0x0000000040112800 nid=0x16e5 in Object.wait() [0x00000000417f4000]
"VM Thread" prio=10 tid=0x00000000402bd800 nid=0x170a runnable "GC task thread#0 (ParallelGC)" prio=10 tid=0x000000004011c800 nid=0x16ea runnable "GC task thread#1 (ParallelGC)" prio=10 tid=0x000000004011e800 nid=0x16ee runnable "GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040120800 nid=0x16f2 runnable "GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040122000 nid=0x16f6 runnable "GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000040124000 nid=0x16fa runnable "GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000040126000 nid=0x16fe runnable "GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000040127800 nid=0x1702 runnable "GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000040129800 nid=0x1706 runnable "VM Periodic Task Thread" prio=10 tid=0x00000000402f5000 nid=0x1728 waiting on condition JNI global references: 1325 Found one Java-level deadlock: Java stack information for the threads listed above:
Found 1 deadlock. Heap |
Comments |
Comment by Eric Deshayes [ 29/Apr/11 ] |
The problem is due to the usage of the SleepyCat database: we're executing a Database.close() and Database.put() concurrently, and the javadoc for Database.close() points out that a database handle should not be closed while operations using that Database object are still running: |
[QFJ-966] Initiator Logout does not appear in QFJ log file on connection reset Created: 03/Dec/18 Updated: 03/Dec/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.1.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Daniel | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | fromAdmin, initiator, logout | ||
Environment: |
linux |
Description |
I think this is an old issue, but still seems to be a problem with the latest version of quickfixj 2.1.0. After my initiator sends a Logon message, the acceptor immediately sends back a Logout Message followed by closing the connection. Looking at the tcpdump, it is clear why the connection is closed x.x.x.x.56988 > x.x.x.x.25265: Flags [P.], cksum 0x9863 (incorrect -> 0xe431), seq 1:104, ack 1, win 21, length 103 The problem is that fromAdmin is never called before onLogout because of the connection reset following the logout message. IE: it is not in the QFJ log file but I can see it in the tcp dump. The initiator stack trace (ie connection reset) looks like "..." and all I receive on the initiator side is "Initiated logon request" message from Christoph John So this seems to be a special case that is not covered by the unit test. But this is only a first quick analysis. Of course this has to be made reproducable in a unit test and then fixed. |
[QFJ-959] Fix session could stuck forever in acceptor mode. Created: 07/Nov/18 Updated: 16/Nov/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.1.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Dmitry | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
Repetitive error: Multiple logons/connections for this session are not allowed Looks like session was not cleared correctly after exception. Nov 6 06:47:32 aemulator: java.io.IOException: Broken pipe Nov 6 06:47:32 aemulator: at sun.nio.ch.FileDispatcherImpl.write0(Native Method) Nov 6 06:47:32 aemulator: at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) Nov 6 06:47:32 aemulator: at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) Nov 6 06:47:32 aemulator: at sun.nio.ch.IOUtil.write(IOUtil.java:65) Nov 6 06:47:32 aemulator: at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) Nov 6 06:47:32 aemulator: at org.apache.mina.transport.socket.nio.NioProcessor.write(NioProcessor.java:384) Nov 6 06:47:32 aemulator: at org.apache.mina.transport.socket.nio.NioProcessor.write(NioProcessor.java:47) Nov 6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.writeBuffer(AbstractPollingIoProcessor.java:1107) Nov 6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.flushNow(AbstractPollingIoProcessor.java:994) Nov 6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.flush(AbstractPollingIoProcessor.java:921) Nov 6 06:47:32 aemulator: at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:688) Nov 6 06:47:32 aemulator: at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) Nov 6 06:47:32 aemulator: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) Nov 6 06:47:32 aemulator: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) Nov 6 06:47:32 aemulator: at java.lang.Thread.run(Thread.java:748) Server logs. 2018-11-07 13:06:56.731 +0000 ERROR [NioProcessor-471] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:06:57.736 +0000 ERROR [NioProcessor-472] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:06:58.740 +0000 ERROR [NioProcessor-473] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:06:59.780 +0000 ERROR [NioProcessor-474] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:07:01.446 +0000 ERROR [NioProcessor-475] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:07:03.446 +0000 ERROR [NioProcessor-476] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:07:05.446 +0000 ERROR [NioProcessor-477] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed 2018-11-07 13:07:07.446 +0000 ERROR [NioProcessor-478] quickfixj.errorEventFIX.4.4:emulator->TRADE: - Multiple logons/connections for this session are not allowed |
Comments |
Comment by Christoph John [ 07/Nov/18 ] |
Hi, is the pasted stack trace complete? I can not gather much from it. There should also be log entries from QFJ along with the IOException. Can you post them? I have never seen this behaviour before. Is it reproducible? Did it stop at some time or did it keep running like that? Thanks, |
Comment by Christoph John [ 07/Nov/18 ] |
And of course as always: would be better to first ask on the mailing list. We can clarify there if it really is a bug. |
Comment by Dmitry [ 08/Nov/18 ] |
Hi,
We do not see another exceptions in log.
I believe we found the way how to achieve this behavior. It is definitely not an expected working behavior, but as a result, quickfix could be stuck in some wrong state forever. From time to time we have an issue with our HyperV cloud. Virtual machine is frozen for ~30-40 seconds. (it is testing environment) Probably qfSession.hasResponder() keeps wrong state forever due to exception in Session.disconnect public void disconnect(String reason, boolean logError) throws IOException { try { final boolean logonReceived = state.isLogonReceived(); final boolean logonSent = state.isLogonSent(); synchronized (responderLock) { if (!hasResponder()) { if (!ENCOUNTERED_END_OF_STREAM.equals(reason)) { getLog().onEvent("Already disconnected: " + reason); } return; } final String msg = "Disconnecting: " + reason; if (logError) { getLog().onErrorEvent(msg); } else { getLog().onEvent(msg); } responder.disconnect(); // Maybe exception is here setResponder(null); } |
Comment by Christoph John [ 09/Nov/18 ] |
Hi, OK, if the whole VM froze for 30 seconds all sort of things can happen. You are right: given the log message "Multiple logons/connections for this session are not allowed" it looks like the responder was not set to null. When you say that the Exception might occur in or after responder.disconnect() is called then you should have something in your log file like "Disconnecting: " because that happens some lines above. Do you see something like that in your log file? Thanks, |
Comment by Dmitry [ 16/Nov/18 ] |
Hi, I am sorry for late response. Dmitry. |
Comment by Christoph John [ 16/Nov/18 ] |
OK, feel free to update the issue when you encounter problems again. |
[QFJ-810] Messages get lost during logon process with sequence gap Created: 06/Oct/14 Updated: 16/Oct/14 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Heribert Steuer | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | condition, gap, logon, race |
Issue Links: |
|
Description |
When a sequence gap exists, all subsequent messages sent by the counterparty are lost when the logon message contains a out-of-sync sequence number. Scenario: Counterparty A is initiating a logon, counterparty B responds to the logon and immediately sends 10 more messages. A now detects the sequence problem in the login message and closes the session. The 10 messages are lost (meaning they are logged as received but never appear in the callback) The problem seems to be that the verify() function detects the gap in the logon messages, creates a logout and a disconnect. This leads to the fact that the ThreadPerSessionEventHandlingStrategy.run method is left and its local queue gets destroyed - still containing the messages that have been received after the logon. Whatever the correct behaviour would be - the current one is a problem because of the messages are received but never processed. It might be better to leave ThreadPerSessionEventHandlingStrategy.run() only if the queue is empty. |
Comments |
Comment by Christoph John [ 08/Oct/14 ] |
Just for clarification: the Logon of counterparty B has a sequence number which is too low? In my opinion the messages following the Logon should not be processed since their sequence number was too low. But maybe I did not understand fully. |
Comment by Heribert Steuer [ 09/Oct/14 ] |
In any case, just dropping messages should never happen. Probably a session level reject should be sent because business level messages appear without the session being completely established. But receiving and simply dropping does not feel good at all because none of the counterparties get a feedback on the problem. Please share your thoughts. |
Comment by Christoph John [ 10/Oct/14 ] |
I see a problem with sending Reject messages if the Session is not fully established. |
Comment by Heribert Steuer [ 10/Oct/14 ] |
Correct, the SeqNum was too low. In fact we had the issue in production where a session went out of sync and we got a periodic Logon/Logout war. This means that both "A" sent a logon, "B" accepts the logon immediately followed by a resend request. "A" then sends a logout because the sequence number of the logon "A" sent earlier did not match. The problem in total seems to be that there is no real handshake in FIX. So you never really know when a session is really established e.g. like in TCP you have SYN, SYN ACK, ACK. In FIX you would - using TCP as an example - have only SYN and SYN ACK. When the responder replies to a logon message, he never knows if the initiator will accept his logon message. Therefore he does not know when he can start to send data. This is a design flaw in FIX, it would be best the have the initiator have something like "established" returned to the acceptor after his logon. Nevertheless, the problem remains that the acceptor sends data immediately after sending his logon without knowing if he is allowed to do so or if the logon gets rejected somehow. I do not think that closing the connection is a solution. While you are processing the logon in quickfix, Mina would probably already read data from the socket or traffic will arrive in the network stacks buffer etc., so simply closing the connection would not be a real solution. It always ends up in a race condition. In my optinion, the reject would be okay because from the perspective of the acceptor, the session is established. He received a logon and he replied to it. Therefore quickfix is able to respond to it - and it does today by sending a logout. Therefore a solution might be to check if the buffer (ThreadPerSessionEventHandlingStrategy) is empty. If so, send a logout (as it does now). If the buffer is not empty, process the messages and send a session level reject in return. As the buffer is empty, send a logout and close the socket. Unfortunately I do not really have a better idea how to solve it, as mentioned earlier its more or less a flaw in the design of the protocol. How does the above idea sound to you? |
Comment by Christoph John [ 11/Oct/14 ] |
Just another question: this issue basically sounds a little like However, your solution sounds feasible. Will take a deeper look at it. As a workaround for the time being: many people implement the handshake as follows: after Logon immediately send a TestRequest. As soon as you receive the Heartbeat message with your TestReqID you can be pretty sure that the session is established. Of course, this is no 100% guarantee but works quite well in most cases. |
[QFJ-752] Instrument component dont show the instrumentParties repeating group Created: 30/Jul/13 Updated: 12/Mar/20 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine, Message Generation |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Christian Avalos | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ |
Description |
the noInstrumentParties(1018) component is in the componentFields int array, and not in componentGroup, so, when it added a instrumentParties ,the generated message has the field 1018, but not the other possibles fields of instrumentParties (1019, 1050, 1051, 1052) |
Comments |
Comment by Christian Avalos [ 30/Jul/13 ] |
this issue is fixed adding the noIntrumentParties fields into componentGroups attribute (and removing from componentFields int[] ) public class Instrument extends quickfix.MessageComponent { ; ; public Instrument() { super(); }...... |
Comment by Jörg Thönnes [ 31/Jul/13 ] |
This message and field classes are generated automatically from the data dictionary so this is most probably a code generation issue. |
Comment by Christoph John [ 31/Jul/13 ] |
... or in the data dictionary. |
Comment by Marcin L [ 12/Mar/20 ] |
I was wondering if this is really a bug. Clearly the definition in FIX5* looks like this <component name="Instrument"> <component name="InstrumentParties"> <component name="InstrumentPtysSubGrp"> Component classes have their own dedicated classes generated so to access the group we need to go via component abstraction. public class Instrument extends quickfix.MessageComponent { ; public class InstrumentParties extends quickfix.MessageComponent { ; public class InstrumentPtysSubGrp extends quickfix.MessageComponent { ; This is not a single case. This is basically a case for every component containing another component with a repeating group e.g. in FIX44 UnderlyingInstrument -> UnderlyingStipulations What kind of errors currently generated classes could produce with this behaviour? |
[QFJ-749] Syntax error in SessionSettings.java/line 572 leading exception from java.util.regex Created: 17/Jun/13 Updated: 08/Jul/13 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Pavel Sagulenko | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ | ||
Environment: |
Eclipse 22 on Debian GNU/Linux 7.0 |
Description |
There is a syntax error in SessionSettings.java class, line 572. "); "); (with double slash before closing parenthesis). I tried to recompile myself the corrected version, but I got the following exception: java.lang.NoClassDefFoundError: quickfix.SessionSettings Probably, you can suggest the possible solution. |
Comments |
Comment by Christoph John [ 08/Jul/13 ] |
Sorry, what do you mean by: "This causes the failure in creating a SessionSettings instance"?? |
[QFJ-734] Properties for configuring Proxool vs. not incremented outgoing sequence number Created: 13/Mar/13 Updated: 13/Mar/13 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Major |
Reporter: | Krzysztof Szalast | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
All |
Description |
I want to remind about comment in quickfix.JdbcUtil in method "static synchronized DataSource getDataSource(String jdbcDriver, String connectionURL, String user, String password, boolean cache)": If "messages" table are locked longer than 5 seconds, connection is aborted by Proxool: and outgoing sequence number is not incremented! Next outgoing message has the same sequence number (sic!). Scenario: |
Comments |
Comment by Jörg Thönnes [ 13/Mar/13 ] |
Good find! I think in that case the message should not be sent. |
[QFJ-681] During load test, some clients are able to connect only after repeated attempts Created: 22/May/12 Updated: 22/May/12 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine, Networking |
Affects Version/s: | 1.5.2 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Viswanath Palutla | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ, logon | ||
Environment: |
1. on my PC, i am running 'Executor' from quickfixj examples 2. one linux box, where i am starting 25-30 clients to put load on the 'Executor' |
Attachments: | load-test-executor.zip |
Description |
I am using Executor service in the example, because it is similar to my FIX Gateway, in terms of using AcceptorTemplate 1. I have a simple client that connects to Executor service and puts a MarketDataRequest for every 10 milliseconds, total 100 market data requests. 2. two clients are started for every second. total 30 clients are started Issues: -> server is not responding to logon requests promptly for some clients |
Comments |
Comment by Viswanath Palutla [ 22/May/12 ] |
attaching batch scripts and class files to load Executor |
[QFJ-554] SleepycatStore can enter infinite loop retrieving messages for resend. Created: 26/Aug/10 Updated: 26/Aug/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | James Olsen | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
SleepycatStore does not check returned OperationStatus from call to com.sleepycat.je.Cursor.getNext(...): public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException { Integer sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey); while (sequenceNumber.intValue() <= endSequence) { messages.add(new String(messageBytes.getData(), charsetEncoding)); if (log.isDebugEnabled()) { log.debug("Found record " + sequenceNumber + "=>" + new String(messageBytes.getData(), charsetEncoding) + " for search key/data: " + sequenceKey + "=>" + messageBytes); } cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT); sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey); } } ... } Should be: public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException { ... if (retVal == OperationStatus.NOTFOUND) { log.debug(sequenceKey + "/" + messageBytes + " not matched in database " + messageDatabase.getDatabaseName()); return; } else { retVal = cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT); |
Comments |
Comment by James Olsen [ 26/Aug/10 ] |
For clarity, the changed lines are: while (retVal == OperationStatus.SUCCESS && sequenceNumber.intValue() <= endSequence) { |
[QFJ-552] Message Stores expected to be thread safe but are not Created: 05/Aug/10 Updated: 21/Apr/17 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.5.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Carmelo Piccione | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Linux + JDBC + SQL Server 2005 |
Attachments: | SynchronizedMessageStore.java SynchronizedMessageStoreFactory.java | ||||||||||||
Issue Links: |
|
Description |
SessionState.java starts with the following bit of code: /**
// MessageStore implementation must be thread safe Yet, looking at the JdbcStore.java and MemoryStore.java implementations, they are in fact not thread safe as designed (may also be true for FileStore.java, but haven't checked). The memory store uses a regular java.util.HashMap for all its operations, which is clearly not thread safe. I have never actually seen this cause a problem in practice, however. The jdbc store does not synchronize its use of the jdbc connection, which can lead to race conditions inside the close() method of the jtds prepared statement, as shown in the stack trace below (specifically, the jtds close() method has a boolean to determine if its connection is already closed in which case it will not doubly close it despite any subsequent calls. This value is set once the close routine is complete but occurs after the connection object has already been assigned a null value- this allows another thread to potentially access the null connection object before the boolean guard has been set). Note that this bug occurs about once a week in a production environment (usually during startup), although its frequency is probably higher than average due to many quickfixj applications accessing the same database concurrently. Example 1: java.lang.NullPointerException Example 2: java.lang.NullPointerException |
Comments |
Comment by Carmelo Piccione [ 17/Aug/10 ] |
A simple but thread safe message store implementation. The constructor takes a MessageStore instance as input and delegates all method calls to it through an object lock. This is a work around which allows safe usage of the original QFJ message stores such as JdbcStore, FileStore, and MemoryStore. |
[QFJ-551] Sender comp id doesnt not get reflected while creating a session if given at runtime Created: 04/Aug/10 Updated: 04/Aug/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | hitesh | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
Desc: Code: public boolean initialize(String senderCompID){ catch (ConfigError e) { System.err.println("Could not find FIX setting file."); e.printStackTrace(); System.exit(-1); } String fixVersion = null; String tgtCompId = null; catch (Exception e) { System.err.println("Could not get [BEGINSTRING] or [TARGETCOMPID] " + "from session id [" + sessID.toString() + "]-[" + e.toString() + "]"); }} String fileStorePath; catch (Exception e) { fileStorePath = null; System.err.println("Could not get [FileStorePath] [" + e.toString() + "]"); } try { } catch (Exception e) { fileLogPath = null; System.err.println("Could not get [FileLogPath] [" + e.toString() + "]"); } MessageStoreFactory storeFactory = new FileStoreFactory(settings); catch (ConfigError e) { System.err.println("FIXClient Could not initiate socket!"); e.printStackTrace(); System.exit(-1); }String fixedString = fixVersion + "" + sndrCompId + "" + tgtCompId; // if i check my settings object i have sender comp id correctly inserted // further code initiator.start(); |
[QFJ-584] Repeated test request with dynamic acceptor Created: 03/Apr/11 Updated: 05/Apr/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Other | Priority: | Major |
Reporter: | Information Management Experts | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Windows XP |
Description |
Dear All, Our application is Client-Server System. Multiple clients(initiators) connect to the server (dynamic acceptor) to exchange fix messages.
[DEFAULT] StartTime=05:00:00 EndTime=05:00:00 FileStorePath=Settings/FIXStorage/AcceptorStore FileLogPath=Settings/FIXStorage/AcceptorStore DataDictionary=data/tools/IMEXQuickFIXDialect42.xml ValidateUserDefinedFields=Y CheckLatency=N ConnectionType=acceptor SocketAcceptPort=2025 [SESSION] BeginString=FIX.4.2 SenderCompID=FIX TargetCompID=* AcceptorTemplate=Y
[DEFAULT] StartTime=05:00:00 EndTime=05:00:00 FileStorePath=Settings/FIXStorage/InitiatorStore FileLogPath=Settings/FIXStorage/InitiatorStore DataDictionary=data/tools/IMEXQuickFIXDialect42.xml ValidateUserDefinedFields=Y CheckLatency=N ConnectionType=initiator ReconnectInterval=5 HeartBtInt=5 SocketConnectHost=123.123.123.123 SocketConnectPort=2025 [SESSION] BeginString=FIX.4.2 SenderCompID= – GENERATED AT RUNTIME TargetCompID=FIX Thanks in advanced for your help ... |
Comments |
Comment by Laurent Danesi [ 05/Apr/11 ] |
Hi, I saw your post but I'm not sure to well understand. Regards, Laurent DANESI |
Comment by Information Management Experts [ 05/Apr/11 ] |
Dear Laurent, First of all, thanks for your reply. No, we didn't use ThreadedSocketAcceptor. We were creating new threads in the implementation of Application.fromApp method and when we were calling Session.sendToTarget(message, sessionID); Also we were sending huge no of orders at the same time(up to 1000) from the external server, which means when each order arrives to our server, our server will start new thread to handle the message, then a new thread will be created to send the message again for each client. |
[QFJ-569] QuickFix/J is not scalable due to overly long duration lock on sequence number in Session.sendRaw Created: 10/Dec/10 Updated: 10/Dec/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Major |
Reporter: | Leon Chadwick | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None |
Description |
I have been writing a capacity test to measure the performance of our FIX engine (not quickfix based), however I can barely get my machine to use any CPU during the test despite my using a large number of threads to generate orders and respond with Acks. After profiling, I tracked down that all these threads are generally single threaded through quickfix.Session.sendRaw() which is locking to hold onto a sequence number. A scalable solution should keep locking to a minimum duration, i.e. do as much message checking and string conversion as possible, then fill in the sequence number as a short last step whilst inside the lock. A StringBuilder could be used to hold the toString()'d message whilst not holding the lock and leave sufficient capacity at the head of the array to shift some of the header content left/right to accomodate the sequence number field whilst during the locked region. |
Comments |
Comment by Steve Bate [ 10/Dec/10 ] |
Are you trying to send all these orders through one session? If so, the incoming messages must be processed sequentially by the session protocol engine. However, that doesn't mean your application interface implementation must be single threaded. The application implementation can hand off the message processing to a worker thread pool, for example. The downside is that the FIX engine won't be able to rollback the sequence numbers if an exception occurs in the application code. As the comments state in sendRaw(), the lock is held until the message is processed and the application callback has completed. This is necessary because any exception thrown during this time will cause the sequence number to be rolled back. In other words, the sequence number can't be incremented until these operations complete successfully. The message throughput depends on many factors. Some of the biggest factors include the number of sessions, the implementations of the message store and message log and the response time of the application implementation. For example, any logging to the disk in the application callback could have a significant impact on message throughput and CPU utilization. I don't know if any of this applies to your situation since you didn't include many details. However, I know of other organizations that have reported throughput in the 10's of thousands of messages/second. |
[QFJ-385] If one Session reset failed, it will cause all existed session failed Created: 17/Dec/08 Updated: 17/Dec/08 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Major |
Reporter: | Mike Gu | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Windows XP |
Description |
If one SessionState reset() failed, it will throw a RuntimeError and cause all existed session failed. It's unacceptable. catch (IOException e) { throw new RuntimeError(e); }} |
[QFJ-992] quickfixj 2.1.1 has dependency convergence error for org.apache.maven:maven-artifact:3.5.0 Created: 06/Mar/20 Updated: 09/Mar/20 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.1.1 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Andrew Peter Marlow | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ |
Description |
the maven plugin enforcer shows that quickfixj has a dependency convergence error for org.apache.maven:maven-artifact:3.5.0. Here is the output: Dependency convergence error for org.apache.maven:maven-artifact:3.5.0 paths to dependency are: |
Comments |
Comment by Christoph John [ 06/Mar/20 ] |
Thanks for the report. Under which cicrumstances does this cause an issue? |
Comment by Andrew Peter Marlow [ 07/Mar/20 ] |
The maven enforcer plugin is being used to report on inconsistent component versions. By default the plugin treats all such inconsistencies as errors, so by default such inconsistencies would make the build fail. This behaviour is configurable so it is possible just to get a report of the issues without making the build fail. But I want such inconsistencies to be flagged as serious issues because some of these inconsistencies involve components that show up on Black Duck security scans. The software I am working on is subject to a corporate decree that the software gets through Black Duck without raising any flags. This job is made harder when the number of flags raised is larger due to the same component appearing multiple times at different versions due to these dependency convergence errors. My aim is to report any such dependency convergence errors that I find via enforcer. If they are then all fixed (there are only a few) then we could set the flag such that any such errors are fatal, so as to detect any regressions as soon as they occur. Without such fixes I will have to write a script that processes the enforcer output and checks any issues against a list of exceptions. This quickfixj issue would have to be such as exception. It just makes ultimate the job of dealing with Black Duck scans harder. |
Comment by Christoph John [ 09/Mar/20 ] |
Understood. If you would like you could also open a pull request on the quickfixj project on github. https://github.com/quickfix-j/quickfixj |
[QFJ-947] Logout message with additional field is not logged Created: 26/Mar/18 Updated: 27/Mar/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.4 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Nikolai Kulakov | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Linux |
Description |
I came across a situation where a Logout messages that have some additional field are not logged anywhere, so it seems that the connection is breaking before the Logout message receiving. Only from our counterparty I found out that they are sending a Logout messages indeed, and my own network traffic analyzing confirmed it. 8=FIX.4.2^A9=80^A35=A^A34=1^A49=fxw2^A52=20180326-14:16:20.879^A56=JLQD^A98=0^A108=30^A554=password^A10=215^A 8=FIX.4.2^A9=80^A35=A^A34=2^A49=fxw2^A52=20180326-14:16:23.840^A56=JLQD^A98=0^A108=30^A554=password^A10=207^A ... Logged events: <2018-03-26 10:16:19.751 -0400> <Session FIX.4.2:fxw2->JLQD:MARKET_DATA schedule is weekly, SUN 21:30:00-UTC - SUN 21:00:00-UTC (weekly, SUN 17:30:00-EDT - SUN 17:00:00-EDT)> <2018-03-26 10:16:19.752 -0400> <Created session: FIX.4.2:fxw2->JLQD:MARKET_DATA> <2018-03-26 10:16:19.833 -0400> <Configured socket addresses for session: [/some_ip:port]> <2018-03-26 10:16:19.863 -0400> <MINA session created: local=/172.20.30.34:33369, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port> <2018-03-26 10:16:20.882 -0400> <Initiated logon request> <2018-03-26 10:16:20.885 -0400> <Disconnecting: Socket exception (/some_ip:port): java.io.IOException: Connection reset by peer> <2018-03-26 10:16:22.888 -0400> <MINA session created: local=/172.20.30.34:33372, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/some_ip:port> <2018-03-26 10:16:23.842 -0400> <Initiated logon request> <2018-03-26 10:16:23.845 -0400> <Disconnecting: Socket exception (/some_ip:port): java.io.IOException: Connection reset by peer> ... I read a pure network traffic and had seen that immediately after out Logon we actually receive the following Logout messages: 8=FIX.4.2^A9=0109^A35=5^A34=88^A49=JLQD^A52=20180326-14:16:20.886^A56=fxw2^A789=51^A58=MsgSeqNum too low, expecting 51 but received 1^A10=075^A 8=FIX.4.2^A9=0109^A35=5^A34=89^A49=JLQD^A52=20180326-14:16:23.846^A56=fxw2^A789=51^A58=MsgSeqNum too low, expecting 51 but received 2^A10=076^A ... I think, this incoming Logout messages must be logged as well as our outcoming Logon messages, but this does not happen. I set logging level to the minimal possible values, but this did not help: there are no this Logout messages anywhere. |
Comments |
Comment by Nikolai Kulakov [ 26/Mar/18 ] |
Of course, it would be great to see this Logout messages not only in the logs, but and in the quickfix.Application#fromAdmin as it usually happens - it allows us to handle this Logout message and set the expected sequence number (I already did this with some other counterparties). |
Comment by Christoph John [ 26/Mar/18 ] |
I think the problem is that the session is disconnected at 2018-03-26 10:16:20.885 (see event log) but the Logout message is coming in at 20180326-14:16:20.886 (SendingTime on message in your network log). This means the session is disconnected 1ms before the counterparty sends the Logout message (given that both your and the counterparty's clock is in sync). This can't be a general problem because this kind of disconnection (message seqnum too low) happens quite often with some counterparties and the messages show up in the log files. I don't know of any occurrences with current QuickFIX/J versions. So unless we can reproduce this in a unit test we cannot do anything about it. Sorry, but I can only repeat my statement from the other issues: please please please write to the mailing list before opening possible "bug" tickets. That way we can clarify if there really is a bug before issues are opened which in the end is saving your and our time. Thanks, |
Comment by Nikolai Kulakov [ 27/Mar/18 ] |
Please, take a look to this network dump of the incoming messages (on the our side): 10:16:19.846538 IP some_ip.port > our_ip.port: Flags [S.], seq 28365822, ack 2573233766, win 32000, options [mss 1380,nop,wscale 8], length 0 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 0030 0000 4000 3406 90c8 cfef 1bda ac14 [email protected]......... 0x0020: 1e22 6204 8259 01b0 d3fe 9960 6e66 7012 ."b..Y.....`nfp. 0x0030: 7d00 8f83 0000 0204 0564 0103 0308 }........d.... 10:16:20.885507 IP some_ip.port > our_ip.port: Flags [P.], seq 1:134, ack 103, win 4096, length 133 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 00ad 0000 4000 3406 904b cfef 1bda ac14 [email protected]...... 0x0020: 1e22 6204 8259 01b0 d3ff 9960 6ecc 5018 ."b..Y.....`n.P. 0x0030: 1000 216b 0000 383d 4649 582e 342e 3201 ..!k..8=FIX.4.2. 0x0040: 393d 3031 3039 0133 353d 3501 3334 3d38 9=0109.35=5.34=8 0x0050: 3801 3439 3d4a 4c51 4401 3532 3d32 3031 8.49=JLQD.52=201 0x0060: 3830 3332 362d 3134 3a31 363a 3230 2e38 80326-14:16:20.8 0x0070: 3836 0135 363d 6678 7732 0137 3839 3d35 86.56=fxw2.789=5 0x0080: 3101 3538 3d4d 7367 5365 714e 756d 2074 1.58=MsgSeqNum.t 0x0090: 6f6f 206c 6f77 2c20 6578 7065 6374 696e oo.low,.expectin 0x00a0: 6720 3531 2062 7574 2072 6563 6569 7665 g.51.but.receive 0x00b0: 6420 3101 3130 3d30 3735 01 d.1.10=075. 10:16:20.885543 IP some_ip.port > our_ip.port: Flags [R.], seq 134, ack 103, win 0, length 0 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 0028 0000 4000 3406 90d0 cfef 1bda ac14 .([email protected]......... 0x0020: 1e22 6204 8259 01b0 d484 9960 6ecc 5014 ."b..Y.....`n.P. 0x0030: 0000 3711 0000 0000 0000 0000 ..7......... 10:16:22.887575 IP some_ip.port > our_ip.33372: Flags [S.], seq 4175399090, ack 3053759955, win 32000, options [mss 1380,nop,wscale 8], length 0 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 0030 0000 4000 3406 90c8 cfef 1bda ac14 [email protected]......... 0x0020: 1e22 6204 825c f8df 88b2 b604 add3 7012 ."b..\........p. 0x0030: 7d00 878b 0000 0204 0564 0103 0308 }........d.... 10:16:23.845393 IP some_ip.port > our_ip.33372: Flags [P.], seq 1:134, ack 103, win 4096, length 133 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 00ad 0000 4000 3406 904b cfef 1bda ac14 [email protected]...... 0x0020: 1e22 6204 825c f8df 88b3 b604 ae39 5018 ."b..\.......9P. 0x0030: 1000 1b6f 0000 383d 4649 582e 342e 3201 ...o..8=FIX.4.2. 0x0040: 393d 3031 3039 0133 353d 3501 3334 3d38 9=0109.35=5.34=8 0x0050: 3901 3439 3d4a 4c51 4401 3532 3d32 3031 9.49=JLQD.52=201 0x0060: 3830 3332 362d 3134 3a31 363a 3233 2e38 80326-14:16:23.8 0x0070: 3436 0135 363d 6678 7732 0137 3839 3d35 46.56=fxw2.789=5 0x0080: 3101 3538 3d4d 7367 5365 714e 756d 2074 1.58=MsgSeqNum.t 0x0090: 6f6f 206c 6f77 2c20 6578 7065 6374 696e oo.low,.expectin 0x00a0: 6720 3531 2062 7574 2072 6563 6569 7665 g.51.but.receive 0x00b0: 6420 3201 3130 3d30 3736 01 d.2.10=076. 10:16:23.845427 IP some_ip.port > our_ip.33372: Flags [R.], seq 134, ack 103, win 0, length 0 0x0000: 000c 2909 731b 001f 9edf 3ebf 0800 4500 ..).s.....>...E. 0x0010: 0028 0000 4000 3406 90d0 cfef 1bda ac14 .([email protected]......... 0x0020: 1e22 6204 825c f8df 8938 b604 ae39 5014 ."b..\...8...9P. 0x0030: 0000 2f19 0000 0000 0000 0000 ../......... ... As I see, there are constantly repeating 3 records:
I think that the time between our and the counterparty sides is not so good synchronized to rely on this (in fact, we have some problems with time synchronization on the used machine). Thanks and my apologize if I'm using not the best procedure of issues notifying, |
Comment by Christoph John [ 27/Mar/18 ] |
Hi Nikolai, Thanks, |
[QFJ-946] Improve misleading error message during Logon Created: 21/Mar/18 Updated: 26/Mar/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.4 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Nikolai Kulakov | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Linux |
Description |
It seems that the network connection between my company and our partner is very fast and the QuickFix/J engine suppose that their answering Logon message is coming before we send our Logon message: 8=FIX.4.2^A9=86^A35=A^A34=1^A49=fxw2^A52=20180321-14:09:01.447^A56=JLQD^A98=0^A108=30^A141=Y^A554=password^A10=253^A <2018-03-21 10:09:00.329 So, I do not see any way to establish the connection. |
Comments |
Comment by Nikolai Kulakov [ 21/Mar/18 ] |
After further investigation of the problem, I guess that the problem is not in the speed of the connection, but in the fact that we set ResetSeqNumFlag=Y in the our Logon message but in the answering Logon message it is not presented (so, the priority of the problem can be lowered). The code that is generating this error is in the method quickfix.Session#nextLogon: // Check for proper sequence reset response if (state.isResetSent() && !state.isResetReceived()) { disconnect("Received logon response before sending request", true); } It can be a problem indeed, but the disconnect message seems totally misleading for me. |
Comment by Christoph John [ 21/Mar/18 ] |
I don't know but what sense does it make to reset the sequence numbers only on one side? So I guess the check might have a point. Unless I'm missing something. |
Comment by Nikolai Kulakov [ 21/Mar/18 ] |
Just a real situation: our partner is ready to reset the sequence number on the one side only. It seems that using ResetSeqNumFlag for this purpose it is not nice by FIX standards, but also it is a fact that this situation is not related with the "Received logon response before sending request". |
Comment by Christoph John [ 21/Mar/18 ] |
I think it does not make sense to open a bug ticket for every counterparty that does not conform to normal FIX standard. IMHO the best would be to direct such questions to the mailing list. Please post follow up questions and your answers there. Thanks |
Comment by Nikolai Kulakov [ 21/Mar/18 ] |
John, I agree that it is no sence to open a bug ticket for a counetrparty that does not conform regular FIX standard (of course, I'll discuss this situation with them directly). But the ticket is about another thing and I'm really sorry if I'm describing it not clear. |
[QFJ-918] Initiator failover funtion Created: 03/Apr/17 Updated: 03/Apr/17 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.3 |
Fix Version/s: | None |
Type: | Other | Priority: | Default |
Reporter: | Cheng Guo | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ | ||
Environment: |
Linux |
Description |
Hi, by the way, i also developing to send the FIX msg to IBM MQ, could you suggest if QuickFIXJ has the persistence module/ api to store the un-delivery msg if MQ down, and then send out once MQ is up? |
Comments |
Comment by Christoph John [ 03/Apr/17 ] |
You'll need to implement the HA functionality by yourself. You could use a database, synced file system or clustered in-memory cache to store the session information. But the whole failover decision has to be implemented by your application. Since JIRA is the bug tracker you could ask on the mailing list whether anyone has already implemented something similar: https://lists.sourceforge.net/lists/listinfo/quickfixj-users Thanks, |
[QFJ-930] FIXSettings not set for new DataDictionary objects created by DefaultDataDictionaryProvider Created: 18/Jul/17 Updated: 29/Jan/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Sachin Agrawal | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Attachments: | quickfix_930.rar |
Description |
While DefaultSessionsFactory.processFixtDataDictionaries adds ApplicationDictionary to dataDictionaryProvider it uses beginStringQualifier only. While method of MessageUtils method used session,messageString both to get the ApplVerId. this results in mismatched key inside DefaultDataDictionaryProvider and DefaultDataDictionaryPovider need to create a new DataDictionary but FIX sesion settings are not set here at all, which are set by DefaultSessionsFactory while executing processFixtDataDictionaries inside createDataDictionary. Please let me know in case this is intentional. |
Comments |
Comment by Christoph John [ 18/Jul/17 ] |
I am sorry, but I cannot follow your description. Could you maybe describe the problem with some code examples or a unit test? |
Comment by Sachin Agrawal [ 19/Jul/17 ] |
Thanks Chris for looking into,i now try to be more specific. Please look into DeafultSessionFactory.processFixtDataDictionaries, we find following piece of code, please note here that toApplVerID is used to create the instance of ApplVerID before the applicationDictionary is added to the datadictionaryProvider.
final DataDictionary dd = createDataDictionary(sessionID, settings, key,
beginStringQualifier);
dataDictionaryProvider.addApplicationDictionary(MessageUtils.toApplVerID(beginStringQualifier), dd);
and createDataDictionary method reads the settings private DataDictionary createDataDictionary(SessionID sessionID, SessionSettings settings, String settingsKey, String beginString) throws ConfigError, FieldConvertError { final String path = getDictionaryPath(sessionID, settings, settingsKey, beginString); final DataDictionary dataDictionary = getDataDictionary(path); if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER)) { dataDictionary.setCheckFieldsOutOfOrder(settings.getBool(sessionID, Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER)); } if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES)) { dataDictionary.setCheckFieldsHaveValues(settings.getBool(sessionID, Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES)); } if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS)) { boolean chkUnorderedGrpFields = settings.getBool(sessionID, Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS); settings.getLogger().error("Msg SttString "+Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS +" value "+ chkUnorderedGrpFields +" dataDictionary " +dataDictionary.toString() +" beginString "+beginString); dataDictionary.setCheckUnorderedGroupFields(chkUnorderedGrpFields); } if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_USER_DEFINED_FIELDS)) { dataDictionary.setCheckUserDefinedFields(settings.getBool(sessionID, Session.SETTING_VALIDATE_USER_DEFINED_FIELDS)); } if (settings.isSetting(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS)) { dataDictionary.setAllowUnknownMessageFields(settings.getBool(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS)); } return dataDictionary; } Please note that DefaultDataDictionaryProvider uses ApplVerID as the key of the map. While there is a method called MessageUtils.parse(Session session, String messageString), which uses getApplVerID(session, messageString) to create the ApplVerId and uses the same to fetch the applicationDictionary from dataDictionaryProvider. MessageUtils.parse(Session session, String messageString) ..... final String beginString = getStringField(messageString, BeginString.FIELD); final String msgType = getMessageType(messageString); ApplVerID applVerID; if (FixVersions.BEGINSTRING_FIXT11.equals(beginString)) { applVerID = getApplVerID(session, messageString); } else { applVerID = toApplVerID(beginString); } final MessageFactory messageFactory = session.getMessageFactory(); final DataDictionaryProvider ddProvider = session.getDataDictionaryProvider(); final DataDictionary sessionDataDictionary = ddProvider == null ? null : ddProvider.getSessionDataDictionary(beginString); final DataDictionary applicationDataDictionary = ddProvider == null ? null : ddProvider.getApplicationDataDictionary(applVerID); .... So when an application message is received a new DataDictionary is created by DefaultDataDictionaryProvider, look at method public DataDictionary getApplicationDataDictionary(ApplVerID applVerID)
so the problem is that this new DataDictionary created by DefaultdataDictionaryProvider does not copy the FIX configurations i.e. settings. Hence even though the application dev has set the settings they are not picked while parsing messages. Please ask further clarifications. Thanks, |
Comment by Christoph John [ 19/Jul/17 ] |
Hi, I have changed https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/DefaultSessionFactoryTest.java (method testFixtDataDictionaryConfiguration) slightly to set "AllowUnknownMessageFields" to true. @Test public void testFixtDataDictionaryConfiguration() throws Exception { SessionID sessionID = new SessionID(FixVersions.BEGINSTRING_FIXT11, "SENDER", "TARGET"); setUpDefaultSettings(sessionID); settings.setBool(sessionID, Session.SETTING_USE_DATA_DICTIONARY, true); settings.setString(sessionID, Session.SETTING_TRANSPORT_DATA_DICTIONARY, "FIXT11.xml"); settings.setString(sessionID, Session.SETTING_DEFAULT_APPL_VER_ID, "FIX.4.2"); settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY, "FIX42.xml"); settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY + "." + FixVersions.BEGINSTRING_FIX40, "FIX40.xml"); settings.setString(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS, "Y"); // added Session session = factory.create(sessionID, settings); DataDictionaryProvider provider = session.getDataDictionaryProvider(); assertThat(provider.getSessionDataDictionary(sessionID.getBeginString()), is(notNullValue())); final DataDictionary applicationDataDictionary = provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX42)); assertThat(applicationDataDictionary, is(notNullValue())); assertThat(provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX40)), is(notNullValue())); System.err.println("XXXX " + provider.getSessionDataDictionary(sessionID.getBeginString()).isAllowUnknownMessageFields() ); System.err.println("XXXX " + applicationDataDictionary.isAllowUnknownMessageFields() ); quickfix.fixt11.Logon logon = new quickfix.fixt11.Logon(new EncryptMethod(EncryptMethod.NONE_OTHER), new HeartBtInt(30), new DefaultApplVerID(ApplVerID.FIX42)); MessageUtils.parse(session, logon.toString()); This gives the following as output: XXXX true XXXX true So the setting is picked up by both dictionaries. Then I stepped into MessageUtils.parse(session, logon.toString()) with the debugger and could also see that the setting is true on the applicationDataDictionary. So maybe I am taking a different code path than you? BTW I tested it on master, i.e. with version 1.7.0-SNAPSHOT. Cheers, |
Comment by Sachin Agrawal [ 20/Jul/17 ] |
Hi Chris, I understand in your use case a new dictionary is never created, but there are some configurations in which a scenario occurs which creates a new dictionary by DefaultDataDictionaryProvider, then the settings will not be copied. 1) I correct the Fix configurations so that a new dictionary is never created. I see you are suggesting option 1. --Regards, |
Comment by Christoph John [ 20/Jul/17 ] |
Hi, when there are such configurations then please give an example configuration. Otherwise I don't know how to reproduce the problem. Thanks, |
Comment by Sachin Agrawal [ 29/Jan/18 ] |
Attached is the zip src files with change Add fix for correctly reading the settings in all the cases, i.e. whenever DataDictionary object is created from SessionFactory or MessageUtils.parse |
Comment by Sachin Agrawal [ 29/Jan/18 ] |
only improvement the changes i uploaded have is that whereever the DataDictionary object is created, the existing settings of current session are always applied on DataDictionary created. |
Comment by Christoph John [ 29/Jan/18 ] |
Thanks for the file. Are you maybe able to generate a patch or even better open a pull request on github? |
[QFJ-880] qfj doesn't send the next batch when ResendRequestChunkSize > 0 Created: 26/Feb/16 Updated: 10/Oct/18 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.0 |
Fix Version/s: | None |
Type: | Other | Priority: | Default |
Reporter: | Xiaojun Zhang | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | CME, resend | ||
Environment: |
FIX4.2 |
Description |
Quickfixj seems to send the 1st resend batch only after receiving a seq reset. i.e. 15:28:37.142 [QFJ Message Processor] - fromAdmin PNPBBBN: 8=FIX.4.29=10735=134=44749=CME50=G52=20160225-20:28:37.22356=PNPBBBN57=DROPCOPY143=US,NY369=45621112=IDiky5miuo10=025 15:28:37.142 [QFJ Message Processor] - toAdmin PNPBBBN: 8=FIX.4.29=10735=034=4562249=PNPBBBN50=DropCopy52=20160225-20:28:37.14256=CME57=G142=US,NY143=CME112=IDiky5miuo10=004 I manually reduced the incoming seq no to 428 15:29:08.120 [QFJ Message Processor] - toAdmin PNPBBBN: 8=FIX.4.29=10535=234=4562449=PNPBBBN50=DropCopy52=20160225-20:29:08.12056=CME57=G142=US,NY143=CME7=42816=43710=188 15:29:08.148 [QFJ Message Processor] - fromAdmin PNPBBBN: 8=FIX.4.29=13635=434=42843=Y49=CME50=G52=20160225-20:29:08.25556=PNPBBBN57=DROPCOPY122=20160225-20:29:08.255143=US,NY369=4562436=449123=Y10=001 I expect quickfix to send the next batch (438-447) but it sent a normal heartbeat instead. 15:29:38.704 [QFJ Timer] - toAdmin PNPBBBN: 8=FIX.4.29=9235=034=4562549=PNPBBBN50=DropCopy52=20160225-20:29:38.70456=CME57=G142=US,NY143=CME10=069 |
Comments |
Comment by Christoph John [ 26/Feb/16 ] |
Sorry, this is a little hard to follow. I don't know about the command "fix PNPBBBN". Is this some custom stuff that you are executing? Actually, I would expect QFJ to send a Logout if it received too low a sequence number. The last log line is half an hour later than the log line before. What happened in between? Actually, there are some unit tests around the chunked resend requests behaviour. But of course, this does not mean that there could be no bugs inside that portion of the code. But I cannot really follow your example. Do you have a unit test or some more concise steps which can be followed? |
Comment by Xiaojun Zhang [ 26/Feb/16 ] |
Sorry the log is a bit hard to read. Hope this helps. |
Comment by Xiaojun Zhang [ 26/Feb/16 ] |
Sorry there was a typo in 6) client should send the next resend request begin=12 end=20 but it didn't |
Comment by Christoph John [ 26/Feb/16 ] |
If you look at the incoming sequence reset you see the field 36/NewSeqNo set to 449. So that is why there are no more messages resent. This should actually also be a message in your event log. Something along the lines of: "received sequence reset to 449." |
Comment by Xiaojun Zhang [ 26/Feb/16 ] |
Hmm i am reading the FIX doc. Actually I was connecting to CME drop copy and encountered this issue. |
Comment by Christoph John [ 26/Feb/16 ] |
If you send a resend request and the counterparty sends SequenceReset with NewSeqNo = 449 then QFJ will continue with that incoming seqnum. That means that the gap has been filled up to that seqnum, yes. Probably there were only session messages in between, or messages that CME does not want to resend. |
Comment by Christoph John [ 26/Feb/16 ] |
NB: I think this belongs more onto the mailing list or with CME support than here. It's no bug after all. |
Comment by Xiaojun Zhang [ 27/Feb/16 ] |
Sorry I still don't quite understand the logic here. If my understanding is not correct can you please give me an example how chunked resend should work? Appreciate your help. |
Comment by Xiaojun Zhang [ 01/Mar/16 ] |
If you don't think it is a bug I will add code in my Application class to override the logic. I just want to point out that current logic does NOT work for CME iLink and Drop Copy 4.0 which place a 2500 limit on resend request. QFJ would lose all messages after the first batch. It makes more sense to assume the corresponding chunk gap has been filled based on the sequence reset, not the whole gap. |
Comment by Christoph John [ 01/Mar/16 ] |
Sorry, I totally forgot about this one. What logic do you want to override? Actually, you do not need to override any of the session logic.
The NewSeqNo on a sequence reset is the next-to-be sequence number of the other communication side.
NewSeqNo should be 11.
Why does the seq no decrease? It is as follows: you connect to the counterparty. Your expected target seq no is 1. The other side comes in with 20. Now you send a resend request from 1-10. I know that there are several users which use QFJ to connect to CME. The last issue around chunked resend requests I remember was Please write a mail to the mailing list if you have further questions. https://lists.sourceforge.net/lists/listinfo/quickfixj-users Thanks. |
Comment by John [ 07/Apr/16 ] |
I can confirm that I am seeing the same behavior when connecting to the CME's Drop Copy 4.0. It appears this issue arises due to the way in which the CME increments tag 36 of SequenceReset messages and the way QuickFIX/J handles the sending of ResendRequest messages. The following code is an excerpt from the Session class in the quickfix package which I believe is where the logic breaks down. It appears that QuickFIX/J will attempt to send chunks of ResendRequests depending upon when it receives incoming SequenceReset messages.The code below requires the value of the newSequence variable to be less than the EndSeqNo value defined in the range of messages that the client is supposed to fetch. This case fails and some ResendRequest messages will not be sent. Would it be safe to remove this logic from the if statement allowing subsequent RequestsRequests to be sent to the counterparty despite the fact that the newSequence value will be past the range of messages defined in the range variable? If so, the code would also need to be modified to not leverage the value of the newSequence variable when sending the ResendRequest to define the range of sequence numbers to send in the subsequent ResendRequest. newSequence < range.getEndSeqNo() private void nextSequenceReset(Message sequenceReset) throws IOException, RejectLogon, if (!verify(sequenceReset, isGapFill, isGapFill)) { return; } if (validateSequenceNumbers && sequenceReset.isSetField(NewSeqNo.FIELD)) { getLog().onEvent( } getLog().onErrorEvent( generateReject(sequenceReset, SessionRejectReason.VALUE_IS_INCORRECT, |
Comment by Xiaojun Zhang [ 07/Apr/16 ] |
John, I received your email. Do you have a non-personal email address? My corporate email doesn't allow sending to gmail, hotmail, etc. |
Comment by Christoph John [ 08/Apr/16 ] |
I very briefly read through http://www.cmegroup.com/confluence/display/EPICSANDBOX/Drop+Copy+Session+Layer+-+Resend+Request and I do not see anything that is contradictory to how QFJ interprets the NewSeqNo tag. However, it seems that CME is implementing something very custom. A test case would definitely help here. |
Comment by John [ 09/Apr/16 ] |
Chrstoph, What format would you prefer to see a test case in? I could provide the FIX messaging log so that it might be easier to see how the CME is increasing the sequence numbers in the SequenceReset messages they send in the middle of sending ResendRequest chunks past the end of the original ResendRequest range such that QuickFIX/J defines after logging on which causes some of the ResendRequest chunks to not be sent to the exchange. As I mentioned in my earlier post, I believe the issue is in the code snippet I posted above where upon receiving SequenceReset messages QuickFIX/J will determine if a new ResendRequest chunk message needs to get sent out based on the following criteria in the nextSequenceReset method of the Session class by checking if newSequence < range.getEndSeqNo(). Since this fails, some ResendRequests fail to ever get sent out. Say the defined ResendRequest range is 1-10000. I would assume this to be custom logic that the CME is implementing on their end. I think the difference comes down to the fact that QuickFIX/J appears to rely more on the newSequence number that is received on an incoming SequenceReset message to dictate whether further ResendRequest chunks will be sent out and the CME may send out a SequenceReset message taking you past the originally defined EndSeqNo range. I am not a FIX expert so I can't say if this solution would work but it seems that if the logic for sending chunks of ResendRequest messages is changed such that each of the ResendRequest chunk messages is defined immediately after the need for a ResendRequest arises and put into a queue (or some other object) QuickFIX/J could then pop the each ResendRequest message off the queue once it receives all of the messages for a given ResendRequest from the exchange and continue onto the next one and decouple itself in that way from the newSequenceNumber. This might ensure that any of the chunks of ResendRequest messages that need to get sent out don't get skipped. This might have negative implications as it might not be the standard logic for communicating via FIX however so it might not be a feasible solution however. It would be really great if this functionality could be modified in QuickFIX/J as this is an extremely useful engine however for users attempting to use QuickFIX/J with CME's Drop Copy 4 this case will cause problems in a recovery scenario. |
Comment by Christoph John [ 29/Apr/16 ] |
Hi John, yes, a message log would be a good starting point. From this I should hopefully be able to construct an automated test case. Thanks, Chris. |
Comment by Christoph John [ 26/Aug/16 ] |
Hi John, Xiaojun Zhang: is there still the need to support this? Do you have any message logs? |
Comment by John [ 01/Sep/16 ] |
Hi Christoph John, I can confirm that there is a need to support this functionality. To be clear, I was experiencing this issue when working with the 1.6.0 release of QuickFIX/J and have not tested with more recent releases. I don't have any message logs available currently aside from what Xiaojun posted earlier in this issue. Thanks |
Comment by Christoph John [ 01/Sep/16 ] |
Hi John, OK, if this functionality was implemented could you test it against CME to verify if it works? Do they have some sort of certification test? |
Comment by Dmitry Razumov [ 10/Oct/18 ] |
Hey John |
[QFJ-889] Leading zeros in int and float fields Created: 23/Apr/16 Updated: 28/Apr/16 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Roman Liverovskiy | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ | ||
Environment: |
Fedora 23 x86_64, Windows 7 64 bit SP1 |
Description |
I use quickfixj-examples-executor-1.6.0.jar as FIX 5.0 server. Server does not respond on any client messages if there are leading zeros in int or float fields in client message, for example if there are leading zeros in sequnce number in message header (in login message or in heart beat message). |
Comments |
Comment by Christoph John [ 28/Apr/16 ] |
Could you please attach some logs? Thanks |
[QFJ-867] Thread handling after exiting QuickFIX Created: 06/Nov/15 Updated: 16/Nov/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.1 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Martin Vrábel | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ | ||
Environment: |
Apache Tomcat 8 on Windows 10, Spring 4 web application |
Description |
Hi, I stumbling over this thing over and over. When I stop my application I always get error messages in output about spawning a new thread during execution of application but failing to remove it when application stops: ``` It has something to do with `UtcTimestampConverter` but I have no idea why is it happening. I correctly shutdown all threads I manually start. |
Comments |
Comment by Christoph John [ 16/Nov/15 ] |
Hmm, looks more like a false positive to me. But to be honest I never tried running QFJ in a managed environment or appserver. |
Comment by Martin Vrábel [ 16/Nov/15 ] |
Would it be possible to call a method or something from outside of quickfix to manually shutdown all spawned threads when exiting my webapp? |
Comment by Christoph John [ 16/Nov/15 ] |
From what I understand from the output it does not have to do anything with threads but ThreadLocal variables that are still held. But AFAIK there is no built-in mechanism to clean them up. |
[QFJ-863] In the quickfix version that we use (1.5.3b), an invalid checksum error does not result in a session reject Created: 12/Oct/15 Updated: 12/Oct/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Saurabh Desai | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
The scenario: The initiator parses the following message with quickfix.MessageUtils.parse(), and sends it to the acceptor. 8=FIX.4.4|9=238|35=AE|31=1.35|32=1000|48=EURUSD|22=6|55=EURUSD|64=20131106|194=1.35|15011=1.33|15012=1.33|461=RCSXXX|487=2|571=1105200000001234|829=0|572=M1006050412587|552=1|54=1|37=SYS1:4545556002|11=SYS2:1006050412587|526=SSP:BDID001|453=2|448=a123456|447=D|452=3|802=2|523=5730607|803=10|523=5730606|803=15|1=123456-789| It is not a valid FIX message, as we have 453<NoPartyIDs>=2, but there is only one group in the message. We would expect the acceptor to return a validation error, but instead the following happens: 1) An error message is logged: 2013-12-07 15:47:30,919 [SocketAcceptorIoProcessor-0.0] ERROR [] q.mina.acceptor.AcceptorIoHandler - Invalid message: Expected CheckSum=29, Received CheckSum=28 in 8=FIX.4.4?9=367?35=AE?34=82?49=SIDE1?52=20131207-14:47:30.618?56=SIDE2?22=6?31=1.35?32=1000?48=EURUSD?55=EURUSD?64=20131106?194=1.35?461=RCSXXX?487=2?571=1105200000001234?572=M1006050412587?829=0?15011=1.33?15012=1.33?552=1?54=1?37=SYS1:4545556002?11=SYS2:1006050412587?526=SSP:BDID001?453=2?448=a123456?447=D?452=3?802=2?523=5730607?803=10?523=5730606?803=15?1=123456-789?10=028? 2) The acceptor does not send any response and does not increase the message sequence count, either. The suspect: The acceptor validates the fix message with the checksum(String s) method of Message class. It calculates 29. But the checksum calculated on the initiator side is 28. The initiator calculates the checksum with a different method, which is called by the toString() method of Message class. This is the method which does the calculation (in FieldMap class) - my comments added: int calculateTotal() { int result = 0; result += field.getTotal(); // JK: groups variable contains the group tags, but only the tag number (the key is Integer, not Field - why?) } return result; So the checksum will be the checksum of a different message actually, where 453=1. |
[QFJ-852] Missing synchronized blocks discovered in code review. Created: 01/Jul/15 Updated: 04/Jul/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Nathan Tippy | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | sequnece |
Attachments: | quickfixjQFJ-852.patch |
Description |
In the class Session in the following method private boolean verify(Message msg, boolean checkTooHigh, boolean checkTooLow) we find that synchronized (state.getLock()) is used to ensure atomic updates to the multiple fields in the state object. If this is needed then we must also check to ensure partial or dirty reads are avoided. This would happen any place that we call for more than one field of the state and assume the fields remain consistent with one another until we make a second or third request for other fields. Directly after this same block the usage of isChunkedResendRequest, getCurrentEndSeqNo and getEndSeqNo run the risk of partial (dirty) reads because they are out side the sync. This same problem also exists here where it may be the cause of other more serious issues. And to a a lesser degree in |
Comments |
Comment by Nathan Tippy [ 02/Jul/15 ] |
For immediate use I apply this patch file to be safe. However this should be re-visited with a more elegant fix. |
Comment by Christoph John [ 03/Jul/15 ] |
Thanks for the patch! |
Comment by Nathan Tippy [ 04/Jul/15 ] |
No I have not seen any problems but after discovery in code review I was uncomfortable running it without a fix. I have not seen any negative consequences to applying the patch. So it was a defensive move. |
[QFJ-847] Transport Data Dict vs App Data Dict - settings not passed to App Data Dict Created: 19/May/15 Updated: 19/May/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.6.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Chris | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
N/a its in the java code |
Description |
The Quickfixj settings are not being passed to the ApplicationDataDictionary. This seems to be because the transport dictionary is created via Whereas the app DataDictionary is created in Default Data Dictionary |
Comments |
Comment by Chris [ 19/May/15 ] |
Not sure how best to address this - should the application DataDictionary be created when the Transport one is? Or should the settings be made available to the DefaultDataDictionaryProvider class? |
[QFJ-843] Some times QuickFix Message received on Fix session is processed first by Application thread and later by socket connection Processor Thread causing sequence error by later processing Created: 04/May/15 Updated: 07/May/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | anurag jain | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | sequnece, session |
Description |
Message on Fix session received by both Application thread (ProcessQ) and socket connection Processor Thread. Please FInd application logs as below .. |
Comments |
Comment by anurag jain [ 04/May/15 ] |
Please note that ProcessQ Thread is application thread which read objects from application Queue and not the the Quick Fix internal queue . |
Comment by Christoph John [ 04/May/15 ] |
When this happens, do you have to restart your application or does the problem correct itself? Are you starting/stopping your Initiator/Acceptor at certain times? This sounds a little like |
Comment by anurag jain [ 04/May/15 ] |
Sorry rephrasing the above JIRA:
All the log files below: 2015-04-28 14:16:03,856 [ProcessQ] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A 2015-04-28 14:16:03,856 [SocketConnectorIoProcessor-0.0] INFO quickfixj.msg.incoming - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=1500^A35=X^A34=158757^A49=ANZD_SANDPIT_MD^A52=20150428-06:16:03.814^A56=MDAQ_DEV2_SANDPIT_MD^A262=9^A268=26^A279=2^A269=0^A278=7usB0^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA0^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB1^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA1^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB2^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA2^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB3^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA3^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB4^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA4^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB5^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA5^A55=USD/CHF^A15=USD^A279=2^A269=0^A278=7usB6^A55=USD/CHF^A15=USD^A279=2^A269=1^A278=7usA6^A55=USD/CHF^A15=USD^A279=0^A269=0^A278=7utB0^A55=USD/CHF^A270=0.95388^A15=USD^A271=1000000^A346=1^A279=0^A269=1^A278=7utA0^A55=USD/CHF^A270=0.95413^A15=USD^A271=1000000^A346=1^A279=0^A269=0^A278=7utB1^A55=USD/CHF^A270=0.95385^A15=USD^A271=2000000^A346=1^A279=0^A269=1 ^A278=7utA1^A55=USD/CHF^A270=0.95416^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB2^A55=USD/CHF^A270=0.95382^A15=USD^A271=2000000^A346=1^A279=0^A269=1^A278=7utA2^A55=USD/CHF^A270=0.95419^A15=USD^A271=2000000^A346=1^A279=0^A269=0^A278=7utB3^A55=USD/CHF^A270=0.95380^A15=USD^A271=5000000^A346=1^A279=0^A269=1^A278=7utA3^A55=USD/CHF^A270=0.95421^A15=USD^A271=5000000^A346=1^A279=0^A269=0^A278=7utB4^A55=USD/CHF^A270=0.95377^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA4^A55=USD/CHF^A270=0.95424^A15=USD^A271=10000000^A346=1^A279=0^A269=0^A278=7utB5^A55=USD/CHF^A270=0.95374^A15=USD^A271=10000000^A346=1^A279=0^A269=1^A278=7utA5^A55=USD/CHF^A270=0.95427^A15=USD^A271=10000000^A346=1^A10=041^A 2015-04-28 14:16:03,857 [QFJ Message Processor] INFO quickfixj.msg.outgoing - FIX.4.2:MDAQ_DEV2_SANDPIT_MD->ANZD_SANDPIT_MD: 8=FIX.4.2^A9=141^A35=5^A34=11955^A49=MDAQ_DEV2_SANDPIT_MD^A52=20150428-06:16:03.857^A56=ANZD_SANDPIT_MD^A58=MsgSeqNum too low, expecting 158758 but received 158757^A10=030^A Could you please tell us:
|
Comment by Christoph John [ 05/May/15 ] |
Could you please answer my questions from the last comment? |
Comment by anurag jain [ 06/May/15 ] |
Please find my answer here:
|
Comment by Christoph John [ 06/May/15 ] |
OK, without the stack dump I cannot really tell anything. Just let me know once you have it. And I guess you are certain that the counterparty did not just send the same message twice? |
Comment by anurag jain [ 07/May/15 ] |
1) We are using default Socket Initiator. 4)and yes there is no replay of same message from counter party . |
Comment by Christoph John [ 07/May/15 ] |
Hmm, now that I see that you are using SSL I am asking myself if it could be the same problem as in QFJ-745. However, there is not really much information to find the problem and the issue creator did not come back with answers. |
[QFJ-793] When RuntimeException is thrown within a call-back and DisconnectOnError=Y the session does not get disconnected automatically Created: 12/Jun/14 Updated: 23/Aug/16 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Andrzej Hajderek | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Attachments: | SessionTest.java.add SessionTest.java.add.fixed |
Description |
Hi, When a RuntimeException is thrown within a call-back and the "DisconnectOnError" parameter is set to "Y" the session does not get disconnected automatically unless the "RejectMessageOnUnhandledException" parameter is also set to "Y". This dependency is not documented, but even if it was, it wouldn’t make too much sense. It seems to be logical to understand an occurrence of a RuntimeException within a call-back as an error condition on its own, which should be sufficient to trigger the disconnection logic dictated by the "DisconnectOnError" parameter (regardless of any other settings). Currently, if RejectMessageOnUnhandledException=N, the engine simply ignores the message for which the exception was thrown even though DisconnectOnError=Y (although it reports the exception as an “error” in the log). Test code attached. Please note that probably all call-backs exhibit the same behaviour, not just the fromApp() call-back used in the TestApp class. It is also possible that the “ResetOnError” parameter is treated in a similar manner. Regards, |
Comments |
Comment by Andrzej Hajderek [ 13/Jun/14 ] |
Attached a fixed version of the test. |
[QFJ-798] Refactor Session construction in test code Created: 13/Jun/14 Updated: 13/Jun/14 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Andrzej Hajderek | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
Currently to create some of the Session related test cases it is necessary to use constructs like this: Session session = new Session(new UnitTestApplication(), new MemoryStoreFactory(), , false, disconnectOnError, false, true, false, true, false, null, This kind of code turns from an asset into a liability pretty quickly, so the sooner it is rationalised the better. The Session constructor itself isn't very pretty either with its 30+ parameters. |
[QFJ-797] Add a clean and safe mechanism to disconnect a session from within a call-back Created: 13/Jun/14 Updated: 13/Jun/14 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Andrzej Hajderek | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
Hi, There is currently no clean and safe mechanism to completely stop and disconnect a session from within a call-back (e.g. when processing an incoming message inside fromApp()) with the effect that the session is immediately disconnected, no reject messages are sent back to the counter-party and no change to the session store is made. Such a mechanism is useful for implementing failure handling scenarios, e.g. when a target database becomes suddenly unavailable, or the application needs to be shutdown in emergency. In such cases it's best to close the TCP socket immediately and keep the session state unchanged. One could try to use something like SocketInitiatior.stop(true), but it's not generic (does not work for acceptors) and it results in a call to Session.disconnect(). Then, if Session.disconnect() is called from fromApp(), we get an immediate call to the onLogout() callback - a call is made from within the fromApp() call-back into the onLogout() call-back. This isn't a particularly safe scenario, especially when in most cases the onLogout() call is made by from QuickFIX/J threads. Another solution is to throw a RuntimeException within the call-back method, but it's not reliable right now (QFJ-793, I propose to add a special a subclass of RuntimeException, e.g. DisconnectSessionExcepion, which when thrown from a call-back method would notify QuickFIX/J that the session should be disconnected immediately. In such case only an informational message would be send to the log instead of error with a stack trace. Regards, |
[QFJ-796] There is no mechanism to clear or update the data dictionary cache Created: 13/Jun/14 Updated: 13/Jun/14 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Andrzej Hajderek | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
A data dictionary file loaded into memory is cached without any option to reload the file after its contents has been updated. Worse - it's not even possible to clear the cache. This means that in order to use an updated data dictionary it is necessary to restart the application. See: quickfix.DefaultSessionFactory.dictionaryCache |
[QFJ-794] Misleading message logged when RuntimeException thrown within a call-back: "Rejecting message: ..." Created: 12/Jun/14 Updated: 23/Aug/16 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Andrzej Hajderek | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
This is in relation to QFJ-793. When both settings RejectMessageOnUnhandledException and DisconnectOnError are set to Y and a RuntimeException is thrown within a call-back, QuickFIX/J will write to the log output messages like below: <20140612-16:25:06, FIX.4.4:SENDER->TARGET, error> (Rejecting message: java.lang.RuntimeException: ... (with more details and the message) 2014-06-12 18:25:06 quickfix.Session disconnect INFO: [FIX.4.4:SENDER->TARGET] Disconnecting: Auto disconnect This is misleading because nothing is actually rejected (no reject message is sent back to the sender). The session is simply disconnected. I believe a slightly different wording should be used for this scenario, e.g "Failed to process message" instead of "Rejecting message". |
[QFJ-801] Validation fail on ClassCastException if using XML Created: 14/Jul/14 Updated: 14/Jul/14 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Benoit Xhenseval | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | validation | ||
Environment: |
Mac OSX |
Attachments: | IOIValidation.java |
Description |
It appears that the Message validation fails on a ClassCastException if the message contains the XmlDataLen tag. Example: It seems that the validation code only expects StringField but the XmlDataLen is an IntField. The output is: Testing WITHOUT XML 8=FIXT.1.19=7935=649=Sender56=Target15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=079 Testing WITH XML 8=FIXT.1.19=10335=649=Sender56=Target212=12213=<a>Hello</a>15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=086 Exception in thread "main" java.lang.ClassCastException: quickfix.field.XmlDataLen cannot be cast to quickfix.StringField at quickfix.DataDictionary.iterate(DataDictionary.java:668) at quickfix.DataDictionary.validate(DataDictionary.java:653) at quickfix.DataDictionary.validate(DataDictionary.java:624) at quickfix.DataDictionary.validate(DataDictionary.java:606) at fixfun.IOIValidation.validateIoi(IOIValidation.java:45) at fixfun.IOIValidation.validateIoiWithXml(IOIValidation.java:37) at fixfun.IOIValidation.main(IOIValidation.java:78) |
Comments |
Comment by Benoit Xhenseval [ 14/Jul/14 ] |
test case. |
Comment by Christoph John [ 14/Jul/14 ] |
It turned out that the setField( int, Field<?> ) method should rather not be used with non-String tags. Rather use setField( Field ), e.g.: Of course, there should be means to prevent your original error. Either setField( int, Field<?> ) should convert the values (as setField( Field ) does) or the validate-method should not assume that all fields are Strings. |
[QFJ-743] JdbcLog doesn't take into consideration the sessionID to verify if a parameter is set like JdbcMessage does Created: 13/May/13 Updated: 06/Nov/15 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Leandro Garcia Herrera | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | JdbcLog, QuickfixJ, jdbc | ||
Environment: |
Windows 7, using JBoss and camel-quickfix |
Attachments: | JdbcLog.java TemaJdbcLog.java |
Description |
I'm trying to configure JdbcLogFactory dynamically passing a SessionSettings object as argument to constructor. All my configurations are under an especific session ID and I have no configurations under default session. Looking at the source code of JdbcLog I was able to confirm that only the default session configurations are taken into consideration instead of checking first in session ID configurations informed via constructor, wich I was expecting. The JdbcStore class does this verification by session ID and I think JdbcLog should work in the same way. |
Comments |
Comment by Christoph John [ 09/Jan/14 ] |
Would it be OK for you to attach the patch to this ticket? I do not have the necessary rights to grant you SVN access but could integrate the patch shortly. |
Comment by Leandro Garcia Herrera [ 06/Nov/15 ] |
Hi, Christoph. Look if it helps. I've attached the patch as you requested. Best regards. |
Comment by Leandro Garcia Herrera [ 06/Nov/15 ] |
bug fix |
[QFJ-733] Provide customisable hooks into the core QuickFIX/J workflow Created: 11/Mar/13 Updated: 21/Mar/13 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Ryan Lea | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ, modules, performance |
Description |
Background: In order to achieve my goal, I have made (what I consider) minor modifications to the core codebase to provide locatable service lookups where need be, Improvement: The services that I have currently customised are:
This approach seemed less intrusive than a DI framework, and perhaps a simpler fit into QuickFIX core. It provides the benefits of maintaining a library that can be used out of the box, whilst providing the capability of customisation where required or necessary. I'd be mostly interested to know whether this (or something like it) would be considered as part of the roadmap with QuickFIX/J and how or whether I would be able to contribute. |
Comments |
Comment by Jörg Thönnes [ 15/Mar/13 ] |
Dear Ryan, we appreciate your idea and would like to consider it.
But currently there is no roadmap what should be done. But it would be good if you could start a page in QF/J Confluence to outline your thoughts Thanks, Jörg |
Comment by Ryan Lea [ 21/Mar/13 ] |
Hi Jorg, Thanks for your feedback. I will certainly attach a patch once I've extracted out the pieces I need to. Following that, I will look at creating a Confluence page as well (quite a few of the Confluence pages seem quite old as well I've noticed). Do you have a preference of which hierarchy you would like it placed in? Ryan |
[QFJ-729] Allow MessageCrackers to handle polymorphic Message types Created: 07/Feb/13 Updated: 07/Feb/13 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | S Raf | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
The MessageCracker expects an invoker to be registered for the specific actual Message type being processed. If an invoker exists for a supertype of the received Message, the invoker won't be called. For those extending MessageCracker, the onMessage(quickfix.Message message, SessionID sessionID) method can be overridden to allow implementers to handle the most general quickfix.Message type (but not, for example, only quickfix.fix44.Message messages). However, for those providing a delegate object to the MessageCracker, there is no option to provide a handler for a more general Message type to provide custom fallback logic. The crack(..) method should look for invokers registered for the received Message's superclass(es), as well as its actual type, before resorting to the fallback onMessage call in MessageCracker. |
[QFJ-762] Message stores can become corrupted on acceptor/initiator shutdown Created: 09/Dec/13 Updated: 21/Apr/17 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Alexey Ermakov | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None |
Issue Links: |
|
Description |
Since MessageStore implementations aren't thread safe, all operations on them should be performed from the session thread only. However, all 4 acceptor and initiator implementations perform shutdown (and thus call Session.unregisterSessions) in the calling thread. Since that calls MessageStore.close(), it can easily lead to store corruption if, for example, FileStore.close() gets called while the session thread is in the middle of persisting a Logout response. |
Comments |
Comment by Constantin Florescu [ 20/Apr/17 ] |
The same issue could be reproduced when the logon request is timing out. There are 2 threads trying to do the same thing: And from: This causes unexpected behaviour and leaks File Descriptors. |
[QFJ-780] QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect Created: 01/May/14 Updated: 23/Aug/16 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Tarun Bahadur | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect After a resend request is received from server subsequent messages should be ignored unless SequenceReset is received or messages replayed. In below test case the logout message is sent with sequence number 6, this should actually be ignored by the server and the test case should fail. This would require fixing both QuickFixJ behavior and the test case(probably logout should have MsgSeqNum=1). from fix42/1a_ValidLogonMsgSeqNumTooHigh.def |
Comments |
Comment by Christoph John [ 05/May/14 ] |
This is related to the QF/J-specific behaviour to always accept a Logout message regardless of the sequence number. But it probably would not hurt to change this in order to be more in line with the FIX spec. |
[QFJ-659] Sequence number issues when acceptor sends a message immediately following logon response Created: 09/Jan/12 Updated: 09/Jan/12 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0, 1.5.2 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Andrew Richards | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 2 |
Labels: | logon, sequnece |
Description |
I have an initiator connection to a foreign acceptor which has sequence number confusion on logon as the foreign acceptor sends a TestRequest immediately after the logon response. It looks like the following is happening: Quickfix/J accepts both (Logon & TestRequest) messages. The and proceeds to check the sequence number of the Logon to make sure the session is valid. It does this via Session.isTargetTooHigh() which looks at the last message in the message store which is now 1 higher than is should be because the TestRequest has been received and stored. The "MsgSeqNum too high" is issued and the engine tries to recover. In some most circumstances, but not all it (with the engine in question) it can recover however it's not ideal. The effect can be simulated (QuickFIX/J -> QuickFIX/J) by using and acceptor with the toAdmin below. public void toAdmin(Message message, SessionID sessionID) try { } 09-Jan-2012 08:49:19 INFO : Initialize |
[QFJ-657] TestRequest should be sent before a Logout is sent. (Initiator) Created: 05/Dec/11 Updated: 06/Dec/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0, 1.5.1 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Ken Douglas | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | Logout, testRequest | ||
Environment: |
Windows 7 Ultimate, Java 6 u29 |
Description |
TestRequest should be sent before a Logout is sent (Initiator). [1] a TestRequest is sent and a hearbeat is expected to be sent in return I have seen a question relating to this on the forum but this goes way back Currently the way the reset() on the Session is done it does not allow you to fire a TestRequest off and wait for the hearbeat. Any idea guys? Thanks |
Comments |
Comment by Christoph John [ 06/Dec/11 ] |
I think this would be a good improvement. BTW, the link to the forum thread is wrong. It leads to a discussion about log rotation. |
[QFJ-610] java.io.IOException when using FileStore Created: 17/Jun/11 Updated: 17/Jun/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Daniel Clusin | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | QuickfixJ | ||
Environment: |
uname -a: Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:56:28 EST 20 06 x86_64 x86_64 x86_64 GNU/Linux |
Description |
java.io.IOException: Bad file descriptor This occurred during a period of intermittent network disconnects. I did not see any forcible closes of the network sockets being done. Another time this occured is during a shutdown, after we called Initiator/Acceptor.stop FIX.4.2:XXXX->XXXX: Error during message processing quickfix.RuntimeError: java.io.IOException: Bad file descriptor Caused by: java.io.IOException: Bad file descriptor |
[QFJ-619] Late Resend Request causes sequence number mayhem : what happened? Created: 18/Jul/11 Updated: 18/Jul/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0 |
Fix Version/s: | None |
Type: | Other | Priority: | Default |
Reporter: | Nicolas Chaufette | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Attachments: | trace.txt |
Description |
Hello everyone, A client seems to have a problem that I cannot reproduce. Basically, the third party asks for a resend request of a message sent 7 minutes earlier. During those 7 mintures, heartbeats have been exchanged and the sequence increased. When the Resend Request is received, this happens (look at the sequence numbers). Thanks, best regards, Nicolas <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:30:40.704" level="Enabled" thread="QFJ Timer"> [... nothing of importance, just heartbeats in and out for 7 minutes...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.695" level="Enabled" thread="QFJ Timer"> [... here seqnum is 267, right? ...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.788" level="Enabled" thread="SocketConnectorIoProcessor-0.0"> [... counterparty asks for 260, sent 7 minutes earlier ...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.185" level="Enabled" thread="QFJ Message Processor"> [... sending 260, reset to 261 ?? ...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor"> [... sending 260 AGAIN, reset to 267 ????? ...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:46.819" level="Enabled" thread="SocketConnectorIoProcessor-0.0"> [... and it goes on and on and on and on...] <fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.831" level="Enabled" thread="QFJ Message Processor"> |
[QFJ-623] How do you change CompIDs, IP and port of a session at runtime? Created: 26/Jul/11 Updated: 19/Oct/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Other | Priority: | Default |
Reporter: | Andrew Niland | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ | ||
Environment: |
java 1.6 on Windows |
Description |
How do you change CompIDs, IP and port of a session at runtime? I would like to repoint the initiator engine to a different FIX server while the java app is still running. |
Comments |
Comment by Grant Birchmeier [ 26/Jul/11 ] |
Did you try posting your question to the mailing list? This doesn't look like it should be a bug report. |
Comment by Laurent Danesi [ 19/Oct/11 ] |
CompIDs are part of the SessionID, if you change them it is not the same Session anymore. Regards, Laurent DANESI |
[QFJ-558] Can no longer send messages to initiators that have not yet been started Created: 13/Sep/10 Updated: 16/Sep/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Rhys Yarranton | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Attachments: | diff.txt diff.txt |
Description |
We have some system states where we disable network connection but still may have messages that we wish to deliver to the gateway, i.e., message store, for future delivery to the counterparty when we re-enable the network connection. Our technique is to create a SocketAcceptor or SocketInitiator, but not to start it. Since we upgraded from QF/J 1.2.1 to QF/J 1.4.0, this is no longer working for initiators. Traced this back to I don't see any reason why the session creation cannot be moved back into the constructor. It does not interfere with the intent of While in the neighbourhood, noticed the following two lines in the AbstractSocketInitiator constructor: ByteBuffer.setAllocator(new SimpleByteBufferAllocator()); ByteBuffer.setUseDirectBuffers(false); This should be some sort of static initialization. e.g., for applications that use Mina for their own purposes as well as QF/J, they may need control over the buffer type and allocator, and shouldn't have to worry about QF/J changing it on the fly. |
Comments |
Comment by Rhys Yarranton [ 16/Sep/10 ] |
Updated diff includes mod to ThreadedSocketInitiator.java. |
[QFJ-555] Problems with heartbeating in one session can obstruct heartbeating on other sessions Created: 01/Sep/10 Updated: 07/Jan/20 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Rhys Yarranton | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Issue Links: |
|
Description |
SessionConnector.scheduledExecutorService is a single-threaded executor, and is used by all sessions for heartbeating. If there is a problem in one session, it can impact other sessions. This is really |
Comments |
Comment by Christoph John [ 07/Jan/20 ] |
This will be improved with https://github.com/quickfix-j/quickfixj/issues/254. |
[QFJ-531] To set the number of logon attempts Created: 23/Jun/10 Updated: 23/Jun/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | Future Releases |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Pranab Kumar Naik | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None | ||
Environment: |
Linux |
Description |
I was trying to set up the number of failed logon attempts after which the logon session would be created with a different host. The configuration does match with the interval between reconnects, but I thought it would be useful if the number of reconnect attempts may also be configurable from the config file. Or is there an option that I maybe missing ? |
[QFJ-583] Session.forceStoreResync doesn't check persistMessages Created: 31/Mar/11 Updated: 31/Mar/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Dmitri Lenna | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Windows & Linux. |
Description |
We set the "ForceResync" to true and "PersistMessages" to false. Attempting connect to a target with a different sequence number, we get a logout message with the expected sequence number and the following is called: (in Session.java): in that method, a heartbeat message is created for each missing sequence number and it's stored in the Message Store without checking if persisteMessage is true. Is there a reason for this or this a bug? If it's a bug, I'd suggest this fix: Line 1601:
Thank you. |
[QFJ-529] RTT benchmarking Created: 05/Jun/10 Updated: 05/Jun/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.4.0 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Wayne Zhu | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
windows 7 professional 64 bits, multiple core Intel |
Description |
I have tested with a simple server and client setup using a single session in the same machine(tcp socket on local host). The client sent in new orders and got execution reports back. If the orders were sent at a high data rate for a period of 100-200ms, the round trip time for a new order and the corresponding report was not good. My understanding is that there are two different threads in sending and receiving messages. What could be the reason causing QuickFix engine to slow down processing the incoming messages? Since there is no network overhead, it's most likely to be caused by the decoder and the way FIX message is constructed. |
[QFJ-521] Support use of partial session ID for routing purposes Created: 10/May/10 Updated: 09/Jun/11 |
|
Status: | Reopened |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.1, 1.3.2, 1.3.3, 1.4.0 |
Fix Version/s: | None |
Type: | New Feature | Priority: | Default |
Reporter: | Rhys Yarranton | Assignee: | Grant Birchmeier |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None |
Attachments: | diff3.txt |
Description |
In 1.2.1 and below, MessageUtils.getReverseSessionID included only the comp IDs, not the sub-IDs and SenderLocationID. This breaks us in two ways. One is that we have been configuring our sessions by the comp IDs, which works well with the 1.2.1 code but mismatches with the 1.3.1+ code even if both sides get all their IDs right. The other, and more important, is that several of our customer gateways will vary the SenderSubID and/or TargetSubID. For example, the FIX spec describes SenderSubID as being useful to distinguish between traders, and that messages not intended for a particular trader may optionally be flagged as ADMIN. It seems to me that the changes in Fortunately there is a mechanism to provide your own AcceptorSessionProvider, which can be used to work around this issue. That said, the default behaviour is not ideal. Attached is a patch to replace StaticAcceptorSessionProvider with a new class, DefaultAcceptorProvider, which first tries a "strict" 1.3.1-style lookup, and if it fails, tries a 1.2.1-style lookup based on the Comp IDs only. If the maintainers have an objection to the patch, I would suggest at the least that something like DefaultAcceptorProvider (plus instructions) be included in the distribution for those who cannot use StaticAcceptorSessionProvider. |
Comments |
Comment by Steve Bate [ 10/May/10 ] |
The specifications include subID and locationID in information such as DeliverTo and OnBehalfOf, which are used for session routing. A route endpoint in this case is identified by CompID/SubID/LocationID in the originator sessionID and in the third-party routing cases. As you mentioned, I have seen the specification describe senderSubID as being useful to distinquish between traders, but this appears to me to still be in the context of message routing to a specific trader (or trading desk) within a broader organization (represented by the compID). Every counterparty I've worked with that used subIDs have used them for session identification and routing. That said, from what I've seen on the FPL forums there was confusion in FIX 4.2 and before about the use of session ID fields for non-routing purposes. In FIX 4.3 and later, the Parties component is recommended for identifying traders outside the context of message routing. I'm not the only one making the decision, but I'm open to to a feature that optionally ignores parts of the session ID for routing purposes. |
Comment by Grant Birchmeier [ 19/Jul/10 ] |
I am having similar issues as Rhys. While reading the 4.4 spec to puzzle out my issues, I could not find where it says that SubID and LocationID should be used for anything substantial, let alone for message routing or session identification. Like Rhys, I'm not sure For now I'm integrating Rhys' patch into our local QF/J build, and being able to ignore the |
Comment by Grant Birchmeier [ 19/Jul/10 ] |
To integrate this patch into r958, DefaultAcceptorSessionProvider::getSession() needs a slight update to match an updated AcceptorSessionProvider interface:
|
Comment by Grant Birchmeier [ 12/Apr/11 ] |
Committed to 1015 |
Comment by Jörg Thönnes [ 09/Jun/11 ] |
Re-opening the issue to document the changes introduced IMHO, the handling of sub IDs and location IDs in QF/J deserves an extra section in the manual. There was some |
[QFJ-564] Operating on (modifying) repeating groups Created: 10/Nov/10 Updated: 10/Nov/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Default |
Reporter: | Addy Bhardwaj | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
N/A |
Description |
Currently if you want to add a value in a repeating group, there isn't a straightforward way of doing so. For instance, //Just for illustration, if noSides group is read from TCR //add account information in TCR This action doesn't modify the original TCR message as the group returned by getGroup() method is a copy of the actual group object. Is there a reason for this behaviour? Also, I noticed that when we add a typed group object it is copied into a Group object and then stored in the underlying TreeMap of groups. The only reason I could find is that String to Object parsing doesn't create concrete group objects hence copy is required but I hoping there is another reason that I have overlooked. ps: I have modified source code locally to improve this behaviour by
|
[QFJ-400] Make scheduledExecutorService non-static Created: 03/Feb/09 Updated: 16/Feb/09 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.3 |
Fix Version/s: | None |
Type: | New Feature | Priority: | Default |
Reporter: | Mihai Sardarescu | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
Hello all, In order to start our FIX connector, we have a "start" call from our middleware (there could be more than one FIX connectors started at different times on the same JVM). On this call, a logger is paased to the connector and all the log of the FIX connector should be dumped to this logger. The log of the QuickfixJ session is easily routed. However, the log messages from Mina and of the QuickfixJ initiators are more difficult to route. We use InheritableThreadLocal to pass the logger to all threads started by our FIX connector. We expect our application to go in production very soon (at most 2 month for now). Thank you, |
Comments |
Comment by Steve Bate [ 05/Feb/09 ] |
Please remind me why you weren't able to write a SLF4J logger implementation that delegates to the thread local logger provided by the middleware product. All logging from all threads goes through SLF4J so it seems that this would route those messages through the logger you provided. |
Comment by Mihai Sardarescu [ 05/Feb/09 ] |
Hi Steve, In order to start the FIX connector, I get a start call from the middleware. On this call, a logger is passed to my code and I shall redirect all log message written by quickfixj and mina to this logger. To do that I have used an InheritableThreadLocal that holds the middleware logger object in order to pass it to all the created by quickfixj and mina code. The loggers created by the slf4j are just proxies to the current thread middleware logger. I can thus redirect the quickfixj and mina log messages to the middleware log. This technique does not work in case a static executor (or a static thread) is used. For example: Also, I do not see the benefit of using a static scheduledExecutorService for all the initiators (which also uses a daemon worker thread). Wouldn't it be better to use a non-static executor that can be shutdown once the initiator is stopped? Regards, |
Comment by Steve Bate [ 06/Feb/09 ] |
Hello Mihai, As we discussed before, the purpose of the static (singleton) executor is to minimize the number of threads used by QFJ for it's own purposes. The executor is shared among all initiators and acceptors. I'm still not understanding why using a custom SLF4J implementation didn't work for you. If the custom logger implementation delegates to the /current/ middleware logger then it doesn't matter when the logger client actually retrieved the SLF4J logger instance. Actually, you could do this even without SLF4J by just implementing the QFJ Log interface. Your custom Log would delegate to the middleware logger. The middleware callback would modify the logger delegate in your Log implementation and everything that uses the Log implementation would automatically be using the new middleware logger. If the middleware logger is thread-local then that could also be supported with this approach. I'm probably missing something, but this is my understanding at this point based on the information I have. Feel free to clarify any key issues I'm not seeing. I want to understand why this can't be done in the way I describe before changing the QFJ internals to support it. Thanks, Steve |
Comment by Mihai Sardarescu [ 16/Feb/09 ] |
Hi Steve, The middleware provides a different logger object for each FIX connector. As I have one and only one QFJ session per connector, I have implemented a special QFJ logger that routes the log of one QFJ session to the middleware logger - this works fine and the log messages are routed to the correct logger. To make it more clear: let's suppose I have two concurrent FIX connectors running in the same JVM having:
Thanks, |
[QFJ-405] Fields missing on resends Created: 26/Feb/09 Updated: 26/Feb/09 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.1 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Rhys Yarranton | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
In some circumstances fields may be omitted from re-sent messages. We tracked this down to the following line in Session.parseMessage(String): So, a workaround is to edit the data dictionary. But given the silent misbehaviour, I would rate this a bug. Probably could fix this by creating a subclass of Message that treats the body as an immutable string --> overrides to parseBody, calculateString, calculateLength and calculateTotal. Hopefully there is a more elegant solution. |
[QFJ-373] Add a new validation configruation to bypass the data type check Created: 20/Nov/08 Updated: 26/Nov/08 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | New Feature | Priority: | Default |
Reporter: | Alvin Wang | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Comments |
Comment by Steve Bate [ 26/Nov/08 ] |
Can you be more specific about what you want in this feature? Thanks. |
Comment by Alvin Wang [ 26/Nov/08 ] |
For example, if there is a configuration called ValidateDateType=N in the configuration file, my QFJ server will not reject a message in which there is "867=100,0" rather than "867=100.0" because I do not care about tag 867. thanks. |
[QFJ-369] Send too many logout messages when continuely received messages which has BadTime or doBadCompID Created: 06/Nov/08 Updated: 06/Nov/08 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Mike Gu | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Windows XP |
Description |
Send too many logout messages when continuely received messages which has BadTime or doBadCompID. private void doBadTime(Message msg) throws IOException, FieldNotFound { generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0); generateLogout(); }Before Send a logout message, should check the session statues. isLogoutSent() private void doBadCompID(Message msg) throws IOException, FieldNotFound { generateReject(msg, SessionRejectReason.COMPID_PROBLEM, 0); if (!state.isLogoutSent()) generateLogout("Bad CompID"); }private void doBadTime(Message msg) throws IOException, FieldNotFound { generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0); if (!state.isLogoutSent()) generateLogout(); } |
[QFJ-394] SocketInitiator.stop(false) always waits sessions logoutTimeout before exit Created: 15/Jan/09 Updated: 03/Feb/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.3.3 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Alexey Utkin | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None |
Description |
SocketInitiator.stop(false) doesn't check sessions state. It always waits session logoutTimeout It seems a bug in SessionConnection.waitForLogout() |
Comments |
Comment by Peter Franzen [ 03/Feb/10 ] |
As far as I can tell, the problem lies in SessionConnection.waitForLogout(). If a session hasn't received its logout response when this method is entered I noticed this problem when moving from 1.2.1 to 1.3.x, it seems to have been The solution would be to check if the session has been logged out each time This could be achieved by changing the inner while loop in waitForLogout() to while (sessionItr.hasNext()) { catch (IOException e) { log.error(e.getMessage(), e); } } |
[QFJ-968] SessionState messageQueue causing out of memory exception Created: 06/Jan/19 Updated: 11/Jan/19 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.0.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | Ryan | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None | ||
Environment: |
Java 8 |
Description |
Comments |
Comment by Christoph John [ 08/Jan/19 ] |
I am not sure I understand fully: when you want to limit the queue size and discard messages if the queue is full, then why do you do a resend request for that many messages anyway? You could then just not do any resends (e.g. reset to seqnum 1)? Apart from that: if you are transmitting millions of messages in a few hours then I am not sure if FIX is the right choice of protocol. I assume that some kind of volatile data is transmitted (e.g. market data) which in turn leads to the question why you need resends at all if the data is outdated in the meantime anyway. Or did I misunderstand something? Thanks, |
Comment by Ryan [ 09/Jan/19 ] |
Hi Chris - Thank you for the reply! Let me try to give more details. 1. The reason we want to persist these messages is for future analysis. Normally the system works perfectly fine, but we want to handle the rarely happened situation where there is an issue and the system is down in the middle of the day. When that happens, we do need to send resend requests to catch up the outdated data. 2. My understanding is that there are two queues involved:
Every message has to go through the eventQueue, but only messages with higher seqNum will be put into messageQueue. Let's assume the batch size is 20 and consider the following event sequence:
In the given sequence, messages 501-540 will be handled correctly, but 10001-10005 will be added to messageQueue and will not be touched until all messages 501-10000 have been requested and processed. My proposal is to limit the size of the messageQueue (the map) to prevent the unbounded growing. For example if we set the limit to 4 messages, then the message 10005 and later messages will not be added to the messageQueue. When the initiator finishes handling messages 501-10004, it will expect 10005, but since it's discarded, initiator will try to request messages from 10005 to 10024 and move on. This scenario will happen only when both of the following conditions are satisfied:
Hope this better explained the scenario. Thanks, |
Comment by Ryan [ 09/Jan/19 ] |
BTW this is not a blocking issue for us, just want to report what we've observed during the development. |
Comment by Christoph John [ 11/Jan/19 ] |
Hi Ryan, after thinking a little more about it I think it should work the way you said, i.e. if we limited the size of the queued messages QFJ will request them anyway as soon as it has dequeued all messages. I have the following questions: Thanks, |
Comment by Christoph John [ 11/Jan/19 ] |
Of course it will lead to massive number of messages since messages will be re-requested multiple times if the counterparty already sent them and we re-request them because we did not put them to the queue. |
Comment by Ryan [ 11/Jan/19 ] |
Hi Chris, Let me pull the example sequence from above and try to explain my understanding:
Thanks, |
[QFJ-974] Session+FileStore reset fails indefinitely when stream closed Created: 23/Apr/19 Updated: 06/Jun/21 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.1.1 |
Fix Version/s: | None |
Type: | Bug | Priority: | Default |
Reporter: | amichair | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 1 |
Labels: | None |
Description |
If the FileStore stream is closed for any unexpected reason, the reset will fail indefinitely even if the original cause is fixed (or was just a transient error), since it will continue to try and flush/closed the failed stream, instead of truly resetting and opening a new one. In our case the error was triggered by the disk being full (space was then freed up), but any reason for closing the stream could get the store stuck in this unrecoverable state. Perhaps the store reset mechanism should be made more robust so that it really tries to reset even in this unexpected state rather than require the application to be restarted manually. quickfix.RuntimeError: java.io.IOException: Stream Closed |
Comments |
Comment by Sergei [ 06/Jun/21 ] |
13:35:17.885 QFJ Timer ERROR quickfix.SocketInitiator.run:356 - Error during timer processing Same issue What file might be the problem? - java.io.FileOutputStream.writeBytes |
[QFJ-969] Thread Safety FileStore.set Created: 28/Jan/19 Updated: 28/Jan/19 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 2.1.0 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Minor |
Reporter: | Yanick Salzmann | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | Message, QuickfixJ |
Description |
When seeking for optimizations for file sync I have come across the FileStore.set method and was wondering about its thread safety. I labelled this issue as improvement, because I am not sure if it is maybe already been taken care of somewhere else, however lets look at this method: As you can see, at the very beginning of the function the offset inside the RandomAccessFile is taken as the message offset. The actual writing of the information happens after many more instructions, one of them even including a synchronization on a file descriptor. According to my understanding the offset obtained on line 365 could easily be outdated by the time the message is written at line 377. Of course it is possible, that synchronization is supposed to be happening outside of the set method, however I do not see any contract enforcing this in MessageStore.set. Should this method not be thread safe to avoid potential data inconsitencies? |
Comments |
Comment by Christoph John [ 28/Jan/19 ] |
Hi Yanick Salzmann, as far as I am aware the MessageStore.set() method is only accessed from Session.sendRaw() where it is guarded by a lock. But yeah, in theory it could happen that it is accessed concurrently. It seems like this could happen especially on shutdown. See QFJ-762. Cheers, |
[QFJ-646] MiscFeeType field is handled incorrectly Created: 27/Oct/11 Updated: 17/Nov/11 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine, Metadata/Specs |
Affects Version/s: | 1.5.0, 1.5.1 |
Fix Version/s: | None |
Type: | Bug | Priority: | Minor |
Reporter: | John | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | QuickfixJ |
Description |
The "MiscFeeType" extends StringField and the constants into it are "String"s. The specification however (and the dictionary) indicates that this field is a "char". Workarround, either:
|
Comments |
Comment by Laurent Danesi [ 15/Nov/11 ] |
Hi John, Which FIX version are you using? In fact, in the spec, this field was ever a char and recently changed to String with FIX5.0. Now, of course, if you need to downgrade FIX5 feature code to a FIX4X session, you need a custom dictionary but it is not a bug, isnt it? Regards, Laurent |
Comment by John [ 15/Nov/11 ] |
Hi, |
Comment by Laurent Danesi [ 15/Nov/11 ] |
Hi, In fact, we generate fields for each dictionary but only once for a field (no override) and we generate FIX50 before FIX4X so MiscFeeType is a StringField. But I understand that if a client sends a String and MiscFeeType is a CharField, QFJ will reject but the current code should work as the parser will set the char as a String.anyway and we can match constants with equals method or I'm missing something perhaps? Can you provide me a simple test to well understand and fix it please? Laurent |
Comment by John [ 17/Nov/11 ] |
|
[QFJ-543] "connector" parameter in interface AcceptorSessionProvider.getSession() is undocumented and unclear Created: 19/Jul/10 Updated: 19/Jul/10 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Bug | Priority: | Minor |
Reporter: | Grant Birchmeier | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
See paste below from AcceptorSessionProvider. The parameter connector is absent from the javadoc, and it's not clear what it's for. As it's an interface, the parameter's purpose should be documented. From the two instantiations of this interface (DynamicAcceptorSessionProvider and AbstractSocketAcceptor::StaticAcceptorSessionProvider), it appears this parameter is only used for dynamic sessions and is otherwise ignored. /**
|
Comments |
Comment by Grant Birchmeier [ 19/Jul/10 ] |
Paste above is from r959 (the latest at time of bug creation). |
[QFJ-663] FieldMap's "getGroup()", "replaceGroup()", and possibly others... should start with 0 (not 1) Created: 23/Jan/12 Updated: 23/Jan/12 |
|
Status: | Open |
Project: | QuickFIX/J |
Component/s: | Engine |
Affects Version/s: | 1.5.0 |
Fix Version/s: | None |
Type: | Improvement | Priority: | Trivial |
Reporter: | John | Assignee: | Unassigned |
Resolution: | Unresolved | Votes: | 0 |
Labels: | None |
Description |
The first index for anything in the IT world is "0", not "1". Most certainly any programmer in this world expects the index "0" to be the first one. My solution would be to deprecate the methods and have new replacing methods (remove the deprecated methods and rename the new ones in a later version). |