[QFJ-626] ResendRequest (silently) aborted when one of the message to resend was (sent and) stored with a bad checksum Created: 11/Aug/11  Updated: 10/Sep/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Olivier Lourdais Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, encoding, session
Environment: Ubuntu SMP (2.6.35-25-generic)

java version "1.6.0_24"
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) Client VM (build 19.1-b02, mixed mode, sharing)

Attachments: Text File Session.patch    

 Description   
When a bug of the application around QuickFIX/J causes the generation of a message containing non-ISO-8859-1 characters (e.g. a price set to NaN or Infinity will be formatted to a special character by java.text.DecimalFormat), the computed checksum is wrong (checksum is computed on Unicode characters, before they are converted to bytes, and non-ISO-8859-1 characters will be replaced by '?' during the chars to bytes conversion, making the checksum inconsistent).
The peer FIX application will detect the problem and skip the affected message, then it will send a ResendRequest.

To handle the ResendRequest, QuickFIX/J will retrieve asked messages in bytes version, will convert them to chars (still containing '?' instead of non-ISO-8859-1 characters), then parse them (and hopefully resend them).
When parsing the message, QuickFIX/J detects the invalid checksum and throws an InvalidMessage exception.
The problem is that this exception is not caught when iterating over the messages to parse, but in the next(Message message) method, as if the error was in the ResendRequest message.

The consequence is that following messages are never sent (and that no SequenceReset message is sent for the peer FIX application to skip the erroneous message).

 Comments   
Comment by Olivier Lourdais [ 11/Aug/11 ]

Attached patch fixes the bug, making InvalidMessage exception simply caught around message parsing.

Comment by Jason Quek [ 04/Sep/14 ]

Hi,

I would like to raise that we should throw an InvalidArgumentException if a NaN, Positive Infinity, Negative Infinity value is set on a double field rather than letting it fail when being sent over the session and ignoring the message. This could be done until multi-byte transmission is supported.

Regards,
Jason

Comment by Staffan Ulfberg [ 10/Sep/14 ]

I agree that raising an exception in the setters is the correct thing to do since there is no way to transmit a NaN or Infinity over FIX.

Just to point out, however, that adding multi-byte support would not help. The fact that Java formats NaN and infinity values using unicode characters is irrelevant for how to represent them on the wire when using FIX. FIX float fields consists of ASCII characters "-", "0" - "9" and ".", and I do not think it is desirable to make an extension to a basic FIX type in quickfixj that will not be supported by other FIX engines.

Basically I'm arguing that throwing an exception is not just a temporary fix until multi-byte support is implemented. I think it is the correct thing to do. (Another possibility would be to create a "FixFloat" class to model this FIX type that does not have an exact corresponding Java type, but that is indeed a larger project and might be unpractical for other reasons.)

Staffan





[QFJ-788] StackOverflowError still an issue when processing larger queues Created: 06/Jun/14  Updated: 28/Jul/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.1

Type: Bug Priority: Major
Reporter: Andrzej Hajderek Assignee: Christoph John
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File SessionTest.java.patch    

 Description   
Hi,

QuickFIX/J fails regularly with the StackOverfloError when processing large queues (around 1000 messages or more - depending on Java stack size).

A typical scenario:
- Client sends a ResendRequest to server
- Server is slow in delivering old messages, but keeps sending real-time messages quickly
- The queue builds up quickly
- After the last old message is resent by server the client starts processing the queue using recursion and it fails with the StackOverflowError.

This is reproducible with trunk revision #1187. Test code attached.

Regards,
Andrzej Hajderek

 Comments   
Comment by Christoph John [ 06/Jun/14 ]

Hi Andrzej, thanks for the good test, really appreciated. Saves a lot of time.

Comment by Andrzej Hajderek [ 06/Jun/14 ]

Please note that for smaller queues the test passes correctly. Only larger queue sizes cause problems.

The same issue can be reproduced in a real-world scenario, when a large backlog of messages (say 100k messages) is present on the server. The client is in the process of retrieving the entire backlog with a relatively small chunk size (say 500) and it is not very quick at processing the responses. It takes certain amount of time to process the backlog (say 15 minutes). During that time the server keeps sending new real-time trades to the client. These new real-time trades get queued. After the processing of the backlog is completed QuickFIX/J blows up with the StackOverflowError.

The test code I attached represents the minimal scenario for reproducing the issue effectively.





[QFJ-790] Logout message not propagated to the fromAdmin() callback Created: 09/Jun/14  Updated: 28/Jul/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.1

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Java Source File LostLogoutTest.java    

 Description   
Hi,

A QuickFIX/J based application may miss a Logout request message, if the sender closes the socket shortly after sending the Logout request message. As result the application may lose important information about the reason of disconnection.

Most likely the reason of such behaviour of QuickFIX/J is such that the QFJ Message Processor stops processing queued messages as soon as it detects hasResponder() == false, which may happen even, if the input queue contains messages (or merely the Logout message). Basically the more loaded the receiver's machine is (in terms of CPU/IO) the more likely it is that it misses the Logout message.

Please note that even though the fromAdmin() callback is not called in such cases, the onLogout() is always called (because of the the closed socket). However, multiple occurrences of the same sender-side series of events (Logout + socket close) will lead to two different receiver-side behaviours on a random basis. In some cases the receiver will see the proper fromAdmin() callback and the onLogout() callback, while in some cases it will see only the onLogout() callback (triggered by socket close rather than by the incoming Logout message).

This has been an issue since 1.0.x, but only now I found some time to write a proper test.

Test code attached.

Regards,
Andrzej Hajderek


 Comments   
Comment by Andrzej Hajderek [ 09/Jun/14 ]

If the delay code is removed from the Server.fromApp() method the Logout message will be received correctly in many cases, but it's not guaranteed. The test code is written in such a way that the server always misses the message, but the probability of missing the Logout message is really a function of the delay value within Server.fromApp(). In other words - the more time is spent it Server.fromApp() the more likely it is that the server misses the Logout message.





[QFJ-801] Validation fail on ClassCastException if using XML Created: 14/Jul/14  Updated: 14/Jul/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Benoit Xhenseval Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: validation
Environment: Mac OSX

Attachments: Java Source File IOIValidation.java    

 Description   
It appears that the Message validation fails on a ClassCastException if the message contains the XmlDataLen tag.

Example:
https://gist.github.com/benoitx/e7da15dba9da133ac7b9

It seems that the validation code only expects StringField but the XmlDataLen is an IntField.

The output is:
{code}
Testing WITHOUT XML
8=FIXT.1.19=7935=649=Sender56=Target15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=079
Testing WITH XML
8=FIXT.1.19=10335=649=Sender56=Target212=12213=<a>Hello</a>15=USD22=523=IOD-127=M28=N48=IBM.N54=255=IBM.N10=086
Exception in thread "main" java.lang.ClassCastException: quickfix.field.XmlDataLen cannot be cast to quickfix.StringField
at quickfix.DataDictionary.iterate(DataDictionary.java:668)
at quickfix.DataDictionary.validate(DataDictionary.java:653)
at quickfix.DataDictionary.validate(DataDictionary.java:624)
at quickfix.DataDictionary.validate(DataDictionary.java:606)
at fixfun.IOIValidation.validateIoi(IOIValidation.java:45)
at fixfun.IOIValidation.validateIoiWithXml(IOIValidation.java:37)
at fixfun.IOIValidation.main(IOIValidation.java:78)
{code}

 Comments   
Comment by Benoit Xhenseval [ 14/Jul/14 ]

test case.

Comment by Christoph John [ 14/Jul/14 ]

It turned out that the setField( int, Field<?> ) method should rather not be used with non-String tags. Rather use setField( Field ), e.g.:
ioi.getHeader().setField(new XmlData(xml));
ioi.getHeader().setField(new XmlDataLen(xml.length()));

Of course, there should be means to prevent your original error. Either setField( int, Field<?> ) should convert the values (as setField( Field ) does) or the validate-method should not assume that all fields are Strings.





[QFJ-792] Wrong order of fields in a repeating group makes the group ignored Created: 10/Jun/14  Updated: 18/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File MessageTest.java.patch    
Issue Links:
Relates
relates to QFJ-770 Message fromString don't report about... Open

 Description   
Hi,

Another issue related to repeating groups in the message parsing logic. If two fields within a repeating group are in different order than expected the entire repeating group is ignored. The parsed message does not contain the ignored repeating group.

The better approach would be to process the repeating group regardless of the order of fields or to report and error in order to signal the wrong order (assuming ValidateUnorderedGroupFields=Y).

Regards,
Andrzej Hajderek


 Comments   
Comment by Andrzej Hajderek [ 10/Jun/14 ]

Unit test attached.

Comment by Andrzej Hajderek [ 10/Jun/14 ]

Possibly related to QFJ-770.





[QFJ-770] Message fromString don't report about REPEATING_GROUP_FIELDS_OUT_OF_ORDER Created: 23/Jan/14  Updated: 18/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Andrey Alekov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Message
Environment: Windows 7.
Version: 6.1
Build: 7606 SP1

Eclipse.
Version: Juno Service Release 2
Build id: 20130225-0426

Issue Links:
Relates
is related to QFJ-792 Wrong order of fields in a repeating ... Open

 Description   
When I tried to parse message from String I faced with not expected behavior of method Message.fromString()

Source message in Repeating group have unordered tags and values. Example are below.
SOH replaced to pipe (|)
String test = new String("8=FIX.4.4|9=16858|35=d|49=1|34=2|52=20140117-18:20:26.629|56=3|57=21|322=388721|323=4|320=1|393=42|82=1|67=1|711=1|311=780508|309=text|305=8|463=FXXXXX|307=text|542=20140716|436=10.0|9013=1.0|9014=1.0|9017=10|9022=1|9024=1.0|9025=Y|916=20140701|917=20150731|9201=23974|9200=17|9202=text|9300=727|9301=text|9302=text|9303=text|998=text|9100=text|9101=text|9085=text|9083=0|9084=0|9061=579|9062=text|9063=text|9032=10.0|9002=F|9004=780415|9005=780503|10=128|");

When call fromString I see parsed message without any error.
8=FIX.4.4|9=100|35=d|34=2|49=1|52=20140117-18:20:26.629|56=3|57=21|67=1|82=1|320=1|322=388721|323=4|393=42|711=42|10=156|

I checked Message.Exception and found next "Out of order repeating group members, field=916"


Well, I found what in Message.parse method (line Message.java:485) written exception but not raised.

        } catch (final FieldException e) {
            exception = e;
        }

Please fix to raise InvalidMessage instead filling this.exception to case below.




[QFJ-798] Refactor Session construction in test code Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
Currently to create some of the Session related test cases it is necessary to use constructs like this:

Session session = new Session(new UnitTestApplication(), new MemoryStoreFactory(),
    sessionID, null, null, new ScreenLogFactory(true, true, true),
    new DefaultMessageFactory(), isInitiator ? 30 : 0, false, 30, true, resetOnLogon,
    false, false, false, false, false, true, false, 1.5, null, validateSequenceNumbers,
    new int[] { 5 }, false, disconnectOnError, false, true, false, true, false, null,
    true, 0, false, false);

This kind of code turns from an asset into a liability pretty quickly, so the sooner it is rationalised the better. The Session constructor itself isn't very pretty either with its 30+ parameters.




[QFJ-796] There is no mechanism to clear or update the data dictionary cache Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
A data dictionary file loaded into memory is cached without any option to reload the file after its contents has been updated. Worse - it's not even possible to clear the cache. This means that in order to use an updated data dictionary it is necessary to restart the application.

See: quickfix.DefaultSessionFactory.dictionaryCache





[QFJ-797] Add a clean and safe mechanism to disconnect a session from within a call-back Created: 13/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
Hi,

There is currently no clean and safe mechanism to completely stop and disconnect a session from within a call-back (e.g. when processing an incoming message inside fromApp()) with the effect that the session is immediately disconnected, no reject messages are sent back to the counter-party and no change to the session store is made.

Such a mechanism is useful for implementing failure handling scenarios, e.g. when a target database becomes suddenly unavailable, or the application needs to be shutdown in emergency. In such cases it's best to close the TCP socket immediately and keep the session state unchanged.

One could try to use something like SocketInitiatior.stop(true), but it's not generic (does not work for acceptors) and it results in a call to Session.disconnect(). Then, if Session.disconnect() is called from fromApp(), we get an immediate call to the onLogout() callback - a call is made from within the fromApp() call-back into the onLogout() call-back. This isn't a particularly safe scenario, especially when in most cases the onLogout() call is made by from QuickFIX/J threads.

Another solution is to throw a RuntimeException within the call-back method, but it's not reliable right now (QFJ-793, QFJ-795) and makes QuickFIX/J send an error message and the stack trace into the log.

I propose to add a special a subclass of RuntimeException, e.g. DisconnectSessionExcepion, which when thrown from a call-back method would notify QuickFIX/J that the session should be disconnected immediately. In such case only an informational message would be send to the log instead of error with a stack trace.

Regards,
Andrzej Hajderek





[QFJ-793] When RuntimeException is thrown within a call-back and DisconnectOnError=Y the session does not get disconnected automatically Created: 12/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File SessionTest.java.add     File SessionTest.java.add.fixed    

 Description   
Hi,

When a RuntimeException is thrown within a call-back and the "DisconnectOnError" parameter is set to "Y" the session does not get disconnected automatically unless the "RejectMessageOnUnhandledException" parameter is also set to "Y". This dependency is not documented, but even if it was, it wouldn’t make too much sense.

It seems to be logical to understand an occurrence of a RuntimeException within a call-back as an error condition on its own, which should be sufficient to trigger the disconnection logic dictated by the "DisconnectOnError" parameter (regardless of any other settings).

Currently, if RejectMessageOnUnhandledException=N, the engine simply ignores the message for which the exception was thrown even though DisconnectOnError=Y (although it reports the exception as an “error” in the log).

Test code attached.

Please note that probably all call-backs exhibit the same behaviour, not just the fromApp() call-back used in the TestApp class. It is also possible that the “ResetOnError” parameter is treated in a similar manner.

Regards,
Andrzej Hajderek


 Comments   
Comment by Andrzej Hajderek [ 13/Jun/14 ]

Attached a fixed version of the test.





[QFJ-794] Misleading message logged when RuntimeException thrown within a call-back: "Rejecting message: ..." Created: 12/Jun/14  Updated: 13/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
This is in relation to QFJ-793.

When both settings RejectMessageOnUnhandledException and DisconnectOnError are set to Y and a RuntimeException is thrown within a call-back, QuickFIX/J will write to the log output messages like below:

<20140612-16:25:06, FIX.4.4:SENDER->TARGET, error> (Rejecting message: java.lang.RuntimeException: ... (with more details and the message)

2014-06-12 18:25:06 quickfix.Session disconnect

INFO: [FIX.4.4:SENDER->TARGET] Disconnecting: Auto disconnect

This is misleading because nothing is actually rejected (no reject message is sent back to the sender). The session is simply disconnected. I believe a slightly different wording should be used for this scenario, e.g "Failed to process message" instead of "Rejecting message".




[QFJ-791] An unexpected field in a repeating group makes QuickFIX/J fail to detect the number of repeating groups correctly Created: 10/Jun/14  Updated: 12/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File MessageTest.java.patch    

 Description   
Hi,

When an unexpected tag (a tag not defined in the data dictionary) is present in a repeating group QuickFIX/J fails to detect the number of repeating groups correctly. For example in the following message the unexpected tag 58 is present at the end of the first leg (600, 687, 654, 566, [58]):

8=FIX.4.4 9=233 35=AE 34=1 49=SENDER 52=20140610-15:04:53.377 56=TARGET 31=5.6789 32=1000 60=20140610-15:04:53.367 75=20140101 570=N 571=ABC1234 555=2 600=L1-XYZ 687=333 654=ABC1234-L1 566=1.2345 58=TXT1 600=L2-XYZ 687=777 654=ABC1234-L2 566=2.3456 10=017

With tag 58 in the first leg, the number of repeating groups detected by QuickFIX/J message parser will be 1 (incorrect)!
Remove tag 58 from the first leg and QuickFIX/J will sees the number of repeating groups for as 2 (correct).

This is very dangerous because it makes a QuickFIX/J applications very sensitive to changes in repeating groups. For example, when a trading platform decides to add a new tag to the repeating group, an existing application will fail completely because of the incorrect number of repeating groups detected. Instead the application should simply ignore the new tag, especially when AllowUnknownMsgFields=Y.

Of course when the new tag is added to the data dictionary this problem will not occur, however, the current behaviour of the repeating group parsing logic is inconsistent with the message body parsing logic, inconsistent with the configuration (AllowUnknownMsgFields=Y) and generally counter-intuitive.

Regards,
Andrzej Hajderek


 Comments   
Comment by Andrzej Hajderek [ 10/Jun/14 ]

Unit test attached.





[QFJ-780] QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect Created: 01/May/14  Updated: 10/Jun/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Tarun Bahadur Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
QuickFixJ behavior for Acceptance test fix42/1a_ValidLogonMsgSeqNumTooHigh.def looks incorrect

After a resend request is received from server subsequent messages should be ignored unless SequenceReset is received or messages replayed. In below test case the logout message is sent with sequence number 6, this should actually be ignored by the server and the test case should fail. This would require fixing both QuickFixJ behavior and the test case(probably logout should have MsgSeqNum=1).

from fix42/1a_ValidLogonMsgSeqNumTooHigh.def
...
E8=FIX.4.29=5835=234=249=ISLD52=00000000-00:00:00.00056=TW7=116=410=0
I8=FIX.4.235=534=649=TW52=<TIME>56=ISLD
...



 Comments   
Comment by Christoph John [ 05/May/14 ]

This is related to the QF/J-specific behaviour to always accept a Logout message regardless of the sequence number. But it probably would not hurt to change this in order to be more in line with the FIX spec.





[QFJ-743] JdbcLog doesn't take into consideration the sessionID to verify if a parameter is set like JdbcMessage does Created: 13/May/13  Updated: 28/Jan/14

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Leandro Garcia Herrera Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: JdbcLog, QuickfixJ, jdbc
Environment: Windows 7, using JBoss and camel-quickfix


 Description   
I'm trying to configure JdbcLogFactory dynamically passing a SessionSettings object as argument to constructor. All my configurations are under an especific session ID and I have no configurations under default session.

Looking at the source code of JdbcLog I was able to confirm that only the default session configurations are taken into consideration instead of checking first in session ID configurations informed via constructor, wich I was expecting.

The JdbcStore class does this verification by session ID and I think JdbcLog should work in the same way.

 Comments   
Comment by Christoph John [ 09/Jan/14 ]

Would it be OK for you to attach the patch to this ticket? I do not have the necessary rights to grant you SVN access but could integrate the patch shortly.





[QFJ-762] Message stores can become corrupted on acceptor/initiator shutdown Created: 09/Dec/13  Updated: 09/Dec/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Alexey Ermakov Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Relates
relates to QFJ-552 Message Stores expected to be thread ... Open

 Description   
Since MessageStore implementations aren't thread safe, all operations on them should be performed from the session thread only. However, all 4 acceptor and initiator implementations perform shutdown (and thus call Session.unregisterSessions) in the calling thread. Since that calls MessageStore.close(), it can easily lead to store corruption if, for example, FileStore.close() gets called while the session thread is in the middle of persisting a Logout response.




[QFJ-552] Message Stores expected to be thread safe but are not Created: 05/Aug/10  Updated: 09/Dec/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.4.0, 1.5.0
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Carmelo Piccione Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Linux + JDBC + SQL Server 2005

Attachments: Java Source File SynchronizedMessageStore.java     Java Source File SynchronizedMessageStoreFactory.java    
Issue Links:
Relates
is related to QFJ-762 Message stores can become corrupted o... Open

 Description   
SessionState.java starts with the following bit of code:

/**
 * Used by the session communications code. Not intended to be used by
 * applications.
 *
 * All dynamic data is protected by the session's intrinsic lock. The
 * log and message store implementation must be thread safe.
 */
public final class SessionState {
    private final Object lock;
    private final Log log;

    // MessageStore implementation must be thread safe
    private final MessageStore messageStore;
...
}

Yet, looking at the JdbcStore.java and MemoryStore.java implementations, they are in fact not thread safe as designed (may also be true for FileStore.java, but haven't checked).

The memory store uses a regular java.util.HashMap for all its operations, which is clearly not thread safe. I have never actually seen this cause a problem in practice, however.

The jdbc store does not synchronize its use of the jdbc connection, which can lead to race conditions inside the close() method of the jtds prepared statement, as shown in the stack trace below (specifically, the jtds close() method has a boolean to determine if its connection is already closed in which case it will not doubly close it despite any subsequent calls. This value is set once the close routine is complete but occurs after the connection object has already been assigned a null value- this allows another thread to potentially access the null connection object before the boolean guard has been set). Note that this bug occurs about once a week in a production environment (usually during startup), although its frequency is probably higher than average due to many quickfixj applications accessing the same database concurrently.

Example 1:

java.lang.NullPointerException
at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:868)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.close(JtdsPreparedStatement.java:485)
at org.logicalcobwebs.proxool.AbstractProxyStatement.close(AbstractProxyStatement.java:115)
at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:93)
at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)
at quickfix.JdbcUtil.close(JdbcUtil.java:143)
at quickfix.JdbcStore.storeSequenceNumbers(JdbcStore.java:288)
at quickfix.JdbcStore.setNextTargetMsgSeqNum(JdbcStore.java:272)
at quickfix.JdbcStore.incrNextTargetMsgSeqNum(JdbcStore.java:173)
at quickfix.SessionState.incrNextTargetMsgSeqNum(SessionState.java:359)
at quickfix.Session.nextLogon(Session.java:1535)
at quickfix.Session.next(Session.java:720)
at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:106)
at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:86)
at java.lang.Thread.run(Thread.java:619)

Example 2:

java.lang.NullPointerException
at net.sourceforge.jtds.jdbc.JtdsStatement.close(JtdsStatement.java:868)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.close(JtdsPreparedStatement.java:485)
at org.logicalcobwebs.proxool.AbstractProxyStatement.close(AbstractProxyStatement.java:115)
at org.logicalcobwebs.proxool.ProxyStatement.invoke(ProxyStatement.java:93)
at org.logicalcobwebs.proxool.ProxyStatement.intercept(ProxyStatement.java:57)
at quickfix.JdbcUtil.close(JdbcUtil.java:143)
at quickfix.JdbcStore.set(JdbcStore.java:259)
at quickfix.SessionState.set(SessionState.java:299)
at quickfix.Session.sendRaw(Session.java:1736)
at quickfix.Session.generateLogon(Session.java:1430)
at quickfix.Session.next(Session.java:1357)
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:248)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)

 Comments   
Comment by Carmelo Piccione [ 17/Aug/10 ]

A simple but thread safe message store implementation. The constructor takes a MessageStore instance as input and delegates all method calls to it through an object lock. This is a work around which allows safe usage of the original QFJ message stores such as JdbcStore, FileStore, and MemoryStore.





[QFJ-752] Instrument component dont show the instrumentParties repeating group Created: 30/Jul/13  Updated: 31/Jul/13

Status: Open
Project: QuickFIX/J
Component/s: Engine, Message Generation
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Christian Avalos Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ


 Description   
the noInstrumentParties(1018) component is in the componentFields int array, and not in componentGroup, so, when it added a instrumentParties ,the generated message has the field 1018, but not the other possibles fields of instrumentParties (1019, 1050, 1051, 1052)

 Comments   
Comment by Christian Avalos [ 30/Jul/13 ]

this issue is fixed adding the noIntrumentParties fields into componentGroups attribute (and removing from componentFields int[] )

public class Instrument extends quickfix.MessageComponent {
static final long serialVersionUID = 20050617;
public static final String MSGTYPE = "";
private int[] componentFields =

{ 55, 65, 48, 22, 454, 460, 461, 167, 762, 200, 541, 224, 225, 239, 226, 227, 228, 255, 543, 470, 471, 472, 240, 202, 947, 206, 231, 223, 207, 106, 348, 349, 107, 350, 351, 691, 667, 875, 876, 864, 873, 874, 965, 966, 1049, 967, 968, 969, 970, 971, 996, 997, 1079, }

;
private int[] componentGroups =

{1018, }

;

public Instrument()

{ super(); }

......
}

Comment by Jörg Thönnes [ 31/Jul/13 ]

This message and field classes are generated automatically from the data dictionary so this is most probably a code generation issue.

Comment by Christoph John [ 31/Jul/13 ]

... or in the data dictionary.





[QFJ-749] Syntax error in SessionSettings.java/line 572 leading exception from java.util.regex Created: 17/Jun/13  Updated: 08/Jul/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Pavel Sagulenko Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment: Eclipse 22 on Debian GNU/Linux 7.0


 Description   
There is a syntax error in SessionSettings.java class, line 572.
original line: private final Pattern variablePattern = Pattern.compile("\\$\\{(.+?)}");
Should be: private final Pattern variablePattern = Pattern.compile("\\$\\{(.+?)\\}");

(with double slash before closing parenthesis).
This causes the failure in creating a SessionSettings instance.

I tried to recompile myself the corrected version, but I got the following exception:

java.lang.NoClassDefFoundError: quickfix.SessionSettings

Probably, you can suggest the possible solution.
Thank you!
 

 Comments   
Comment by Christoph John [ 08/Jul/13 ]

Sorry, what do you mean by: "This causes the failure in creating a SessionSettings instance"??
Are you able to supply a test case which shows the failure?





[QFJ-733] Provide customisable hooks into the core QuickFIX/J workflow Created: 11/Mar/13  Updated: 21/Mar/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Ryan Lea Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, modules, performance


 Description   
Background:
  I have been using QuickFIX for some time and found it a great library. As my usage has increased, I have become more aware that it's not necessarily GC-friendly. As such I'm in the process of extending the Message object to become more GC-friendly, but really a step away from the message type-safety that is currently present within QuickFIX/J.

  In order to achieve my goal, I have made (what I consider) minor modifications to the core codebase to provide locatable service lookups where need be,

Improvement:
  It would be great if QuickFIX/J was more customisable in the services that were able to be overriden/extended or generally customised. I've currently created a (very simple - based on http://martinfowler.com/articles/injection.html) ServiceLocator that prepares services with defaults (current 1.5.3 behaviour) that can be overriden with custom services on startup (if required).

The services that I have currently customised are:
# Message validation with the Data Dictionary
# Message creation from String (the FIXMessageDecoderFactory)
# Handling of FIX Messages within the abstract IoHandler (as it depends upon MINA passing it a String)


This approach seemed less intrusive than a DI framework, and perhaps a simpler fit into QuickFIX core. It provides the benefits of maintaining a library that can be used out of the box, whilst providing the capability of customisation where required or necessary.

I'd be mostly interested to know whether this (or something like it) would be considered as part of the roadmap with QuickFIX/J and how or whether I would be able to contribute.

 Comments   
Comment by Jörg Thönnes [ 15/Mar/13 ]

Dear Ryan,

we appreciate your idea and would like to consider it.
It is true the QF/J is quite monolithic and would greatly benefit from being split into more components.
E.g.

  • separate IO layer in order to allow to replace MINA by Netty, etc (QFJ-598)

But currently there is no roadmap what should be done.

But it would be good if you could start a page in QF/J Confluence to outline your thoughts
and perhaps attached some code.

Thanks, Jörg

Comment by Ryan Lea [ 21/Mar/13 ]

Hi Jorg,

Thanks for your feedback. I will certainly attach a patch once I've extracted out the pieces I need to.

Following that, I will look at creating a Confluence page as well (quite a few of the Confluence pages seem quite old as well I've noticed). Do you have a preference of which hierarchy you would like it placed in?

Ryan





[QFJ-734] Properties for configuring Proxool vs. not incremented outgoing sequence number Created: 13/Mar/13  Updated: 13/Mar/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Krzysztof Szalast Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: All


 Description   
I want to remind about comment in quickfix.JdbcUtil in method "static synchronized DataSource getDataSource(String jdbcDriver, String connectionURL, String user, String password, boolean cache)":
// TODO JDBC Make these configurable
setMaximumActiveTime(ds, 5000);
ds.setMaximumConnectionLifetime(28800000);
ds.setMaximumConnectionCount(10);
ds.setSimultaneousBuildThrottle(10);

If "messages" table are locked longer than 5 seconds, connection is aborted by Proxool:
[13.03.2013|12:42:07:875] [HouseKeeper] [DEBUG] [proxool.quickfixj-1] [org.logicalcobwebs.proxool.ConnectionPool] [removeProxyConnection] [441] : 000026 (01/02/00) - #0001 removed because it has been active for too long.
[13.03.2013|12:42:07:875] [HouseKeeper] [WARN ] [proxool.quickfixj-1] [org.logicalcobwebs.proxool.HouseKeeper] [sweep] [149] : #0001 was active for 98688 milliseconds and has been removed automaticaly. The Thread responsible was named 'QFJ Timer', but the last SQL it performed is unknown because the trace property is not enabled.
[13.03.2013|12:42:07:906] [QFJ Timer] [ERROR] [quickfixj.errorEvent] [quickfix.SLF4JLog] [logError] [147] : Error Reading/Writing in MessageStore
java.io.IOException: Couldn't perform the operation prepareStatement: You can't perform any operations on this connection. It has been automatically closed by Proxool for some reason (see logs).
at quickfix.JdbcStore.set(JdbcStore.java:252)
at quickfix.SessionState.set(SessionState.java:308)
at quickfix.Session.sendRaw(Session.java:2313)
at quickfix.Session.generateHeartbeat(Session.java:1861)
at quickfix.Session.next(Session.java:1834)
at quickfix.mina.SessionConnector$SessionTimerTask.run(SessionConnector.java:283)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.sql.SQLException: Couldn't perform the operation prepareStatement: You can't perform any operations on this connection. It has been automatically closed by Proxool for some reason (see logs).
at org.logicalcobwebs.proxool.WrappedConnection.invoke(WrappedConnection.java:207)
at org.logicalcobwebs.proxool.WrappedConnection.intercept(WrappedConnection.java:87)
at $java.sql.Wrapper$$EnhancerByProxool$$a5ac9e8b.prepareStatement(<generated>)
at quickfix.JdbcStore.set(JdbcStore.java:245)
... 14 more

and outgoing sequence number is not incremented! Next outgoing message has the same sequence number (sic!).

Scenario:
1. Send NOS message.
2. Lock "messages" table (on Postgres: LOCK TABLE messages IN ACCESS EXCLUSIVE MODE)
3. Send NOS message - message will be sent, but stacktrace from above will be happen. For example message has t34=666
4. Send NOS message - message will be sent with t34=666 (!)

 Comments   
Comment by Jörg Thönnes [ 13/Mar/13 ]

Good find! I think in that case the message should not be sent.





[QFJ-729] Allow MessageCrackers to handle polymorphic Message types Created: 07/Feb/13  Updated: 07/Feb/13

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: S Raf Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
The MessageCracker expects an invoker to be registered for the specific actual Message type being processed. If an invoker exists for a supertype of the received Message, the invoker won't be called.

For those extending MessageCracker, the onMessage(quickfix.Message message, SessionID sessionID) method can be overridden to allow implementers to handle the most general quickfix.Message type (but not, for example, only quickfix.fix44.Message messages).

However, for those providing a delegate object to the MessageCracker, there is no option to provide a handler for a more general Message type to provide custom fallback logic.

The crack(..) method should look for invokers registered for the received Message's superclass(es), as well as its actual type, before resorting to the fallback onMessage call in MessageCracker.




[QFJ-416] Support Header/Trailer field ordering in code generation Created: 16/Mar/09  Updated: 15/Oct/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: Future Releases

Type: Improvement Priority: Default
Reporter: Laurent Danesi Assignee: Laurent Danesi
Resolution: Unresolved Votes: 2
Labels: None

Issue Links:
Relates
relates to QFJ-512 How do I change the order of the head... Closed
Requires
requires QFJ-709 expose field order constructor for me... Resolved

 Description   
To fully support custom FIX dictionary, we should take care of header fields defined in XML dictionary.

 Comments   
Comment by Jörg Thönnes [ 16/Mar/09 ]

In reply to QFJ-416:
> To fully support custom FIX dictionary, we should take care of header fields
> defined in XML dictionary.

I would like to add that the ordering as defined in the data dictionary should be honoured.
Same applies to trailer fields.

Do you think this should be separate tickets?

Comment by Laurent Danesi [ 16/Mar/09 ]

Yes, please.

Comment by Steve Bate [ 16/Mar/09 ]

Ordering by default will lead to a potential performance hit for users that don't care about ordering.

Comment by Jörg Thönnes [ 16/Mar/09 ]

In reply to comment #3:
>
> Ordering by default will lead to a potential performance hit for users that
> don't care about ordering.

Yes, but at the moment it is not possible to generate ordered Header and Trailer fields since the FieldMap constructor is not exposed.

My suggestion is to expose the field order constructor and extend the message generation to honour the generator option to create ordered messages.

In addition, the field sort is generated for every single message instance, but could by generated by a factory to use a single instance for every message type.

Comment by Jörg Thönnes [ 16/Mar/09 ]

Sorry for the duplicate comments.

The JIRA version used by QuickFIX/J has a bug which triggers errors with my Eclipse Mylyn plugin. So I did not know that I already submitted the comment.

Comment by Steve Bate [ 05/Apr/10 ]

It appears the request is also to validate the ordering.

Comment by Phill Dixon [ 27/Sep/10 ]

To add a custom field to a Trailer I also found that I had to edit the quickfix.Message object as the Trailer class has a hardcoded field order and any tag added to it which did not appear resulted in an invalid tag error. This occured when sending the Logon message with the new custom field in the trailer. The DataDictionary xml file had been modified but was apparently ignored.

There is also a hardcoded case statement for isTrailerField which needs modifying.

It would be useful if the XML file was respected.





[QFJ-247] Allow for dynamic session addition in AbstractSocketInitiator Created: 22/Sep/07  Updated: 01/Oct/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: Future Releases

Type: Improvement Priority: Default
Reporter: Toli Kuznets Assignee: Unassigned
Resolution: Unresolved Votes: 4
Labels: None

Attachments: Text File AbstractSocketInitiator.java.patch    

 Description   
Right now you can have multiple sessions in same SocketInitiator only when you configure them at startup.
Would be nice to have a programmatic way to add new sessions to socket initiator dynamically, either via code an application level or via some JMX adapter.

It's possible to work around this by creating a new SocketIntiator, but that may create threading issues for people using single-threaded strategies and having single-threaded assumptions in their application code.

Not sure if the right way is to provide a AbstractSocketInitiator.addSession() (and removeSession) function, or if there's a better approach.

 Comments   
Comment by Danilo [ 06/May/08 ]

I need this too. Is there any other way to work around this while it's not implemented the way you propose?

Comment by Toli Kuznets [ 06/May/08 ]

Danilo, we've been working on the code to implement this but haven't had the time to polish it yet so we haven't checked it in. But i'm hoping to have that ready in a few weeks.

Comment by Steve Bate [ 10/Jan/10 ]

Hi Toli. Any updated status on this feature?

Comment by Toli Kuznets [ 15/Jan/10 ]

Steve,
Unfortunately, we never ended up implementing this, so it's still in RFE stage...

Comment by Cesar Kamoy [ 27/Jul/11 ]

I fully agree with Steve

Comment by Evan Ross [ 24/May/12 ]

Has this made any progress? I need the ability to create and destroy initiators at runtime.

Comment by Andy Flury [ 01/Oct/12 ]

Would like invest some time in this and come up with a solution and potential patch. Can somebody provide a hint how to approach this?

Comment by Andy Flury [ 01/Oct/12 ]

Ok, this is my first try.

I added a method createDynamicSession which takes a SessionID (that can be retriebed from settings). It will create the session and create the corresponding initiator.

Also, I added a new session level boolean setting "Inactive", that can be added to settings that should not start automatically when starting QuickFix. So this session can be started on demand with above new method. The constant SETTING_INACTIVE_SESSION would probably have to be moved somewhere else.

This might not be the proper way to do this, but it seems to be working.

Any comments are welcome

Andy





[QFJ-656] allowUnknownMsgFields has no effect if the field is not defined in the dictionary Created: 28/Nov/11  Updated: 25/Jun/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0, 1.5.0, 1.5.1
Fix Version/s: None

Type: Bug Priority: Major
Reporter: test Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   
The documentation of allowUnknownMsgFields is a bit misleading, it has no effect if the field is not defined in the dictionary. allowUnknownMsgFields is not checked in void checkValidTagNumber(Field<?> field) .

 Comments   
Comment by Greg Chabala [ 25/Jun/12 ]

This is still an issue in version 1.5.2





[QFJ-681] During load test, some clients are able to connect only after repeated attempts Created: 22/May/12  Updated: 22/May/12

Status: Open
Project: QuickFIX/J
Component/s: Engine, Networking
Affects Version/s: 1.5.2
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Viswanath Palutla Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ, logon
Environment: 1. on my PC, i am running 'Executor' from quickfixj examples
   PC config: windows7, i7, 2.7 GHz, 32-bit, 4GB RAM
   
2. one linux box, where i am starting 25-30 clients to put load on the 'Executor'
   linux config: Intel(R) Xeon(TM) CPU 3.20GHz, 2GB RAM

Attachments: Zip Archive load-test-executor.zip    

 Description   
I am using Executor service in the example, because it is similar to my FIX Gateway, in terms of using AcceptorTemplate

1. I have a simple client that connects to Executor service and puts a MarketDataRequest for every 10 milliseconds, total 100 market data requests.
   note: 'UnsupportedMessageType' errors are expected for all MarketDataRequests

2. two clients are started for every second. total 30 clients are started

Issues:
-> connection is not established for some clients for some period
    From client logs:
   (Pending connection not established after 2146 ms.)
   (Pending connection not established after 2850 ms.)
   (Pending connection not established after 9795 ms.)
   
-> server is not responding to logon requests promptly for some clients
   From client logs:
   (Initiated logon request)
   (Disconnecting: Timed out waiting for logon response)
   (Pending connection not established after 4648 ms.)
   (Pending connection not established after 21948 ms.)
   (Initiated logon request)
   (Received logon)


 Comments   
Comment by Viswanath Palutla [ 22/May/12 ]

attaching batch scripts and class files to load Executor





[QFJ-678] EventHandlingStrategy BlockingQueues are unbounded Created: 02/May/12  Updated: 02/May/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0, 1.5.2
Fix Version/s: None

Type: Improvement Priority: Minor
Reporter: Greg Chabala Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
In reference to a question on the quickfixj-users list from 2012-04-25:
"1. Why are all the LinkedBlockingQueues unbounded in the EventHandlingStrategies?

We've found that if you have to go back to the beginning of a session and have a large amount of traffic to process you can end up with memory issues without using a bounded queue."

Both quickfix.mina.EventHandlingStrategy implementations contain a BlockingQueue of Messages, or a wrapper of Messages in the case of SingleThreadedEventHandlingStrategy, and both of these queues are unbounded. We had an issue where while trying to recover from a period of downtime, the resent traffic would fill this queue faster than it could be processed, leading the app to run out of memory. We solved this by overriding the classes locally and giving the queues a capacity of 5000.

I'd like to see this improvement made into the vanilla library. While hard coding to some large value less than Integer.MAX_VALUE would probably be an improvement, ideally this could become a configuration option or an optional parameter to the constructors of Initiators and Acceptors.




[QFJ-677] The fix for QFJ-318 was reverted (inadvertently?) in r928 Created: 30/Apr/12  Updated: 30/Apr/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.2
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Ryan Caudy Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
We've found that the components (timestamp, message, newline) of log lines are occasionally interleaved in multi-threaded applications that concurrently send and receive FIX messages via QuickFIX/J 1.5.2. This issue wasn't present in QuickFIX/J 1.3.2.
It looks to me like the removal of "synchronized" from FileLog.writeMessage() in r928 was an error. None of the JIRA tickets cited in the commit log seem to explain the change.




[QFJ-663] FieldMap's "getGroup()", "replaceGroup()", and possibly others... should start with 0 (not 1) Created: 23/Jan/12  Updated: 23/Jan/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Trivial
Reporter: John Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
The first index for anything in the IT world is "0", not "1". Most certainly any programmer in this world expects the index "0" to be the first one.
An implementation must add 1 to each index, and the FIX engine then decrements it (will not kill the CPU but it is at least some work that could be avoided).

My solution would be to deprecate the methods and have new replacing methods (remove the deprecated methods and rename the new ones in a later version).




[QFJ-659] Sequence number issues when acceptor sends a message immediately following logon response Created: 09/Jan/12  Updated: 09/Jan/12

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0, 1.5.2
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andrew Richards Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: logon, sequnece


 Description   
I have an initiator connection to a foreign acceptor which has sequence number confusion on logon as the foreign acceptor sends a TestRequest immediately after the logon response.

It looks like the following is happening: Quickfix/J accepts both (Logon & TestRequest) messages. The and proceeds to check the sequence number of the Logon to make sure the session is valid. It does this via Session.isTargetTooHigh() which looks at the last message in the message store which is now 1 higher than is should be because the TestRequest has been received and stored. The "MsgSeqNum too high" is issued and the engine tries to recover. In some most circumstances, but not all it (with the engine in question) it can recover however it's not ideal.

The effect can be simulated (QuickFIX/J -> QuickFIX/J) by using and acceptor with the toAdmin below.


    public void toAdmin(Message message, SessionID sessionID)
    {
        print("toAdmin", message, sessionID);

        try {
            Header header = message.getHeader();
            if ( header.getField( new MsgType() ).valueEquals( MsgType.LOGON ) ) {
                quickfix.fix44.TestRequest tst = new quickfix.fix44.TestRequest(new TestReqID("TEST"));
                Session.sendToTarget(tst, sessionID);
            }
        }
        catch (Exception e) {}
    }


09-Jan-2012 08:49:19 INFO : Initialize
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 SEVERE : FIX.4.4:CLIENT->SERVER: MsgSeqNum too high, expecting 2 but received 3: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Enqueued at pos 3: 8=FIX.4.49=5535=034=349=SERVER52=20120109-08:49:50.01356=CLIENT10=107
09-Jan-2012 08:49:50 INFO : [toAdmin](FIX.4.4:CLIENT->SERVER)[2](MsgType:ResendRequest (2))
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=6435=234=249=CLIENT52=20120109-08:49:50.01956=SERVER7=216=010=238
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Sent ResendRequest FROM: 2 TO: 2
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=9335=434=243=Y49=SERVER52=20120109-08:49:50.02556=CLIENT122=20120109-08:49:5036=4123=Y10=182
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: ResendRequest for messages FROM 2 TO 2 has been satisfied.
09-Jan-2012 08:49:50 INFO : [fromAdmin](FIX.4.4:CLIENT->SERVER)[2](MsgType:SequenceReset (4))
09-Jan-2012 08:49:50 INFO : FIX.4.4:CLIENT->SERVER: Received SequenceReset FROM: 2 TO: 4
09-Jan-2012 08:50:20 INFO : [toAdmin](FIX.4.4:CLIENT->SERVER)[3](MsgType:Heartbeat (0))
09-Jan-2012 08:50:20 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=349=CLIENT52=20120109-08:50:20.21556=SERVER10=100
09-Jan-2012 08:50:20 INFO : FIX.4.4:CLIENT->SERVER: 8=FIX.4.49=5535=034=449=SERVER52=20120109-08:50:20.21856=CLIENT10=104
09-Jan-2012 08:50:20 INFO : [fromAdmin](FIX.4.4:CLIENT->SERVER)[4](MsgType:Heartbeat (0))








[QFJ-657] TestRequest should be sent before a Logout is sent. (Initiator) Created: 05/Dec/11  Updated: 06/Dec/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0, 1.5.1
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Ken Douglas Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: Logout, testRequest
Environment: Windows 7 Ultimate, Java 6 u29
QuickFix J 1.5.1


 Description   
TestRequest should be sent before a Logout is sent (Initiator).
You may not agree this is a bug by the way but perhaps a feature not currently supported but the FIX Protocol recommends the following:-

[1] a TestRequest is sent and a hearbeat is expected to be sent in return
[2] NB No logout is sent until a heartbeat (with the TestRequest ID) is received.
[3] Once receive a logout is sent and waitfor Logout is carried out.
[4] then disconnect is done.

I have seen a question relating to this on the forum but this goes way back
http://quickfix-j.364392.n2.nabble.com/Log-file-rotation-td665743.html


Currently the way the reset() on the Session is done it does not allow you to fire a TestRequest off and wait for the hearbeat.

Any idea guys?


Thanks


 Comments   
Comment by Christoph John [ 06/Dec/11 ]

I think this would be a good improvement. BTW, the link to the forum thread is wrong. It leads to a discussion about log rotation.





[QFJ-646] MiscFeeType field is handled incorrectly Created: 27/Oct/11  Updated: 17/Nov/11

Status: Open
Project: QuickFIX/J
Component/s: Engine, Metadata/Specs
Affects Version/s: 1.5.0, 1.5.1
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: John Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ


 Description   
The "MiscFeeType" extends StringField and the constants into it are "String"s. The specification however (and the dictionary) indicates that this field is a "char".
One of our clients sends this as a "String" (value "12"), but we reject the message because it is more than one "char" long. If he sent it as a "char", we would not reject, but it would not match the constants in the Class file.

Workarround, either:
- Change the dictionary to have this field be a "STRING" (used this one since our client sends it as a String).
- Have the "MiscFeeType" class extends "CharField" and change the constants in the Class file, and embed it into your own code to replace the one from the library (haven't tested that though).

 Comments   
Comment by Laurent Danesi [ 15/Nov/11 ]

Hi John,

Which FIX version are you using?

In fact, in the spec, this field was ever a char and recently changed to String with FIX5.0.
So the FIX4X.xml dictionaries are declaring it as a CharField and FIX50.xml is specifying it as a StringField accordingly with the spec.

Now, of course, if you need to downgrade FIX5 feature code to a FIX4X session, you need a custom dictionary but it is not a bug, isnt it?
What do you think?

Regards,

Laurent

Comment by John [ 15/Nov/11 ]

Hi,
I'm using the 4.x version. The problem is not to have "String" or "char", I do not care.
The problem is that if we receive a client's message as a char (which is the standard), it would not match the constants in the java source file

Comment by Laurent Danesi [ 15/Nov/11 ]

Hi,

In fact, we generate fields for each dictionary but only once for a field (no override) and we generate FIX50 before FIX4X so MiscFeeType is a StringField.

But I understand that if a client sends a String and MiscFeeType is a CharField, QFJ will reject but the current code should work as the parser will set the char as a String.anyway and we can match constants with equals method or I'm missing something perhaps?

Can you provide me a simple test to well understand and fix it please?

Laurent

Comment by John [ 17/Nov/11 ]
  • If the user sends as a String, he will send "12" => error on our side because it is more than two characters.
  • If the user sends as a character (character \0x000C), then it will be OK, BUT it will not match the constants declared in the java file "12" (this is a guess, maybe your code changes character '\0x000C' in String "12").




[QFJ-623] How do you change CompIDs, IP and port of a session at runtime? Created: 26/Jul/11  Updated: 19/Oct/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Other Priority: Default
Reporter: Andrew Niland Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment: java 1.6 on Windows


 Description   
How do you change CompIDs, IP and port of a session at runtime?

I would like to repoint the initiator engine to a different FIX server while the java app is still running.

 Comments   
Comment by Grant Birchmeier [ 26/Jul/11 ]

Did you try posting your question to the mailing list? This doesn't look like it should be a bug report.
http://quickfixj.org/support/

Comment by Laurent Danesi [ 19/Oct/11 ]

CompIDs are part of the SessionID, if you change them it is not the same Session anymore.
About the IP/port, you can specify multiple ip/port for the session and switch to them.

Regards,

Laurent DANESI





[QFJ-619] Late Resend Request causes sequence number mayhem : what happened? Created: 18/Jul/11  Updated: 18/Jul/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Other Priority: Default
Reporter: Nicolas Chaufette Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File trace.txt    

 Description   
Hello everyone,

A client seems to have a problem that I cannot reproduce. Basically, the third party asks for a resend request of a message sent 7 minutes earlier. During those 7 mintures, heartbeats have been exchanged and the sequence increased. When the Resend Request is received, this happens (look at the sequence numbers).
Is it a known bug in QuickFIX/J 1.4.0 ? fixed in 1.5.0 ?

Thanks, best regards,

Nicolas


<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:30:40.704" level="Enabled" thread="QFJ Timer">
  <fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=260|49=NOYOBU|52=20110705-11:30:40.704|56=NOYOBU|10=215|]]></fix:message>
</fix:trace>

[... nothing of importance, just heartbeats in and out for 7 minutes...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.695" level="Enabled" thread="QFJ Timer">
  <fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=267|49=NOYOBU|52=20110705-11:37:46.695|56=NOYOBU|10=244|]]></fix:message>
</fix:trace>

[... here seqnum is 267, right? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.788" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
  <fix:message length="88" way="incoming"><![CDATA[8=FIX.4.4|9=0064|35=2|49=NOYOBU|56=NOYOBU|34=269|52=20110705-11:37:46|7=260|16=0|10=102|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:46.788" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="45"><![CDATA[Received ResendRequest FROM: 260 TO: infinity]]></fix:event>
</fix:trace>

[... counterparty asks for 260, sent 7 minutes earlier ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.185" level="Enabled" thread="QFJ Message Processor">
  <fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=260|43=Y|49=NOYOBU|52=20110705-11:37:47.185|56=NOYOBU|122=20110705-11:37:47|36=261|123=Y|10=150|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="26"><![CDATA[Sent SequenceReset TO: 261]]></fix:event>
</fix:trace>

[... sending 260, reset to 261 ?? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
  <fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=260|43=Y|49=NOYOBU|52=20110705-11:37:47.186|56=NOYOBU|122=20110705-11:37:47|36=267|123=Y|10=157|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:37:47.186" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="26"><![CDATA[Sent SequenceReset TO: 267]]></fix:event>
</fix:trace>

[... sending 260 AGAIN, reset to 267 ????? ...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:46.819" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
  <fix:message length="77" way="incoming"><![CDATA[8=FIX.4.4|9=0053|35=0|49=NOYOBU|56=NOYOBU|34=270|52=20110705-11:38:46|10=121|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.699" level="Enabled" thread="QFJ Timer">
  <fix:message length="79" way="outgoing"><![CDATA[8=FIX.4.4|9=57|35=0|34=268|49=NOYOBU|52=20110705-11:38:47.698|56=NOYOBU|10=250|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.831" level="Enabled" thread="SocketConnectorIoProcessor-0.0">
  <fix:message length="88" way="incoming"><![CDATA[8=FIX.4.4|9=0064|35=2|49=NOYOBU|56=NOYOBU|34=271|52=20110705-11:38:47|7=261|16=0|10=098|]]></fix:message>
</fix:trace>

[... and it goes on and on and on and on...]

<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:47.831" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="45"><![CDATA[Received ResendRequest FROM: 261 TO: infinity]]></fix:event>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.187" level="Enabled" thread="QFJ Message Processor">
  <fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=261|43=Y|49=NOYOBU|52=20110705-11:38:48.187|56=NOYOBU|122=20110705-11:38:48|36=262|123=Y|10=158|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="26"><![CDATA[Sent SequenceReset TO: 262]]></fix:event>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
  <fix:message length="119" way="outgoing"><![CDATA[8=FIX.4.4|9=97|35=4|34=261|43=Y|49=NOYOBU|52=20110705-11:38:48.188|56=NOYOBU|122=20110705-11:38:48|36=268|123=Y|10=165|]]></fix:message>
</fix:trace>
<fix:trace logger="fix-engine-05301" timestamp="2011-07-05 13:38:48.188" level="Enabled" thread="QFJ Message Processor">
  <fix:event length="26"><![CDATA[Sent SequenceReset TO: 268]]></fix:event>
</fix:trace>




[QFJ-610] java.io.IOException when using FileStore Created: 17/Jun/11  Updated: 17/Jun/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Daniel Clusin Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: QuickfixJ
Environment: uname -a: Linux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:56:28 EST 20 06 x86_64 x86_64 x86_64 GNU/Linux
Java VM: BEA JRockit(R) R27.6.0-50_o-100423-1.5.0_15-20080626-2104-linux-x86_64,BEA Systems, Inc.


 Description   
java.io.IOException: Bad file descriptor
        at java.io.RandomAccessFile.seek(Native Method)
        at quickfix.FileStore.storeSequenceNumbers(FileStore.java:403)
        at quickfix.FileStore.incrNextSenderMsgSeqNum(FileStore.java:288)
        at quickfix.SessionState.incrNextSenderMsgSeqNum(SessionState.java:350)
        at quickfix.Session.sendRaw(Session.java:2177)
        at quickfix.Session.generateHeartbeat(Session.java:1786)
        at quickfix.Session.next(Session.java:1759)
        at quickfix.Session.next(Session.java:1032)
        at quickfix.mina.ThreadPerSessionEventHandlingStrategy$MessageDispatchingThread.run(ThreadPerSessionEventHandlingStrategy.java:119)

This occurred during a period of intermittent network disconnects. I did not see any forcible closes of the network sockets being done. Another time this occured is during a shutdown, after we called Initiator/Acceptor.stop

FIX.4.2:XXXX->XXXX: Error during message processing

quickfix.RuntimeError: java.io.IOException: Bad file descriptor
        at quickfix.SessionState.reset(SessionState.java:373)
        at quickfix.Session.resetState(Session.java:2193)
        at quickfix.Session.nextLogout(Session.java:1279)
        at quickfix.Session.next(Session.java:941)
        at quickfix.mina.ThreadPerSessionEventHandlingStrategy$MessageDispatchingThread.run(ThreadPerSessionEventHandlingStrategy.java:119)

Caused by: java.io.IOException: Bad file descriptor
        at java.io.RandomAccessFile.length(Native Method)
        at quickfix.FileStore.initializeCache(FileStore.java:119)
        at quickfix.FileStore.initialize(FileStore.java:111)
        at quickfix.FileStore.reset(FileStore.java:436)
        at quickfix.SessionState.reset(SessionState.java:370)
        ... 4 more




[QFJ-521] Support use of partial session ID for routing purposes Created: 10/May/10  Updated: 09/Jun/11

Status: Reopened
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.1, 1.3.2, 1.3.3, 1.4.0
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Rhys Yarranton Assignee: Grant Birchmeier
Resolution: Unresolved Votes: 1
Labels: None

Attachments: Text File diff3.txt    

 Description   
QFJ-272/1.3.1 added several fields to SessionID (SenderSubID, TargetSubID, SenderLocationID). These fields will be populated in MessageUtils.getReverseSessionID(...) if they are set in the Login message. They then form part of the session look-up in AbstractSocketAcceptor.StaticAcceptorSessionProvider.

In 1.2.1 and below, MessageUtils.getReverseSessionID included only the comp IDs, not the sub-IDs and SenderLocationID.

This breaks us in two ways. One is that we have been configuring our sessions by the comp IDs, which works well with the 1.2.1 code but mismatches with the 1.3.1+ code even if both sides get all their IDs right.

The other, and more important, is that several of our customer gateways will vary the SenderSubID and/or TargetSubID. For example, the FIX spec describes SenderSubID as being useful to distinguish between traders, and that messages not intended for a particular trader may optionally be flagged as ADMIN. It seems to me that the changes in QFJ-272 have resulted in a restrictions/obstacles beyond what was really intended by the spec, and certainly beyond what our customers will handle.

Fortunately there is a mechanism to provide your own AcceptorSessionProvider, which can be used to work around this issue. That said, the default behaviour is not ideal. Attached is a patch to replace StaticAcceptorSessionProvider with a new class, DefaultAcceptorProvider, which first tries a "strict" 1.3.1-style lookup, and if it fails, tries a 1.2.1-style lookup based on the Comp IDs only.

If the maintainers have an objection to the patch, I would suggest at the least that something like DefaultAcceptorProvider (plus instructions) be included in the distribution for those who cannot use StaticAcceptorSessionProvider.

 Comments   
Comment by Steve Bate [ 10/May/10 ]

The specifications include subID and locationID in information such as DeliverTo and OnBehalfOf, which are used for session routing. A route endpoint in this case is identified by CompID/SubID/LocationID in the originator sessionID and in the third-party routing cases. As you mentioned, I have seen the specification describe senderSubID as being useful to distinquish between traders, but this appears to me to still be in the context of message routing to a specific trader (or trading desk) within a broader organization (represented by the compID). Every counterparty I've worked with that used subIDs have used them for session identification and routing.

That said, from what I've seen on the FPL forums there was confusion in FIX 4.2 and before about the use of session ID fields for non-routing purposes. In FIX 4.3 and later, the Parties component is recommended for identifying traders outside the context of message routing.

I'm not the only one making the decision, but I'm open to to a feature that optionally ignores parts of the session ID for routing purposes.

Comment by Grant Birchmeier [ 19/Jul/10 ]

I am having similar issues as Rhys.

While reading the 4.4 spec to puzzle out my issues, I could not find where it says that SubID and LocationID should be used for anything substantial, let alone for message routing or session identification. Like Rhys, I'm not sure QFJ-272 is a correct interpretation. Such a change is not present in the latest QF/C++.

For now I'm integrating Rhys' patch into our local QF/J build, and being able to ignore the QFJ-272 additions is quite important for my company's usage.

Comment by Grant Birchmeier [ 19/Jul/10 ]

To integrate this patch into r958, DefaultAcceptorSessionProvider::getSession() needs a slight update to match an updated AcceptorSessionProvider interface:

  • public Session getSession(SessionID sessionID)
    + public Session getSession(SessionID sessionID, SessionConnector ignored)
Comment by Grant Birchmeier [ 12/Apr/11 ]

Committed to 1015

Comment by Jörg Thönnes [ 09/Jun/11 ]

Re-opening the issue to document the changes introduced
by this ticket.

IMHO, the handling of sub IDs and location IDs in QF/J deserves an extra section in the manual. There was some
ongoing discussion in the mailings lists and FPL forums
and it also not clear to me how QF/J handles this now.





[QFJ-582] Converters improvement for better performance Created: 30/Mar/11  Updated: 20/May/11

Status: In Progress
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: Future Releases

Type: Improvement Priority: Default
Reporter: Abla Nait Assignee: Eric Deshayes
Resolution: Unresolved Votes: 0
Labels: None


 Description   
Improve performance converters (double, int) and add tests to cover all cases.

 Comments   
Comment by Jörg Thönnes [ 30/Mar/11 ]

Hi Nait,

what kind of performance improvements you are planning. I have also some thoughts collected over the last years.

Cheers, Jörg

Comment by Abla Nait [ 30/Mar/11 ]

Remove Matcher in double converter and verify only if the first charater is not '+', other cases will be handled by parseDouble; and ditto for int converter.

Comment by Abla Nait [ 30/Mar/11 ]

Hi Jorg,
this is mainly very focused parsing improvements that we've done after some profiling and that we are integrating to the trunk (as part of the integration of our changes).

I let you open another JIRA for other or deeper improvements.
We could have some talks about it to share your thoughts with ours.

Regards,
Abla

Comment by Steve Bate [ 26/Apr/11 ]

As we discussed on the developer list, the current converter implementation is not FIX-compliant. The purpose for the Regex match to validate the input before passing it to Double for parsing. If this is causing significant performance issues then we may want to implement a fast double parsing function that validates while is parses/converts the string to a double. We should also add a unit test that shows the conversion fails for formats like "1e6" (scientific notation) which is not allowed by FIX.

Comment by Eric Deshayes [ 29/Apr/11 ]

As discussed previously, changes have been reverted.
Committed in integration branch in Rev#1025.

Comment by Eric Deshayes [ 20/May/11 ]

As discussed, some improvement on the converters could be done but they should not change the FIX compliance.

Changes have been reverted and performance improvements will be handled later.





[QFJ-466] Initiator SocketConnectorIoProcessor-0.0 deadlocks with QFJ Message Processor at logout Created: 01/Sep/09  Updated: 29/Apr/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Steve Borrer Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Ubuntu Linux 2.6.24-23-server #1 SMP Thu Nov 27 18:45:02 UTC 2008 x86_64 GNU/Linux


 Description   
The FIX initiator is logged out at 21:30 UTC by the acceptor (itself built with QFJ). The initiator logs out. An exception is reported in the initiator. Shortly after, the initiator initiates a logon request which is declined as it is outside the time window of the acceptor. There is then a endless sequence of 'Pending connection not established after XXXX ms' messages. See my application log below:-

20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - Received logout request
20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - Sent logout response
20090828-21:30:00 EVENT [FIX.4.2:NONAME->SIMULA] - DisconnectingAug 28, 2009 9:30:00 PM quickfix.mina.AbstractIoHandler exceptionCaught
SEVERE: socket exception (localhost/127.0.0.1:9876): Connection reset by peer

20090828-21:30:00 20090828-09:30:00 DEBUG - loggingOutState:true
20090828-21:30:00 20090828-09:30:00 DEBUG - Session logged out: FIX.4.2:NONAME->SIMULA
Aug 28, 2009 9:30:01 PM quickfix.mina.initiator.InitiatorIoHandler sessionCreated
INFO: MINA session created: /127.0.0.1:47950
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Initiated logon request
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Received logout request
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Disconnecting
20090828-21:30:02 20090828-09:30:02 DEBUG - loggingOutState:true
20090828-21:30:02 20090828-09:30:02 DEBUG - Session logged out: FIX.4.2:NONAME->SIMULA
20090828-21:30:02 EVENT [FIX.4.2:NONAME->SIMULA] - Attempt to send while not connected (message stored until connected).
20090828-21:30:33 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 2000 ms.
20090828-21:30:36 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 5000 ms.
20090828-21:30:39 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 8001 ms.
20090828-21:30:42 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 11001 ms.
20090828-21:30:45 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 14002 ms.
20090828-21:30:48 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 17002 ms.
20090828-21:30:51 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 20003 ms.
20090828-21:30:54 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 23003 ms.
20090828-21:30:57 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 26004 ms.
... etc

A SIGQUIT to the initiator reveals a deadlock:-

20090901-08:07:25 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 38094032 ms.
20090901-08:07:28 EVENT [FIX.4.2:NONAME->SIMULA] - Pending connection not established after 38097032 ms.
2009-09-01 08:07:28
Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.0-b16 mixed mode):

"Checkpointer" daemon prio=10 tid=0x00007f15040fa000 nid=0x1f7f in Object.wait() [0x0000000040d6d000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:161)
    - locked <0x00007f150f850a80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"Cleaner-1" daemon prio=10 tid=0x00007f15040f9800 nid=0x1f7e in Object.wait() [0x0000000041293000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:161)
    - locked <0x00007f150f84ca80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"INCompressor" daemon prio=10 tid=0x00007f1504216000 nid=0x1f7d in Object.wait() [0x0000000042400000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:163)
    - locked <0x00007f150f84ea80> (a java.lang.Object)
    at java.lang.Thread.run(Thread.java:619)

"SocketConnectorIoProcessor-0.0" daemon prio=10 tid=0x0000000040789000 nid=0x1f7c waiting for monitor entry [0x0000000042804000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at com.sleepycat.je.Cursor.close(Cursor.java:242)
    - waiting to lock <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.close(Database.java:242)
    - locked <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at quickfix.SleepycatStore.reset(SleepycatStore.java:285)
    at quickfix.SessionState.reset(SessionState.java:368)
    at quickfix.Session.resetState(Session.java:1828)
    at quickfix.Session.disconnect(Session.java:1542)
    at quickfix.mina.AbstractIoHandler.sessionClosed(AbstractIoHandler.java:100)
    at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.sessionClosed(AbstractIoFilterChain.java:677)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:292)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.common.support.AbstractIoFilterChain$HeadFilter.sessionClosed(AbstractIoFilterChain.java:599)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.fireSessionClosed(AbstractIoFilterChain.java:313)
    at org.apache.mina.common.support.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:271)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.doRemove(SocketIoProcessor.java:225)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$700(SocketIoProcessor.java:44)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:567)
    at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Client:nt@tradera1" prio=10 tid=0x000000004060c000 nid=0x178f runnable [0x0000000042905000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    - locked <0x00007f150ee02ab0> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
    - locked <0x00007f150ee02ab0> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"QFJ Message Processor" daemon prio=10 tid=0x00007f1504272000 nid=0x177b waiting for monitor entry [0x0000000042602000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at com.sleepycat.je.Database.removeCursor(Database.java:879)
    - waiting to lock <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at com.sleepycat.je.Cursor.close(Cursor.java:245)
    - locked <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.putInternal(Database.java:536)
    at com.sleepycat.je.Database.put(Database.java:479)
    at quickfix.SleepycatStore.set(SleepycatStore.java:301)
    at quickfix.SessionState.set(SessionState.java:299)
    at quickfix.Session.sendRaw(Session.java:1810)
    at quickfix.Session.generateLogout(Session.java:1050)
    at quickfix.Session.generateLogout(Session.java:1041)
    at quickfix.Session.nextLogout(Session.java:1024)
    at quickfix.Session.next(Session.java:790)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:107)
    at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:87)
    at java.lang.Thread.run(Thread.java:619)

"QFJ Timer" daemon prio=10 tid=0x00007f15042e1000 nid=0x177a in Object.wait() [0x0000000042501000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:443)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.TimeUnit.timedWait(TimeUnit.java:364)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.DelayQueue.take(DelayQueue.java:156)
    - locked <0x00007f150ec65480> (a java.lang.Object)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:680)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:921)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:980)
    at org.quickfixj.java4.edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:528)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Client:oms@tradera1" prio=10 tid=0x000000004018c000 nid=0x1773 runnable [0x0000000040b04000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    - locked <0x00007f150ede9ec8> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
    - locked <0x00007f150ede9ec8> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Client:oms@tradera1" prio=10 tid=0x00000000405e8000 nid=0x1772 runnable [0x0000000040a03000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    - locked <0x00007f150eddb7c8> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
    - locked <0x00007f150eddb7c8> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Client:oms@tradera1" prio=10 tid=0x0000000040477800 nid=0x1771 runnable [0x0000000040902000]
   java.lang.Thread.State: RUNNABLE
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:129)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
    - locked <0x00007f150ec74260> (a java.io.InputStreamReader)
    at java.io.InputStreamReader.read(InputStreamReader.java:167)
    at java.io.BufferedReader.fill(BufferedReader.java:136)
    at java.io.BufferedReader.readLine(BufferedReader.java:299)
    - locked <0x00007f150ec74260> (a java.io.InputStreamReader)
    at java.io.BufferedReader.readLine(BufferedReader.java:362)
    at oms.SimpleReader.readMessage(SimpleReader.java:26)
    at oms.SimpleSerializer.deserialize(SimpleSerializer.java:178)
    at fixclient.OMS_Client.run(OMS_Client.java:208)

"OMS_Interface" prio=10 tid=0x00007f15040a2000 nid=0x1770 runnable [0x0000000040c6c000]
   java.lang.Thread.State: RUNNABLE
    at java.net.PlainSocketImpl.socketAccept(Native Method)
    at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
    - locked <0x00007f150ebcace0> (a java.net.SocksSocketImpl)
    at java.net.ServerSocket.implAccept(ServerSocket.java:453)
    at java.net.ServerSocket.accept(ServerSocket.java:421)
    at fixclient.OMS_Interface.run(OMS_Interface.java:93)
    at java.lang.Thread.run(Thread.java:619)

"OMS_Sender" prio=10 tid=0x00007f1504059800 nid=0x1735 in Object.wait() [0x00000000422ff000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at fixclient.OMS_Sender.run(OMS_Sender.java:55)
    - locked <0x00007f150ebc01c0> (a java.util.LinkedList)

"MessageQueue" prio=10 tid=0x00007f1504058800 nid=0x1734 in Object.wait() [0x00000000421fe000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00007f150ebc40d8> (a java.util.LinkedList)
    at java.lang.Object.wait(Object.java:485)
    at fixclient.OMS_Interface$MessageQueue.run(OMS_Interface.java:312)
    - locked <0x00007f150ebc40d8> (a java.util.LinkedList)

"Low Memory Detector" daemon prio=10 tid=0x00000000402f2800 nid=0x1726 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x00000000402ef800 nid=0x1724 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x00000000402eb000 nid=0x1721 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x00000000402e9000 nid=0x171e waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00000000402c6000 nid=0x1714 in Object.wait() [0x0000000041ffc000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
    - locked <0x00007f150ebc2190> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x00000000402c4000 nid=0x170f in Object.wait() [0x0000000041efb000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Object.java:485)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
    - locked <0x00007f150ebc6168> (a java.lang.ref.Reference$Lock)

"main" prio=10 tid=0x0000000040112800 nid=0x16e5 in Object.wait() [0x00000000417f4000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00007f150ebc4258> (a fixclient.FIX_Client)
    at java.lang.Object.wait(Object.java:485)
    at fixclient.FIX_Client.run(FIX_Client.java:218)
    - locked <0x00007f150ebc4258> (a fixclient.FIX_Client)
    at fixclient.FIX_Client.main(FIX_Client.java:325)


"VM Thread" prio=10 tid=0x00000000402bd800 nid=0x170a runnable

"GC task thread#0 (ParallelGC)" prio=10 tid=0x000000004011c800 nid=0x16ea runnable

"GC task thread#1 (ParallelGC)" prio=10 tid=0x000000004011e800 nid=0x16ee runnable

"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040120800 nid=0x16f2 runnable

"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040122000 nid=0x16f6 runnable

"GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000040124000 nid=0x16fa runnable

"GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000040126000 nid=0x16fe runnable

"GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000040127800 nid=0x1702 runnable

"GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000040129800 nid=0x1706 runnable

"VM Periodic Task Thread" prio=10 tid=0x00000000402f5000 nid=0x1728 waiting on condition

JNI global references: 1325


Found one Java-level deadlock:
=============================
"SocketConnectorIoProcessor-0.0":
  waiting to lock monitor 0x000000004079f4d8 (object 0x00007f150f8529d8, a com.sleepycat.je.Cursor),
  which is held by "QFJ Message Processor"
"QFJ Message Processor":
  waiting to lock monitor 0x00000000407a0d20 (object 0x00007f150f852a18, a com.sleepycat.je.Database),
  which is held by "SocketConnectorIoProcessor-0.0"


Java stack information for the threads listed above:
===================================================
"SocketConnectorIoProcessor-0.0":
    at com.sleepycat.je.Cursor.close(Cursor.java:242)
    - waiting to lock <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.close(Database.java:242)
    - locked <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at quickfix.SleepycatStore.reset(SleepycatStore.java:285)
    at quickfix.SessionState.reset(SessionState.java:368)
    at quickfix.Session.resetState(Session.java:1828)
    at quickfix.Session.disconnect(Session.java:1542)
    at quickfix.mina.AbstractIoHandler.sessionClosed(AbstractIoHandler.java:100)
    at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.sessionClosed(AbstractIoFilterChain.java:677)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:292)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.access$900(AbstractIoFilterChain.java:54)
    at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:781)
    at org.apache.mina.common.support.AbstractIoFilterChain$HeadFilter.sessionClosed(AbstractIoFilterChain.java:599)
    at org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:321)
    at org.apache.mina.common.support.AbstractIoFilterChain.fireSessionClosed(AbstractIoFilterChain.java:313)
    at org.apache.mina.common.support.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:271)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.doRemove(SocketIoProcessor.java:225)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor.access$700(SocketIoProcessor.java:44)
    at org.apache.mina.transport.socket.nio.SocketIoProcessor$Worker.run(SocketIoProcessor.java:567)
    at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:43)
    at java.lang.Thread.run(Thread.java:619)
"QFJ Message Processor":
    at com.sleepycat.je.Database.removeCursor(Database.java:879)
    - waiting to lock <0x00007f150f852a18> (a com.sleepycat.je.Database)
    at com.sleepycat.je.Cursor.close(Cursor.java:245)
    - locked <0x00007f150f8529d8> (a com.sleepycat.je.Cursor)
    at com.sleepycat.je.Database.putInternal(Database.java:536)
    at com.sleepycat.je.Database.put(Database.java:479)
    at quickfix.SleepycatStore.set(SleepycatStore.java:301)
    at quickfix.SessionState.set(SessionState.java:299)
    at quickfix.Session.sendRaw(Session.java:1810)
    at quickfix.Session.generateLogout(Session.java:1050)
    at quickfix.Session.generateLogout(Session.java:1041)
    at quickfix.Session.nextLogout(Session.java:1024)
    at quickfix.Session.next(Session.java:790)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$SessionMessageEvent.processMessage(SingleThreadedEventHandlingStrategy.java:107)
    at quickfix.mina.SingleThreadedEventHandlingStrategy.block(SingleThreadedEventHandlingStrategy.java:70)
    at quickfix.mina.SingleThreadedEventHandlingStrategy$1.run(SingleThreadedEventHandlingStrategy.java:87)
    at java.lang.Thread.run(Thread.java:619)

Found 1 deadlock.

Heap
 PSYoungGen total 10944K, used 3722K [0x00007f1519660000, 0x00007f151a3e0000, 0x00007f151ebb0000)
  eden space 8448K, 36% used [0x00007f1519660000,0x00007f151995aaa8,0x00007f1519ea0000)
  from space 2496K, 26% used [0x00007f151a170000,0x00007f151a218000,0x00007f151a3e0000)
  to space 2688K, 0% used [0x00007f1519ea0000,0x00007f1519ea0000,0x00007f151a140000)
 PSOldGen total 26368K, used 14066K [0x00007f150ebb0000, 0x00007f1510570000, 0x00007f1519660000)
  object space 26368K, 53% used [0x00007f150ebb0000,0x00007f150f96c9f0,0x00007f1510570000)
 PSPermGen total 86016K, used 66374K [0x00007f15097b0000, 0x00007f150ebb0000, 0x00007f150ebb0000)
  object space 86016K, 77% used [0x00007f15097b0000,0x00007f150d881a38,0x00007f150ebb0000)


 Comments   
Comment by Eric Deshayes [ 29/Apr/11 ]

The problem is due to the usage of the SleepyCat database:

we're executing a Database.close() and Database.put() concurrently, and the javadoc for Database.close() points out that a database handle should not be closed while operations using that Database object are still running:
"The database handle should not be closed while any other handle that refers to it is not yet closed; for example, database handles should not be closed while cursor handles into the database remain open, or transactions that include operations on the database have not yet been committed or aborted."





[QFJ-584] Repeated test request with dynamic acceptor Created: 03/Apr/11  Updated: 05/Apr/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Other Priority: Major
Reporter: Information Management Experts Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Windows XP


 Description   
Dear All,

Our application is Client-Server System. Multiple clients(initiators) connect to the server (dynamic acceptor) to exchange fix messages.
We don't have physical configuration file; we create it at runtime. Our server(as initiator) also connects to other external server(second party).
Our clients can connect to the server through internal or external network and up to 10 concurrent clients may connect to the server.When we was using threads to send and receive messages, the client was sending test request and sometimes login request(when the server didn't respond to the client test request). after we removed the threads, the problem became less but still occurring.
Our time zone is Asia/Amman (GMT+2), but we still using default UTC.Below you will find our configuration file


# Acceptor Configuration

[DEFAULT]

StartTime=05:00:00

EndTime=05:00:00

FileStorePath=Settings/FIXStorage/AcceptorStore

FileLogPath=Settings/FIXStorage/AcceptorStore

DataDictionary=data/tools/IMEXQuickFIXDialect42.xml

ValidateUserDefinedFields=Y

CheckLatency=N

ConnectionType=acceptor

SocketAcceptPort=2025

[SESSION]

BeginString=FIX.4.2

SenderCompID=FIX

TargetCompID=*

AcceptorTemplate=Y







* Initiator Configuration

[DEFAULT]

StartTime=05:00:00

EndTime=05:00:00

FileStorePath=Settings/FIXStorage/InitiatorStore

FileLogPath=Settings/FIXStorage/InitiatorStore

DataDictionary=data/tools/IMEXQuickFIXDialect42.xml

ValidateUserDefinedFields=Y

CheckLatency=N

ConnectionType=initiator

ReconnectInterval=5

HeartBtInt=5

SocketConnectHost=123.123.123.123

SocketConnectPort=2025

[SESSION]

BeginString=FIX.4.2

SenderCompID= -- GENERATED AT RUNTIME

TargetCompID=FIX


Thanks in advanced for your help ...

 Comments   
Comment by Laurent Danesi [ 05/Apr/11 ]

Hi,

I saw your post but I'm not sure to well understand.
What do you mean by "When we was using threads to send and receive messages"? is it ThreadedSocketAcceptor?

Regards,

Laurent DANESI
Chief Architect
smartTrade Technologies

Comment by Information Management Experts [ 05/Apr/11 ]

Dear Laurent,

First of all, thanks for your reply.

No, we didn't use ThreadedSocketAcceptor. We were creating new threads in the implementation of Application.fromApp method and when we were calling Session.sendToTarget(message, sessionID);

Also we were sending huge no of orders at the same time(up to 1000) from the external server, which means when each order arrives to our server, our server will start new thread to handle the message, then a new thread will be created to send the message again for each client.





[QFJ-583] Session.forceStoreResync doesn't check persistMessages Created: 31/Mar/11  Updated: 31/Mar/11

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Dmitri Lenna Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Windows & Linux.


 Description   
We set the "ForceResync" to true and "PersistMessages" to false. Attempting connect to a target with a different sequence number, we get a logout message with the expected sequence number and the following is called:

(in Session.java):
private void forceStoreResync(int nextTargetMsgSeqNum, int nextSenderMsgSeqNum )

in that method, a heartbeat message is created for each missing sequence number and it's stored in the Message Store without checking if persisteMessage is true.

Is there a reason for this or this a bug? If it's a bug, I'd suggest this fix:

Line 1601:
- if (actualSenderMsgSeqNum < nextSenderMsgSeqNum) {
+ if (actualSenderMsgSeqNum < nextSenderMsgSeqNum && persistMessages ) {


Thank you.




[QFJ-569] QuickFix/J is not scalable due to overly long duration lock on sequence number in Session.sendRaw Created: 10/Dec/10  Updated: 10/Dec/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Major
Reporter: Leon Chadwick Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None


 Description   
I have been writing a capacity test to measure the performance of our FIX engine (not quickfix based), however I can barely get my machine to use any CPU during the test despite my using a large number of threads to generate orders and respond with Acks.
Sadly I will have to now drop usage of quickfix/j as this is a showstopper for a high performance system.

After profiling, I tracked down that all these threads are generally single threaded through quickfix.Session.sendRaw() which is locking to hold onto a sequence number. A scalable solution should keep locking to a minimum duration, i.e. do as much message checking and string conversion as possible, then fill in the sequence number as a short last step whilst inside the lock.

A StringBuilder could be used to hold the toString()'d message whilst not holding the lock and leave sufficient capacity at the head of the array to shift some of the header content left/right to accomodate the sequence number field whilst during the locked region.


 Comments   
Comment by Steve Bate [ 10/Dec/10 ]

Are you trying to send all these orders through one session? If so, the incoming messages must be processed sequentially by the session protocol engine. However, that doesn't mean your application interface implementation must be single threaded. The application implementation can hand off the message processing to a worker thread pool, for example. The downside is that the FIX engine won't be able to rollback the sequence numbers if an exception occurs in the application code.

As the comments state in sendRaw(), the lock is held until the message is processed and the application callback has completed. This is necessary because any exception thrown during this time will cause the sequence number to be rolled back. In other words, the sequence number can't be incremented until these operations complete successfully.

The message throughput depends on many factors. Some of the biggest factors include the number of sessions, the implementations of the message store and message log and the response time of the application implementation. For example, any logging to the disk in the application callback could have a significant impact on message throughput and CPU utilization.

I don't know if any of this applies to your situation since you didn't include many details. However, I know of other organizations that have reported throughput in the 10's of thousands of messages/second.





[QFJ-564] Operating on (modifying) repeating groups Created: 10/Nov/10  Updated: 10/Nov/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Addy Bhardwaj Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: N/A


 Description   
Currently if you want to add a value in a repeating group, there isn't a straightforward way of doing so. For instance,

//Just for illustration, if noSides group is read from TCR
TradeCaptureReport.NoSides noSidesGroup = new TradeCaptureReport.NoSides();
tradeCaptureReport.getGroup(1, noSidesGroup);

//add account information in TCR
noSidesGroup.set(new Account("accountA"));

This action doesn't modify the original TCR message as the group returned by getGroup() method is a copy of the actual group object.

Is there a reason for this behaviour? Also, I noticed that when we add a typed group object it is copied into a Group object and then stored in the underlying TreeMap of groups. The only reason I could find is that String to Object parsing doesn't create concrete group objects hence copy is required but I hoping there is another reason that I have overlooked.

ps: I have modified source code locally to improve this behaviour by
 - Removing the copy behaviour all together so concrete groups objects are kept in the message object
 - parsing string to QUICKFIXJ message object handles creation of concrete Groups as well.
I am more than happy to provide a patch if it this functionality is desirable. It should surely help optimise the code by reducing copies and object creation.




[QFJ-558] Can no longer send messages to initiators that have not yet been started Created: 13/Sep/10  Updated: 16/Sep/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: Text File diff.txt     Text File diff.txt    

 Description   
We have some system states where we disable network connection but still may have messages that we wish to deliver to the gateway, i.e., message store, for future delivery to the counterparty when we re-enable the network connection. Our technique is to create a SocketAcceptor or SocketInitiator, but not to start it. Since we upgraded from QF/J 1.2.1 to QF/J 1.4.0, this is no longer working for initiators.

Traced this back to QFJ-354, revision number 874. QFJ-354 allows a SocketInitiator to be stopped and restarted. (Previously it could be stopped, but the only way to restart it was to recreate it.) As part of this change, they moved where the sessions are created out of the constructor. (Session creation is one way a session can be registered.) QFJ-354 was part of the 1.4.0 release. There was no similar change for acceptors.

I don't see any reason why the session creation cannot be moved back into the constructor. It does not interfere with the intent of QFJ-354.
Patch attached.

While in the neighbourhood, noticed the following two lines in the AbstractSocketInitiator constructor:
{code:java}
ByteBuffer.setAllocator(new SimpleByteBufferAllocator());
ByteBuffer.setUseDirectBuffers(false);
{code}

This should be some sort of static initialization. _e.g._, for applications that use Mina for their own purposes as well as QF/J, they may need control over the buffer type and allocator, and shouldn't have to worry about QF/J changing it on the fly.

 Comments   
Comment by Rhys Yarranton [ 16/Sep/10 ]

Updated diff includes mod to ThreadedSocketInitiator.java.





[QFJ-556]  ThreadedSocketAcceptor/DynamicAcceptorSessionProvider do not get checked by SessionTimerTask Created: 03/Sep/10  Updated: 03/Sep/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Andre Mermegas Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
ThreadedSocketAcceptor/DynamicAcceptorSessionProvider Sessions do not get checked by SessionTimerTask because sessionconnector is always null from ThreadPerSessionEventHandlingStrategy.

DynamicAcceptorSessionProvider.getSession(...) {
if(sessionConnector != null) {
    sessionConnector.addDynamicSession(s);
}
}


ThreadPerSessionEventHandlingStrategy.getSessionConnector() {
return null;
}


simple fix i tested was to just add the connector constructor like in SingleThread version

    public ThreadPerSessionEventHandlingStrategy(SessionConnector connector) {
        sessionConnector = connector;
    }


 Comments   
Comment by Andre Mermegas [ 03/Sep/10 ]

and of course return the field instead of null
ThreadPerSessionEventHandlingStrategy {
....
public SessionConnector getSessionConnector()

{ return sessionConnector; }

....
}

updated constructor calls in
ThreadedSocketInitiator, ThreadedSocketAcceptor
private final ThreadPerSessionEventHandlingStrategy eventHandlingStrategy = new ThreadPerSessionEventHandlingStrategy(this);





[QFJ-555] Problems with heartbeating in one session can obstruct heartbeating on other sessions Created: 01/Sep/10  Updated: 01/Sep/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
SessionConnector.scheduledExecutorService is a single-threaded executor, and is used by all sessions for heartbeating. If there is a problem in one session, it can impact other sessions.

This is really QFJ-291 again. QFJ-291 was stated in terms of initiator reconnects, and the fix to it targeted that specific scenario. However, we have just encountered this problem in production involving only heartbeats. Heartbeating seems innocuous, but in fact has most of the same vulnerabilities as sending any other message (logging, toAdmin etc.).




[QFJ-554] SleepycatStore can enter infinite loop retrieving messages for resend. Created: 26/Aug/10  Updated: 26/Aug/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: James Olsen Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
SleepycatStore does not check returned OperationStatus from call to com.sleepycat.je.Cursor.getNext(...):

public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException {
...
if (retVal == OperationStatus.NOTFOUND) {
log.debug(sequenceKey + "/" + messageBytes + " not matched in database "
+ messageDatabase.getDatabaseName());
return;
} else {
Integer sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
while (sequenceNumber.intValue() <= endSequence) {
messages.add(new String(messageBytes.getData(), charsetEncoding));
if (log.isDebugEnabled()) {
log.debug("Found record " + sequenceNumber + "=>"
+ new String(messageBytes.getData(), charsetEncoding) + " for search key/data: "
+ sequenceKey + "=>" + messageBytes);
}
cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);
sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
}
}
...
}

Should be:

public synchronized void get(int startSequence, int endSequence, Collection<String> messages) throws IOException {
...
if (retVal == OperationStatus.NOTFOUND) {
log.debug(sequenceKey + "/" + messageBytes + " not matched in database "
+ messageDatabase.getDatabaseName());
return;
} else {
Integer sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
while (retVal == OperationStatus.SUCCESS && sequenceNumber.intValue() <= endSequence) {
messages.add(new String(messageBytes.getData(), charsetEncoding));
if (log.isDebugEnabled()) {
log.debug("Found record " + sequenceNumber + "=>"
+ new String(messageBytes.getData(), charsetEncoding) + " for search key/data: "
+ sequenceKey + "=>" + messageBytes);
}
retVal = cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);
sequenceNumber = (Integer) sequenceBinding.entryToObject(sequenceKey);
}
}
...
}

 Comments   
Comment by James Olsen [ 26/Aug/10 ]

For clarity, the changed lines are:

while (retVal == OperationStatus.SUCCESS && sequenceNumber.intValue() <= endSequence)

{ ... }


retVal = cursor.getNext(sequenceKey, messageBytes, LockMode.DEFAULT);





[QFJ-551] Sender comp id doesnt not get reflected while creating a session if given at runtime Created: 04/Aug/10  Updated: 04/Aug/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major
Reporter: hitesh Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
Desc:
    i have a quickfix config (for initiator), which doesn't have sender comp id. from my application i get the sender comp id at runtime, which i then set it session settings object. i then create SocketInitiator, and start the initiator. in the onCreate() method, i get sessionID without the sender comp id. am i trying which is not possible? please let me know if i am doing something wrong.

Code:

public boolean initialize(String senderCompID){
SessionSettings settings = null;
try {
settings = new SessionSettings(settingsFileName);
} catch (ConfigError e) {
System.err.println("Could not find FIX setting file.");
e.printStackTrace();
System.exit(-1);
}
String fixVersion = null;
String sndrCompId = senderCompID;
settings.setString(SessionSettings.SENDERCOMPID, senderCompID); // sender compid added in the default session. also tried adding the same in NONE default session in the below lines of code.

String tgtCompId = null;
SessionID sessID;
Iterator<SessionID> ite = settings.sectionIterator();
while(ite.hasNext()) {
sessID = ite.next();
try {
fixVersion = settings.getString(sessID, SessionSettings.BEGINSTRING);
tgtCompId = settings.getString(sessID, SessionSettings.TARGETCOMPID);
                        //settings.setString(sessID, SessionSettings.SENDERCOMPID, senderCompID); // tried adding in one of the sessions.
}catch (Exception e) {
System.err.println("Could not get [BEGINSTRING] or [TARGETCOMPID] " +
"from session id [" + sessID.toString() + "]-[" + e.toString() + "]");
}
}

String fileStorePath;
String fileLogPath;
try {
fileStorePath = settings.getString("FileStorePath");
} catch (Exception e) {
fileStorePath = null;
System.err.println("Could not get [FileStorePath] [" + e.toString() + "]");
}

try {
fileLogPath = settings.getString("FileLogPath");
if(fileStorePath == null) {
fileStorePath = fileLogPath;
System.out.println("[FileStorePath] set to [FileLogPath] - [" + fileStorePath + "]");
}
} catch (Exception e) {
fileLogPath = null;
System.err.println("Could not get [FileLogPath] [" + e.toString() + "]");
}

MessageStoreFactory storeFactory = new FileStoreFactory(settings);
LogFactory logFactory = new FileLogFactory(settings);
MessageFactory messageFactory = new MessageFactory();
try {
initiator = new SocketInitiator(this, storeFactory, settings, logFactory, messageFactory);
} catch (ConfigError e) {
System.err.println("FIXClient Could not initiate socket!");
e.printStackTrace();
System.exit(-1);
}

String fixedString = fixVersion + "_" + sndrCompId + "_" + tgtCompId; // if i check my settings object i have sender comp id correctly inserted

// further code

initiator.start();
}




[QFJ-543] "connector" parameter in interface AcceptorSessionProvider.getSession() is undocumented and unclear Created: 19/Jul/10  Updated: 19/Jul/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.0
Fix Version/s: None

Type: Bug Priority: Minor
Reporter: Grant Birchmeier Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
See paste below from AcceptorSessionProvider. The parameter connector is absent from the javadoc, and it's not clear what it's for. As it's an interface, the parameter's purpose should be documented.

From the two instantiations of this interface (DynamicAcceptorSessionProvider and AbstractSocketAcceptor::StaticAcceptorSessionProvider), it appears this parameter is only used for dynamic sessions and is otherwise ignored.

    /**
     * Return a session for this sessionID. The session might be
     * created dynamically.
     *
     * @param sessionID
     * @return the associated session or null if no session can be
     * associated with the given ID.
     */
    Session getSession(SessionID sessionID, SessionConnector connector);

 Comments   
Comment by Grant Birchmeier [ 19/Jul/10 ]

Paste above is from r959 (the latest at time of bug creation).





[QFJ-531] To set the number of logon attempts Created: 23/Jun/10  Updated: 23/Jun/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: Future Releases
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Pranab Kumar Naik Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Linux


 Description   
I was trying to set up the number of failed logon attempts after which the logon session would be created with a different host. The configuration does match with the interval between reconnects, but I thought it would be useful if the number of reconnect attempts may also be configurable from the config file. Or is there an option that I maybe missing ?




[QFJ-529] RTT benchmarking Created: 05/Jun/10  Updated: 05/Jun/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Wayne Zhu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: windows 7 professional 64 bits, multiple core Intel


 Description   
I have tested with a simple server and client setup using a single session in the same machine(tcp socket on local host). The client sent in new orders and got execution reports back. If the orders were sent at a high data rate for a period of 100-200ms, the round trip time for a new order and the corresponding report was not good.

My understanding is that there are two different threads in sending and receiving messages. What could be the reason causing QuickFix engine to slow down processing the incoming messages? Since there is no network overhead, it's most likely to be caused by the decoder and the way FIX message is constructed.






[QFJ-518] Incomplete addressing on BusinessMessageReject when using a FIX routing network Created: 04/May/10  Updated: 12/May/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Cliff Evans Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: JDK 1.6_20


 Description   
If an exception is thrown in fromApp, e.g. FieldNotFound, and we're using FIX4.2 or greater when using a Routing Network, e.g. Autex, a BusinessMessageReject message is thrown. The message does not currently contain all of the addressing information required to successfully deliver the BMR message if the message isn't going directly to a recipient.

If the message which causes the BMR contains routing information in:

OnBehalfOfCompID (115) and OnBehalfOfSubID (116)

This information should be included in the BMR reply in:

DeliverToCompID (128) and DeliverToSubID (129)

This will allow the Routing network to correctly deliver the message to the Application of the final recipient. Not including these fields results in the message being delivered directly to the Routing Network itself which doesn't know what to do with it.

 Comments   
Comment by Cliff Evans [ 12/May/10 ]

It looks to me like there's just a missing call to:

reject.reverseRoute(header);

In the generateBusinessReject method. I can't think of any reason why you wouldn't route a BMR to the individual broker rather than the FIX routing network.





[QFJ-517] CLONE -On checksum errors, include information on the problem section of the stream Created: 29/Apr/10  Updated: 29/Apr/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.2.1, 1.3.0, 1.3.1
Fix Version/s: 1.3.2

Type: Improvement Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
If there is a checksum error an error is logged, but it does not include the problem section of stream. This would useful in determining if it was a garble, a problem with the counterparty's software or whatever.

A patch we've tried is to add the following snippet to FIXMessageDecoder.handleError(ByteBuffer, int, String, boolean), right before the "if(disconnect))":

        int mark = buffer.position();
        try {
            StringBuilder sb = new StringBuilder(text);
            sb.append("\nBuffer debug info: ").append(getBufferDebugInfo(buffer));
            buffer.position(0);
            sb.append("\nBuffer contents: ");
            try {
                final byte[] array = new byte[buffer.limit()];
                for(int i = 0; i < array.length; ++i)
                    array[i] = buffer.get();
                sb.append(new String(array, "ISO-8859-1"));
            } catch (Exception e) {
                sb.append(buffer.getHexDump());
            }
            text = sb.toString();
        } finally {
            buffer.position(mark);
        }


 Comments   
Comment by Rhys Yarranton [ 29/Apr/10 ]

This is not fixed in 1.4.0.





[QFJ-415] Use data dictionary to determine message category for admin messages Created: 16/Mar/09  Updated: 07/Apr/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.4.0
Fix Version/s: Future Releases

Type: Improvement Priority: Default
Reporter: Laurent Danesi Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
To fully support custom FIX dictionary, we should take care of msgcat to know if the message is an admin message or application message and not use hardcoded values.

 Comments   
Comment by Steve Bate [ 16/Mar/09 ]

The data dictionary is optional, so you'll need to handle the case of not having a data dictionary to determine admin vs. application messages.

Comment by Steve Bate [ 10/Jan/10 ]

In the absence of a data dictionary we can use the hardwired admin/app classifications but allow that to be overridden by the data dictionary.

Comment by Steve Bate [ 05/Apr/10 ]

I've looked at this and it's a bit tricky since there are several places in the code where we don't have a data dictionary and we must determine if the message is an application message or not. I've added predicates to the data dictionary to determine the category of the message, but more work is needed to convert the code to use every place.





[QFJ-362] Enhance message and message_log table Created: 31/Oct/08  Updated: 05/Apr/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: Future Releases

Type: New Feature Priority: Default
Reporter: Alvin Wang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
To faciliate user to monitor and manage FIX messages, I think we should add more columns into those 2 tables:

1. time (missing in messages_log)
2. message type
3. msgseqnum (missing in messages_log)
4. id (missing in messages)
5. isInbound (Y/N. Right now, sendercompid is the company and targetcompid is the counterparty. this is confusing from FIX message point of view and it is difficult to differentiate the direction)

Also I suggest to make column naming of these 2 tables be consistent to help developer to reuse code.

 Comments   
Comment by Steve Bate [ 05/Apr/10 ]

Just FYI, the messages table is for message resend and should not be used by anything other than the internal session functionality.





[QFJ-494] proected empty constructor for Sesson class Created: 29/Dec/09  Updated: 05/Apr/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: Future Releases

Type: Improvement Priority: Default
Reporter: Carlo Marchiori Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: n/a


 Description   
I'd like to create a wrapper implementation and subclass of the Session class.
This wrapper is by default stateless, delegating behaviour to a true Session object.
This is useful for example to start global transactions around the next methods.

Currently Session has two constructor that can only create 'true' Sessions.

An empty protected constructor may suffice.

In fact, it's not clear why there is a SessionFactory interface when actually one cannot modify or decorate the default Session implementation.




 Comments   
Comment by Steve Bate [ 05/Apr/10 ]

IIRC, the Session interface was requested by a user for some purpose related to Spring. It's currently used for unit test support as well.

We could potentially add an interface for Session, but have you considered using some form of AOP to implement the transaction hooks.





[QFJ-394] SocketInitiator.stop(false) always waits sessions logoutTimeout before exit Created: 15/Jan/09  Updated: 03/Feb/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Alexey Utkin Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
SocketInitiator.stop(false) doesn't check sessions state. It always waits session logoutTimeout

It seems a bug in SessionConnection.waitForLogout()


 Comments   
Comment by Peter Franzen [ 03/Feb/10 ]

As far as I can tell, the problem lies in SessionConnection.waitForLogout().

If a session hasn't received its logout response when this method is entered
waitForLogout() keeps it in the local set of logged on sessions until the session's
logout timeout expires.

I noticed this problem when moving from 1.2.1 to 1.3.x, it seems to have been
introduced in version 1.3.

The solution would be to check if the session has been logged out each time
the sessions in the local logged on set are checked.

This could be achieved by changing the inner while loop in waitForLogout() to

while (sessionItr.hasNext()) {
Session session = sessionItr.next();
if (elapsed >= session.getLogoutTimeout() * 1000L) {
try

{ session.disconnect(); }

catch (IOException e)

{ log.error(e.getMessage(), e); }

}
if (!session.isLoggedOn())
sessionItr.remove();
}





[QFJ-446] Tag appears more than once... the error happens for user defined fields which can appear more than once Created: 25/Jun/09  Updated: 10/Jan/10

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Default
Reporter: Ganeshan Guruswamy Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: None
Environment: Tag appears more than once... the error happens for user defined fields which can appear more than once


 Description   
Tag appears more than once... the error happens for user defined fields which can appear more than once

 Comments   
Comment by Ganeshan Guruswamy [ 25/Jun/09 ]

hi... we are in the middle of a project... any help here will help...

some more information on the problem:
------------------------------------------------------

We receieve fix messages that have some user defined tags that repeat. For example when an application returns
errors... it returns same tags again with different error... quickfix/j rejects this with the error message "Tag appears more than once.."
We need to find a way where in we can ignore this error...

8=FIX.4.49=36435=AQ34=449=FCI-ST56=Smoke-Test52=20090625-10:49:52568=1569=2749=99750=2453=2448=CUONGIM1XXX447=B452=13448=CUONGBK2X447=B452=19063=29064=S120189066=%1 %2 is not required field length.7340=27342=CUONGBK2X9064=S140169066=PartyValue %1 is invalid for PartyType %2 and PartyRole %3.7340=37342=BIC9065=7342=MEOR7342=CUONGBK2X774=9955=[N/A]10=246

for example in the above message tag 9064 is repeating. quickfix rejects this message with the error...

20090625-10:51:30.859: 8=FIX.4.49=11735=334=549=Smoke-Test52=20090625-10:51:30.85956=FCI-ST45=458=Tag appears more than once371=9064372=AQ373=1310=238

can someone help us... any help will be truly appreciated...

Comment by Mehul Patel [ 05/Aug/09 ]

Isn't this the same as QFJ-404?





[QFJ-405] Fields missing on resends Created: 26/Feb/09  Updated: 26/Feb/09

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.1
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Rhys Yarranton Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
In some circumstances fields may be omitted from re-sent messages.

We tracked this down to the following line in Session.parseMessage(String):
    msg.fromString(messageData, dataDictionary, false);
In our case there was a repeating group used in outbound messages only that was not present in the data dictionary. Those fields silently disappeared.

So, a workaround is to edit the data dictionary. But given the silent misbehaviour, I would rate this a bug.

Probably could fix this by creating a subclass of Message that treats the body as an immutable string --> overrides to parseBody, calculateString, calculateLength and calculateTotal. Hopefully there is a more elegant solution.




[QFJ-400] Make scheduledExecutorService non-static Created: 03/Feb/09  Updated: 16/Feb/09

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Mihai Sardarescu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
Hello all,

In order to start our FIX connector, we have a "start" call from our middleware (there could be more than one FIX connectors started at different times on the same JVM). On this call, a logger is paased to the connector and all the log of the FIX connector should be dumped to this logger. The log of the QuickfixJ session is easily routed. However, the log messages from Mina and of the QuickfixJ initiators are more difficult to route. We use InheritableThreadLocal to pass the logger to all threads started by our FIX connector.
We encountered a problem due to the fact that the QuickfixJ SessionConnector uses a static executor service for executing SessionTimerTask tasks. Could you please change this to non-static?
For more details, please refer to thread: http://sourceforge.net/mailarchive/message.php?msg_name=1714.65.220.39.123.1208198497.squirrel%40webmail3.hrnoc.net

We expect our application to go in production very soon (at most 2 month for now).

Thank you,
Mihai


 Comments   
Comment by Steve Bate [ 05/Feb/09 ]

Please remind me why you weren't able to write a SLF4J logger implementation that delegates to the thread local logger provided by the middleware product. All logging from all threads goes through SLF4J so it seems that this would route those messages through the logger you provided.

Comment by Mihai Sardarescu [ 05/Feb/09 ]

Hi Steve,

In order to start the FIX connector, I get a start call from the middleware. On this call, a logger is passed to my code and I shall redirect all log message written by quickfixj and mina to this logger.

To do that I have used an InheritableThreadLocal that holds the middleware logger object in order to pass it to all the created by quickfixj and mina code. The loggers created by the slf4j are just proxies to the current thread middleware logger. I can thus redirect the quickfixj and mina log messages to the middleware log.

This technique does not work in case a static executor (or a static thread) is used. For example:
1. The FIX connector is started using the middleware logger LOGGER1. When loading of the quickfixj library, the static executor is initialized and the the logger attached to it is LOGGER1. All log messages written on this logger will be directed to the LOGGER1.
2. A second instance of the FIXConnector is started in the same JVM using LOGGER2. As the static executor was already initialized, the log written by quickfixj on the static executor will be also routed to LOGGER1 (and not to LOGGER2 as expected). This leads to potentials problems - what if the first instance of the FIXConnector is stopped and the LOGGER1 is closed by the middleware (etc.)

Also, I do not see the benefit of using a static scheduledExecutorService for all the initiators (which also uses a daemon worker thread). Wouldn't it be better to use a non-static executor that can be shutdown once the initiator is stopped?

Regards,
Mihai

Comment by Steve Bate [ 06/Feb/09 ]

Hello Mihai,

As we discussed before, the purpose of the static (singleton) executor is to minimize the number of threads used by QFJ for it's own purposes. The executor is shared among all initiators and acceptors.

I'm still not understanding why using a custom SLF4J implementation didn't work for you. If the custom logger implementation delegates to the /current/ middleware logger then it doesn't matter when the logger client actually retrieved the SLF4J logger instance. Actually, you could do this even without SLF4J by just implementing the QFJ Log interface. Your custom Log would delegate to the middleware logger. The middleware callback would modify the logger delegate in your Log implementation and everything that uses the Log implementation would automatically be using the new middleware logger. If the middleware logger is thread-local then that could also be supported with this approach.

I'm probably missing something, but this is my understanding at this point based on the information I have. Feel free to clarify any key issues I'm not seeing. I want to understand why this can't be done in the way I describe before changing the QFJ internals to support it.

Thanks,

Steve

Comment by Mihai Sardarescu [ 16/Feb/09 ]

Hi Steve,

The middleware provides a different logger object for each FIX connector. As I have one and only one QFJ session per connector, I have implemented a special QFJ logger that routes the log of one QFJ session to the middleware logger - this works fine and the log messages are routed to the correct logger.
However, I am missing the mina log and the low-level log of QFJ (the log that is written using slf4j). In fact, each FIX connector has its own initiator - the initiator is started upon the initialization of the connector and is stopped once the connector is destroyed. The problem is that once an initiator writes log messages from the static thread, there is no way to find out the which connector this initiator corresponds in order to route the log message to the correct logger.

To make it more clear: let's suppose I have two concurrent FIX connectors running in the same JVM having:

  • LOGGER1, LOGGER2 - the middleware loggers
  • INITIATOR1, INITIATOR2 - the initiators.
    INITIATOR1 logs something using the slf4j logger: log.debug("MESSAGE1") from the static thread THREAD1.
    INITIATOR2 logs something using the slf4j logger: log.debug("MESSAGE2") from the static thread THREAD1.
    There is no way to decide if the MESSAGE1 should be routed to LOGGER1 or LOGGER2 (the same applies for MESSAGE2). If the thread was non-static, then using inheritable thread locals could allow me to route the message to the right logger.

Thanks,
Mihai





[QFJ-385] If one Session reset failed, it will cause all existed session failed Created: 17/Dec/08  Updated: 17/Dec/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Major
Reporter: Mike Gu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Windows XP


 Description   
If one SessionState reset() failed, it will throw a RuntimeError and cause all existed session failed. It's unacceptable.
    public void reset() {
        try {
            messageStore.reset();
        } catch (IOException e) {
            throw new RuntimeError(e);
        }
    }




[QFJ-373] Add a new validation configruation to bypass the data type check Created: 20/Nov/08  Updated: 26/Nov/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Default
Reporter: Alvin Wang Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Comments   
Comment by Steve Bate [ 26/Nov/08 ]

Can you be more specific about what you want in this feature? Thanks.

Comment by Alvin Wang [ 26/Nov/08 ]

For example, if there is a configuration called ValidateDateType=N in the configuration file, my QFJ server will not reject a message in which there is "867=100,0" rather than "867=100.0" because I do not care about tag 867.

thanks.





[QFJ-369] Send too many logout messages when continuely received messages which has BadTime or doBadCompID Created: 06/Nov/08  Updated: 06/Nov/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.3.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Mike Gu Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Environment: Windows XP


 Description   
Send too many logout messages when continuely received messages which has BadTime or doBadCompID.
Session.java
    private void doBadCompID(Message msg) throws IOException, FieldNotFound {
        generateReject(msg, SessionRejectReason.COMPID_PROBLEM, 0);
        generateLogout("Bad CompID");
    }

    private void doBadTime(Message msg) throws IOException, FieldNotFound {
        generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0);
        generateLogout();
    }


Before Send a logout message, should check the session statues. isLogoutSent()

    private void doBadCompID(Message msg) throws IOException, FieldNotFound {
        generateReject(msg, SessionRejectReason.COMPID_PROBLEM, 0);
        if (!state.isLogoutSent())
              generateLogout("Bad CompID");
    }

    private void doBadTime(Message msg) throws IOException, FieldNotFound {
        generateReject(msg, SessionRejectReason.SENDINGTIME_ACCURACY_PROBLEM, 0);
        if (!state.isLogoutSent())
               generateLogout();
    }





[QFJ-242] Check for presence of conditionally required fields (OpenFIX) Created: 19/Sep/07  Updated: 02/Feb/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.2.1
Fix Version/s: Future Releases

Type: New Feature Priority: Default
Reporter: Graham Miller Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
We are filing this because of a failing OpenFIX test, but we actually believe that the test itself is broken, so I won't describe the interaction here.

It would be nice if QuickFIX/J could check for the presence of conditionally required fields in messages when doing validation with a data dictionary. An example: In an execution report, we have the following description of ExpireDate and ExpireTime:

432 ExpireDate C Conditionally required if TimeInForce (59) = GTD and ExpireTime (126) is not specified.
126 ExpireTime C Conditionally required if TimeInForce (59) = GTD and ExpireDate (432) is not specified.

QuickFIX/J should reject a message if 59=6 but the message is missing both 432 and 126.

Other than that, this request is pretty open-ended. Clearly there would need to be some language for expressing these conditional requirements in the data dictionary, but we would have to think about what that would look like.




[QFJ-169] Message parsing fails on messages with invalid fields in repeating groups with non-obvious errors Created: 28/Apr/07  Updated: 01/Feb/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.1.0
Fix Version/s: Future Releases

Type: Bug Priority: Default
Reporter: Toli Kuznets Assignee: Unassigned
Resolution: Unresolved Votes: 2
Labels: None

Attachments: Text File qfj169.patch    
Issue Links:
Duplicate
is duplicated by QFJ-269 validate method in DataDictionary doe... Closed
Relates
relates to QFJ-168 Add repeating group validation in mes... Closed

 Description   
When you create a Message from incoming string that has invalid fields in the repeating group, the message parsing code will fail with a non-obvious (and not helpful) error. This works when you have 2 or more groups and never on just 1 group.

Create a message and add a group to it, and add an invalid field to the first group.
Add group to the message
Add another (valid or invalid) group to the message.

Then try to "roundtrip" (ie do new Message(origMessage.toString(), dictionary)) and create a new message from the old one.
The creation will fail, and the error you get is "Tag not defined for this message" - but the tag you get is the first tag after the invalid one in the repeating group, not the actual tag that's invalid.

What happens is that in Message.parseGroup() function when are in a repeating group, when we find the first invalid field we essentially "pushback" the invalid field and get out of the group parse, skipping any other group fields.

another Bug: we report back the initially calculated group count, not the # of groups we actually parsed

We then add the pushed-back field to the new message, and try to inspect the next field - which belongs in the repeating group and not in message itself.
So the parser throws an exception in that case, saying that field is invalid, while it really should've given an error about the previous field.

See the attached patch for unit test.

The fix is non-trivial - you have to know whether the repeating groups are over and you just got the next "real message" field, or if it's just an invalid field for the repeating group.
Also, you can't just check if the pushed-back field doesn't belong to the parent message, since it may, but you may still have repeating groups coming up.

Perhaps the best way to fix it is to check if the "offending" field is a field in the parent's message type, and if it's not then report an error. You'd do it before pushing the field back. May need to pass the parent's message type through to parseGroup, in case we have groups nested inside other groups.

 Comments   
Comment by Toli Kuznets [ 28/Apr/07 ]

validation may catch this earlier

Comment by Toli Kuznets [ 02/May/07 ]

Patch containing the unit test exhibiting the failure
patch in core/ directory

Comment by Toli Kuznets [ 02/May/07 ]

Removed the code from the bug and added it as a patch attachments, should make the bug cleaner and easier to read.





[QFJ-278] method extractField(Group group, DataDictionary dataDictionary, FieldMap fields) don't check The length of "sohOffset" Created: 24/Dec/07  Updated: 01/Feb/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.2.1
Fix Version/s: Future Releases

Type: Bug Priority: Default
Reporter: CaiQi Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
In the class Message, method extractField(Group group, DataDictionary dataDictionary, FieldMap fields): The length of "sohOffset" is not checked. This will bring about full range of the message received as Message string out of range.

For all the field, if it is dataField, its length must be calculated by the former Field(Except for Tag 89/93). Because the data may contain a SOH.
For example, field 355 is dataField. Its length is in Field 354.
We must check the offset of the dataField is shorter than length of the message, and the offset's next char is SOH.

Add this code into Message.extractField():

//Judge if sohOffset's next char is '\001'.
if(messageData.indexOf('\001', sohOffset) != sohOffset)
    throw new InvalidMessage("Wrong length of data field.");
//Judge if sohOffset is shorter than fields.
if(sohOffset > messageData.length())
    throw new InvalidMessage("Wrong length of data field.");


 Comments   
Comment by CaiQi [ 24/Dec/07 ]

Initiator:send a message which is not comply with the protocol.
Acceptor:different problems occur.

For example, the dependances of fields. 354: the length of 355
354=10 355=123. The accptor will consider it that String out of range.
354=3 355=123. This is right.





[QFJ-277] if set "\001" in the StringField, it cause lots of problems Created: 24/Dec/07  Updated: 12/Jan/08

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.2.1
Fix Version/s: Future Releases

Type: Bug Priority: Default
Reporter: CaiQi Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None


 Description   
If "\001" is set in the string field by users, it can't be validated before sending. The field is considered as user expected. But it could be validated when received. There are three cases:

(1)Error incorrect format when Agent receives.
e.g. newOrderSingle.set(new Symbol("\001WAKEN"));
8=FIX.4.4_9=142_35=D_34=346_49=ICBC_50=operator001_52=20071128-03:03:37.140_56=XXX_212=346_11=123_38=1_40=2_44=1_54=1_55=_WAKEN_60=20071128-11:03:37.125_10=XXX_
Agent: [ERROR] quickfix.mina.acceptor.AcceptorIoHandler - Invalid message: bad tag format: For input string: "WAKEN_60"

e.g. newOrderSingle.set(new Symbol("WAK\001EN"));
8=FIX.4.4_9=142_35=D_34=329_49=ICBC_50=operator001_52=20071128-
03:00:50.875_56=XXX_212=329_11=123_38=1_40=2_44=1_54=1_55=WAK_EN_60=20071128-11:00:50.875_10=XXX_
Agent: [ERROR] quickfix.mina.acceptor.AcceptorIoHandler - Invalid message: bad tag format: For input string: "EN_60"

e.g. newOrderSingle.set(new Symbol("WAKEN\001"));
8=iMIX.1.0_9=142_35=D_34=365_49=ICBC_50=operator001_52=20071128-
03:06:13.671_56=XXX_212=365_11=123_38=1_40=2_44=1_54=1_55=WAKEN__60=20071128-11:06:13.671_10=XXX_
Agent: [ERROR] quickfix.mina.acceptor.AcceptorIoHandler - Invalid message: bad tag format: For input string: "_60"

e.g. newOrderSingle.set(new Symbol("WAK\001=EN"));
8=iMIX.1.0_9=143_35=D_34=312_49=ICBC_50=operator001_52=20071128-
02:56:44.578_56=XXX_212=312_11=123_38=1_40=2_44=1_54=1_55=WAK_=EN_60=20071128-
10:56:44.562_10=XXX_
Agent:[ERROR] quickfix.mina.acceptor.AcceptorIoHandler - Invalid message: bad tag format: For input string: ""


(2)The message is rejected as expected.

e.g. newOrderSingle.set(new Symbol("WAK\001123=EN"));
8=iMIX.1.0_9=146_35=D_34=269_49=ICBC_50=operator001_52=20071128-
02:50:38.406_56=XXX_212=269_11=123_38=1_40=2_44=1_54=1_55=WAK_123=EN_60=20071128-
10:50:38.406_10=XXX_
Incorrect data format for value

e.g. newOrderSingle.set(new Symbol("\001123=EN"));
8=iMIX.1.0_9=143_35=D_34=248_49=ICBC_50=operator001_52=20071128-
02:49:02.406_56=XXX_212=248_11=123_38=1_40=2_44=1_54=1_55=_123=EN_60=20071128-
10:49:02.406_10=XXX_
Tag specified without a value

e.g. newOrderSingle.set(new Symbol("WAKEN\001123="));
8=iMIX.1.0_9=146_35=D_34=383_49=ICBC_50=operator001_52=20071128-
03:12:25.546_56=XXX_212=383_11=123_38=1_40=2_44=1_54=1_55=WAKEN_123=_60=20071128-
11:12:25.531_10=XXX_
Tag specified without a value

(3)The message is received successfully, but it is wrong. The result is critical. It will be used to attack the Server. This mustn't absolutely happen.
e.g. newOrderSingle.set(new Symbol("WAKEN\00110048=2"));
8=iMIX.1.0_9=149_35=D_34=449_49=ICBC_50=operator001_52=20071128-
03:21:47.031_56=XXX_212=449_11=123_38=1_40=2_44=1_54=1_55=WAKEN_10048=2_60=20071128-
11:21:47.015_10=XXX_

Solution 1
Encode the StringField at the Client, and decode at the Server. The encode and decode can be implemented in class StringField.
Fault:
In the FieldType "DATA", "\001" is permited. Its length is defined by the former field rather than by the position of "\001". So it is not affected by "\001". When encoded, its length will be changed. So the former field's value must be modified. To avoid this, a new class must be added instead of StringField. Make all
the "DATA" type extend StringField.

Solution 2
In the FIX protocol, the String field is not permit to be set as '\001'. So we can throw Exception directly or replace '\001' with ' ' in the StringField.
This can be implemented in class stringField. When construct a StringField except DATA fieldType, if find "\001", throw Exception directly or replace '\001' with ' '.
I think the solution 2 is better. The concisest method is to judge the DataField by tags. Now there are at least 3 pairs of DataField: 93/89, 95/96, 354/355.
Hard code is written:
if(field == 89||field == 96||field == 355)

 Comments   
Comment by Steve Bate [ 08/Jan/08 ]

If I understand your concern correctly, this is more of a FIX protocol issue than a QuickFIX/J issue.On the QFJ side, we can validate
that a value of a String field doesn't contain a "\001". However, this doesn't stop the possibility of "server attacks". From the perspective
of the receiving engine, a "\001" is a field delimeter and the message will be parsed accordingly. Even if we validate the StringFields, your
counterparty can send whatever FIX message string they want with whatever fields they want so you must validate those anyway in your application
code.

Comment by Steve Bate [ 11/Jan/08 ]

This is more involved than I thought it would be. The data fields are based on StringField so I need to think more about to provide validation for String versus data fields.





Generated at Wed Sep 17 19:34:47 CEST 2014 using JIRA 6.2.1#6256-sha1:89f5b3f8f84d40330af33e974b1b83cd9a6871b1.