[QFJ-751] Incorrect handling of SequenceReset-GapFill messages when using ResendRequestChunkSize > 0 Created: 18/Jul/13  Updated: 02/Apr/15  Resolved: 28/Mar/14

Status: Closed
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.5.3
Fix Version/s: 1.6.0

Type: Bug Priority: Default
Reporter: Andrzej Hajderek Assignee: Christoph John
Resolution: Fixed Votes: 0
Labels: ResendRequest,, sequence,
Environment:

FIX 4.4, Initiator, ResendRequestChunkSize = 100

Attachments: Text File log-file.txt     Text File SessionTest.java    

 Description   

While performing a large session-layer recovery operation (around 1 million of messages) in chunks of 100 the QuickFIX/J engine got stuck, as it ignored a valid GapFill message:

18-07-13 08:56:50.673|QFJ Message Processor|INFO|quickfix: Received SequenceReset FROM: 1000102 TO: 1000103

18-07-13 08:56:50.673|QFJ Message Processor|ERROR|quickfix: MsgSeqNum too high, expecting 1000102 but received 1000103: 8=FIX.4.29=45135=834=100010343=Y49=ABFX52=20130718-08:56:50.66356=TEST_CLIENT122=20130718-00:09:001=B2BUSER60.STP6=0.816711=B2BCLIENT60-1374106069-1000102:014=4000015=EUR17=1374106069-100010220=021=131=0.816732=4000037=B2BCLIENT60-1374106069-1000102:038=4000039=240=D54=155=EUR/GBP60=20130718-00:08:54.57564=20140221109=B2BCLIENT60150=2151=0167=FOR194=0.82195=-0.003285544=GBP5549=AcmecoFXTest6054=32668.00999999=137413781066310=200

After analysis of the Session.nextSequenceReset() method it's clear that the incoming SequenceReset-GapFill message is handled differently when using non-zero ResendRequestChunkSize. Also, it looks like in this particular scenario the target sequence number has not been set (from 1000102) to newSequence (1000103) in the Session.nextSequenceReset() method, because:

1) range[2] was > 0 (which means using non-zero ResendRequestChunkSize)

AND

2) newSequence (1000103) was NOT >= range[1] (1084290)

AND

3) newSequence (1000103) was NOT >= range[2] (1000200)

[All values in the log below.]

It looks like the following logic needs to be reviewed because in the discussed scenario the newSequence value was completely ignored:

if (range[2] > 0) {
if (newSequence >= range[1])

{ state.setNextTargetMsgSeqNum(newSequence); }

else if (newSequence >= range[2])

{ state.setNextTargetMsgSeqNum(newSequence + 1); final String beginString = sequenceReset.getHeader().getString( BeginString.FIELD); sendResendRequest(beginString, range[1] + 1, newSequence + 1, range[1]); }

} else

{ state.setNextTargetMsgSeqNum(newSequence); }

The log is attached.

 Comments   
Comment by Andrzej Hajderek [ 18/Jul/13 ]

Attaching the actual log file, as the inline logs look awful.

Comment by Andrzej Hajderek [ 18/Jul/13 ]

BTW: Could someone please fix the incorrect tag: "sequnece" (swapped ne). It's used by a bunch of other issues as well.

Comment by Andrzej Hajderek [ 18/Jul/13 ]

The following test passes for ResendRequestChunkSize=0, but fails forResendRequestChunkSize=10:

// QFJ-751
@Test
public void testSequenceResetGapFillWithZeroChunkSize() throws Exception

{ testSequenceResetGapFillWithChunkSize(0); }

// QFJ-751
@Test
public void testSequenceResetGapFillWithNonZeroChunkSize() throws Exception

{ testSequenceResetGapFillWithChunkSize(10); }

// QFJ-751
@Ignore
private void testSequenceResetGapFillWithChunkSize(int chunkSize)
throws Exception {
final SessionID sessionID = new SessionID(
FixVersions.BEGINSTRING_FIX44, "SENDER", "TARGET");

boolean isInitiator = true, resetOnLogon = false, validateSequenceNumbers = true;

Session session = new Session(new UnitTestApplication(),
new MemoryStoreFactory(), sessionID, null, null,
new ScreenLogFactory(true, true, true),
new DefaultMessageFactory(), isInitiator ? 30 : 0, false, 30,
true, resetOnLogon, false, false, false, false, false, true,
false, 1.5, null, validateSequenceNumbers, new int[]

{ 5 }

,
false, false, false, true, false, true, false, null, true,
chunkSize, false, false);

UnitTestResponder responder = new UnitTestResponder();
session.setResponder(responder);

session.logon();
session.next();

assertEquals(1, session.getStore().getNextTargetMsgSeqNum());

Message logonRequest = new Message(responder.sentMessageData);

// Deliver Logon response with too high sequence 20 instead of 1.
session.next(createLogonResponse(sessionID, logonRequest, 20));

// The expected target sequence should still be 1.
assertEquals(1, session.getStore().getNextTargetMsgSeqNum());

// Deliver the missing message #1.
session.next(createAppMessage(1));
assertEquals(2, session.getStore().getNextTargetMsgSeqNum());

// Deliver the missing message #2.
session.next(createAppMessage(2));
assertEquals(3, session.getStore().getNextTargetMsgSeqNum());

// Deliver SequenceReset-GapFill from 3 to 5
final SequenceReset gapFill = new SequenceReset(new NewSeqNo(5));
gapFill.getHeader().setField(
new SenderCompID(sessionID.getTargetCompID()));
gapFill.getHeader().setField(
new TargetCompID(sessionID.getSenderCompID()));
gapFill.getHeader().setField(new SendingTime(new Date()));
gapFill.getHeader().setField(new MsgSeqNum(3));
gapFill.setField(new GapFillFlag(true));

session.next(gapFill);

// Deliver the missing message #5.
session.next(createAppMessage(5));
/*

  • The expected target sequence number should be 6 now.
    */
    assertEquals(6, session.getStore().getNextTargetMsgSeqNum());
    }
Comment by Andrzej Hajderek [ 19/Jul/13 ]

Please find attached the updated SessionTest.java source file, which now contains the two new test methods:

testSequenceResetGapFillWithZeroChunkSize()

and

testSequenceResetGapFillWithNonZeroChunkSize()

Comment by Andrzej Hajderek [ 19/Jul/13 ]

Not sure why the code I pasted directly got garbled. The attached file is OK.

Comment by Christoph John [ 28/Mar/14 ]

Committed as http://sourceforge.net/p/quickfixj/code/1168/

Generated at Mon May 06 03:09:07 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.