[QFJ-963] ConcurrentModificationException when using DynamicAcceptorSessionProvider Created: 19/Nov/18  Updated: 11/Jan/19  Resolved: 24/Nov/18

Status: Closed
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 2.1.0
Fix Version/s: 2.1.1

Type: Bug Priority: Default
Reporter: Alex Wibowo Assignee: Alex Wibowo
Resolution: Fixed Votes: 0
Labels: None


 Description   

We have recently upgraded from quickfixj 1.5.3 to 2.1.0, and saw the following error:

java.util.ConcurrentModificationException: null
        at java.util.HashMap$HashIterator.nextNode(HashMap.java:1442) ~[?:1.8.0-zing_18.06.0.0]
        at java.util.HashMap$KeyIterator.next(HashMap.java:1466) ~[?:1.8.0-zing_18.06.0.0]
        at java.util.AbstractCollection.finishToArray(AbstractCollection.java:232) ~[?:1.8.0-zing_18.06.0.0]
        at java.util.AbstractCollection.toArray(AbstractCollection.java:143) ~[?:1.8.0-zing_18.06.0.0]
        at java.util.ArrayList.<init>(ArrayList.java:178) ~[?:1.8.0-zing_18.06.0.0]
        at quickfix.mina.SessionConnector.getSessions(SessionConnector.java:160) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at org.quickfixj.jmx.mbean.connector.ConnectorAdmin.registerSessions(ConnectorAdmin.java:178) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at org.quickfixj.jmx.mbean.connector.ConnectorAdmin.lambda$postRegister$0(ConnectorAdmin.java:170) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at java.beans.PropertyChangeSupport.fire(PropertyChangeSupport.java:335) ~[?:1.8.0-zing_18.06.0.0]
        at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:327) ~[?:1.8.0-zing_18.06.0.0]
        at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:263) ~[?:1.8.0-zing_18.06.0.0]
        at quickfix.mina.SessionConnector.addDynamicSession(SessionConnector.java:166) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at quickfix.mina.acceptor.DynamicAcceptorSessionProvider.getSession(DynamicAcceptorSessionProvider.java:150) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at com.anz.axle.direct.connectafix.quickfix.QuickfixAcceptor$1.getSession(QuickfixAcceptor.java:107) ~[spdee-direct-connectafix-4.1798.jar:?]
        at quickfix.mina.acceptor.AcceptorIoHandler.findQFSession(AcceptorIoHandler.java:118) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at quickfix.mina.AbstractIoHandler.messageReceived(AbstractIoHandler.java:129) ~[quickfixj-core-2.1.0.jar:2.1.0]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:997) ~[mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:641) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1114) [mina-core-2.0.19.jar:?]
        at org.apache.mina.filter.codec.ProtocolCodecFilter$ProtocolDecoderOutputImpl.flush(ProtocolCodecFilter.java:437) [mina-core-2.0.19.jar:?]
        at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:256) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:641) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:1114) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.IoFilterAdapter.messageReceived(IoFilterAdapter.java:121) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:641) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.fireMessageReceived(DefaultIoFilterChain.java:634) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:539) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$1200(AbstractPollingIoProcessor.java:68) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.process(AbstractPollingIoProcessor.java:1242) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.process(AbstractPollingIoProcessor.java:1231) [mina-core-2.0.19.jar:?]
        at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:683) [mina-core-2.0.19.jar:?]
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core-2.0.19.jar:?]

which seems like a bug in quickfixj itself. Looking at how the map inside SessionConnector get set... it looks like it is just a HashMap from AbstractSocketAcceptor#createSessions().
Shouldnt it use a concurrent map instead?

 Comments   
Comment by Christoph John [ 19/Nov/18 ]

Hi Alex Wibowo,

how often does this occur? Every time when adding a Session? Or only from time to time?
I am not sure if the problem is caused by the intermediate call to ConnectorAdmin.registerSessions().

Cheers,
Chris.

Comment by Alex Wibowo [ 19/Nov/18 ]

Not that often. I think we've seen it only today after we upgraded to 2.1.0

Comment by Christoph John [ 19/Nov/18 ]

Hmkay, best would be if we had a reproducer. Just tried to reproduce it but didn't have any luck yet.

Comment by Christoph John [ 19/Nov/18 ]

I mean there are of course ways around the CME but first I'd like to reproduce it to be on the safe side.

Comment by Alex Wibowo [ 19/Nov/18 ]

Apology for asking the obvious... but you tried establishing with more than 1 session, yeah? I was able to reproduce it using 50 connections just now..

Comment by Christoph John [ 19/Nov/18 ]

Yeah, more than 1 but not 50.

Comment by Christoph John [ 19/Nov/18 ]

Tried to reproduce using two threads and creating 1000 sessions concurrently but to no avail. Seems like I'm missing something specific to your configuration...

Generated at Sun Apr 28 18:42:32 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.