[QFJ-982] Using the same dictionary file with different settings on different sessions causes problems Created: 02/Aug/19  Updated: 15/Jun/20

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: None
Fix Version/s: 3.0.0

Type: Bug Priority: Default
Reporter: Philip Whitehouse Assignee: Philip Whitehouse
Resolution: Unresolved Votes: 0
Labels: None

Issue Links:
Duplicate
is duplicated by QFJ-877 Multiple sessions with same dictionary Closed
Relates
relates to QFJ-981 Validation settings do not get applie... Closed

 Description   

[Affects all vaguely recent versions]

The DefaultSessionFactory stores a cache DataDictionary objects based on the file name of that object.

createDataDictionary fetches a DataDictionary object based on the file name and then applies settings to it based on the session.

Because it applies the settings to and returns the DataDictionary object in the cache, not a copy of it, when a different session uses a dictionary from the same file, it will also have the settings applied.

This means that configuration that's supposed to be session specific is not.

What we should do is instead have a `SessionDataDictionarySettings` object to encapsulate the settings. Then when using the data dictionary, pass in the relevant settings for the session the message is being validated for.

This would allow us to gain the advantages of reusing DataDictionary objects (lower memory usage) with the configuration settings required.

It will however likely involve API changes.

 Comments   
Comment by Christoph John [ 05/Aug/19 ]

Hi Philip Whitehouse,
funny, I also stumbled over this a few days ago while investigating https://github.com/quickfix-j/quickfixj/pull/224 / QFJ-981. Didn't get to investigate very far but IIRC the only settings that might differ are validation options 

    checkFieldsOutOfOrder
    checkFieldsHaveValues
    checkUserDefinedFields
    checkUnorderedGroupFields
    allowUnknownMessageFields

so I thought I would just use the path plus the validation options as key for the cache in DefaultSessionFactory. Of course that means that there might be several DDs with different validation options. But on the other hand, I do not know if this really is a big downside. Of course depends how much sessions with the same dictionary and different validation options one might have in one VM.
Your proposed solution is of course a little more sophisticated since we would only have one instance of a DD, however as you said there might be API changes.

What do you think?
Thanks,
Chris.

Comment by Philip Whitehouse [ 14/Aug/19 ]

The problem with multiple copies of the same DataDictionary is that they are a massive memory hog given the size of them (especially once you add all the service packs).

I agree it's just the validation options however. My VMs generally run lots of sessions with the same dictionary so I'm keen to avoid an unexpected RAM usage increase from fixing the bug.

The other side to it is a proxy layer we have allowing you to live-reload the dictionary file to correct issues. I might try to upstream this at the same time.

This is likely to be something I have time to proof-of-concept relatively soon as currently the bug provides run-time unreliability on what will actually be applied to validate a given message.
Which is fairly ugly in terms of potential impact.

Comment by Christoph John [ 14/Aug/19 ]

OK, so did I understand you correctly that you are going to take a stab on this?
Or do you want me to do it?

Sorry, non-native english speaker here, so it's sometimes hard for me to get the nuances.

Thanks,
Chris.

Generated at Sun May 05 19:21:32 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.