[QFJ-930] FIXSettings not set for new DataDictionary objects created by DefaultDataDictionaryProvider Created: 18/Jul/17  Updated: 29/Jan/18

Status: Open
Project: QuickFIX/J
Component/s: Engine
Affects Version/s: 1.6.3
Fix Version/s: None

Type: Bug Priority: Default
Reporter: Sachin Agrawal Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None

Attachments: File quickfix_930.rar    

 Description   

While DefaultSessionsFactory.processFixtDataDictionaries adds ApplicationDictionary to dataDictionaryProvider it uses beginStringQualifier only.

While method of MessageUtils
(Message parse(Session session, String messageString) throws InvalidMessage)

method used session,messageString both to get the ApplVerId. this results in mismatched key inside DefaultDataDictionaryProvider and DefaultDataDictionaryPovider need to create a new DataDictionary but FIX sesion settings are not set here at all, which are set by DefaultSessionsFactory while executing processFixtDataDictionaries inside createDataDictionary.

Please let me know in case this is intentional.

 Comments   
Comment by Christoph John [ 18/Jul/17 ]

I am sorry, but I cannot follow your description. Could you maybe describe the problem with some code examples or a unit test?
Thanks,
Chris.

Comment by Sachin Agrawal [ 19/Jul/17 ]

Thanks Chris for looking into,i now try to be more specific.

Please look into DeafultSessionFactory.processFixtDataDictionaries, we find following piece of code, please note here that toApplVerID is used to create the instance of ApplVerID before the applicationDictionary is added to the datadictionaryProvider.

final DataDictionary dd = createDataDictionary(sessionID, settings, key,
                            beginStringQualifier);
dataDictionaryProvider.addApplicationDictionary(MessageUtils.toApplVerID(beginStringQualifier), dd);

and createDataDictionary method reads the settings 

private DataDictionary createDataDictionary(SessionID sessionID, SessionSettings settings,
            String settingsKey, String beginString) throws ConfigError, FieldConvertError {
        final String path = getDictionaryPath(sessionID, settings, settingsKey, beginString);
        final DataDictionary dataDictionary = getDataDictionary(path);

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER)) {
            dataDictionary.setCheckFieldsOutOfOrder(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_FIELDS_OUT_OF_ORDER));
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES)) {
            dataDictionary.setCheckFieldsHaveValues(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_FIELDS_HAVE_VALUES));
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS)) {        	
            boolean chkUnorderedGrpFields = settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS);
            settings.getLogger().error("Msg SttString "+Session.SETTING_VALIDATE_UNORDERED_GROUP_FIELDS +" value "+ chkUnorderedGrpFields +" dataDictionary " +dataDictionary.toString() +" beginString "+beginString);
			dataDictionary.setCheckUnorderedGroupFields(chkUnorderedGrpFields);
        }

        if (settings.isSetting(sessionID, Session.SETTING_VALIDATE_USER_DEFINED_FIELDS)) {
            dataDictionary.setCheckUserDefinedFields(settings.getBool(sessionID,
                    Session.SETTING_VALIDATE_USER_DEFINED_FIELDS));
        }

        if (settings.isSetting(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS)) {
            dataDictionary.setAllowUnknownMessageFields(settings.getBool(sessionID,
                    Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS));
        }

        return dataDictionary;
    }

Please note that DefaultDataDictionaryProvider uses ApplVerID as the key of the map.

While there is a method called MessageUtils.parse(Session session, String messageString), which uses getApplVerID(session, messageString) to create the ApplVerId and uses the same to fetch the applicationDictionary from dataDictionaryProvider.

MessageUtils.parse(Session session, String messageString)
.....
final String beginString = getStringField(messageString, BeginString.FIELD);
		final String msgType = getMessageType(messageString);

		ApplVerID applVerID;

		if (FixVersions.BEGINSTRING_FIXT11.equals(beginString))
		{
			applVerID = getApplVerID(session, messageString);
		}
		else
		{
			applVerID = toApplVerID(beginString);
		}
		final MessageFactory messageFactory = session.getMessageFactory();

		final DataDictionaryProvider ddProvider = session.getDataDictionaryProvider();
		final DataDictionary sessionDataDictionary = ddProvider == null ? null : ddProvider.getSessionDataDictionary(beginString);
		final DataDictionary applicationDataDictionary = ddProvider == null ? null : ddProvider.getApplicationDataDictionary(applVerID);
....

So when an application message is received a new DataDictionary is created by DefaultDataDictionaryProvider, look at method 

 public DataDictionary getApplicationDataDictionary(ApplVerID applVerID)

so the problem is that this new DataDictionary created by DefaultdataDictionaryProvider does not copy the FIX configurations i.e. settings. Hence even though the application dev has set the settings they are not picked while parsing messages.

Please ask further clarifications.

Thanks,
Sachin

Comment by Christoph John [ 19/Jul/17 ]

Hi,

I have changed https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/test/java/quickfix/DefaultSessionFactoryTest.java (method testFixtDataDictionaryConfiguration) slightly to set "AllowUnknownMessageFields" to true.

 @Test
    public void testFixtDataDictionaryConfiguration() throws Exception {
        SessionID sessionID = new SessionID(FixVersions.BEGINSTRING_FIXT11, "SENDER", "TARGET");
        setUpDefaultSettings(sessionID);
        settings.setBool(sessionID, Session.SETTING_USE_DATA_DICTIONARY, true);
        settings.setString(sessionID, Session.SETTING_TRANSPORT_DATA_DICTIONARY, "FIXT11.xml");
        settings.setString(sessionID, Session.SETTING_DEFAULT_APPL_VER_ID, "FIX.4.2");
        settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY, "FIX42.xml");
        settings.setString(sessionID, Session.SETTING_APP_DATA_DICTIONARY + "." + FixVersions.BEGINSTRING_FIX40, "FIX40.xml");
        settings.setString(sessionID, Session.SETTING_ALLOW_UNKNOWN_MSG_FIELDS, "Y");   // added

        Session session = factory.create(sessionID, settings);

        DataDictionaryProvider provider = session.getDataDictionaryProvider();
        assertThat(provider.getSessionDataDictionary(sessionID.getBeginString()),
                is(notNullValue()));
        final DataDictionary applicationDataDictionary = provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX42));

        assertThat(applicationDataDictionary,
                is(notNullValue()));
        assertThat(provider.getApplicationDataDictionary(new ApplVerID(ApplVerID.FIX40)),
                is(notNullValue()));
        
        System.err.println("XXXX " + provider.getSessionDataDictionary(sessionID.getBeginString()).isAllowUnknownMessageFields() );
        System.err.println("XXXX " + applicationDataDictionary.isAllowUnknownMessageFields() );
        
        quickfix.fixt11.Logon logon = new quickfix.fixt11.Logon(new EncryptMethod(EncryptMethod.NONE_OTHER), new HeartBtInt(30),
                new DefaultApplVerID(ApplVerID.FIX42));
        MessageUtils.parse(session, logon.toString());

This gives the following as output:

XXXX true
XXXX true

So the setting is picked up by both dictionaries.

Then I stepped into MessageUtils.parse(session, logon.toString()) with the debugger and could also see that the setting is true on the applicationDataDictionary.

So maybe I am taking a different code path than you? BTW I tested it on master, i.e. with version 1.7.0-SNAPSHOT.

Cheers,
Chris.

Comment by Sachin Agrawal [ 20/Jul/17 ]

Hi Chris,

I understand in your use case a new dictionary is never created, but there are some configurations in which a scenario occurs which creates a new dictionary by DefaultDataDictionaryProvider, then the settings will not be copied.
so to solve the problem we have 2 options

1) I correct the Fix configurations so that a new dictionary is never created.
2) Somehow set the session settings on each datadictionary created by DefaultDataDictionaryProvider

I see you are suggesting option 1.

--Regards,
Sachin

Comment by Christoph John [ 20/Jul/17 ]

Hi,

when there are such configurations then please give an example configuration. Otherwise I don't know how to reproduce the problem.

Thanks,
Chris.

Comment by Sachin Agrawal [ 29/Jan/18 ]

Attached is the zip src files with change

Add fix for correctly reading the settings in all the cases, i.e. whenever DataDictionary object is created from SessionFactory or MessageUtils.parse
    a) Update interface DataDictionaryProvider.java add DataDictionary getApplicationDataDictionary(SessionID sessionId, ApplVerID applVerID);
    b) Update DefaultDataDictionaryProvider
        1) add field SessionSettings
        2) add another constructor with SessionSettings, update old constructors to initialize field settings
        3) add implementation of new method getApplicationDataDictionary(SessionID sessionId, ApplVerID applVerID); which also readSettings\
        4) add public static method fillSettings
        5) add local method readSettings which calls static method fillsettings which is extracted from DefaultSessionFactory
    c) Update DefaultSessionFactory
        1) Update method create use new Constructor added for DefaultDataDictionaryProvider, pass settings.
        2) Update method createDataDictionary remove filling of settings locally and call DefaultDataDictionaryProvider.fillSettings instead
    d) Update MessageUtils, by updating method parse(Session session, String messageString) throws InvalidMessage so that appDataDictionary is fetched from ddProvider by also passing SessionID
    e) Update Session. next(Message message, boolean isProcessingQueuedMessages) throws FieldNotFound, RejectLogon, IncorrectDataFormat,
            IncorrectTagValue, UnsupportedMessageType, IOException, InvalidMessage, so that getApplicationDataDictionary is called with sessionID param as well.

Comment by Sachin Agrawal [ 29/Jan/18 ]

only improvement the changes i uploaded have is that whereever the DataDictionary object is created, the existing settings of  current session are always applied on DataDictionary created.

Comment by Christoph John [ 29/Jan/18 ]

Thanks for the file. Are you maybe able to generate a patch or even better open a pull request on github?
Thanks,
Chris.

Generated at Mon Apr 29 02:48:49 UTC 2024 using JIRA 7.5.2#75007-sha1:9f5725bb824792b3230a5d8716f0c13e296a3cae.