java 1.6.0._07
windows
quickfixj 1.4.0

I have solved this by modifying Session.java and placing the .class ahead of quickfix.jar.
rk2153 at gmail

Discovered the following:

• If unknown field is defined in the data dictionary, but not on the message type in question, then AllowUnknownMsgFields = true will have an effect.
• If the unknown fields is not in the data dictionary at all, AllowUnknownMsgFields = true will have no effect.

The problem appears to be in the order of steps in the DataDictionary.iterate() method.

• shouldCheckTag(field) only checks against user-defined fields (>= 5000) so it returns true.
• dd.checkValidTagNumber(field) checks against the whole dictionary's field list
• checkIsInMessage(field, msgType) is where AllowUnknownMsgFields is consulted. But this is too late if the field wasn't in the data dictionary at all.

Also note that the documentation for AllowUnknownMsgFields is misleading. The 1.4.0 implementation can have an effect regardless of the tag number.

Attached a patch which modifies iterate to skip dd.checkValidateTagNumber(field) and checkIsInMessage(field, msgType) if AllowUnknownMsgFields is true. I left dd.checkGroupCount(...) in place.

There may be some more fishiness in this area. For example, if you set ValidateUserDefinedFields to false, dd.checkGroupCount(...) will be skipped even if the field is in the data dictionary for the message. Is that right or wrong? It is not clear what the expected behaviour should be, especially given the inaccuracies in the documentation.

Please change this from "Other" to "Bug". We have customers who do weird things in production, such as sending us "mystery tags", and much as we might like them to adhere strictly to the spec, it isn't going to happen. So it is imperative that we are able to relax validation sufficiently to handle their behaviour.