ParseMessage messes up order of repeating group tags about quickfix HOT 11 OPEN

The problem is on this line.

write() sorts the tags in the body without taking into account repeating groups.

from quickfix.

Good catch, the arbitrary use of sort() in the sortedTags is not FIX protocol compliant and removes a key advantage of using the FIX protocol for integrations. That being said, the sort here is just a hard coded hidden rule based on an opinion matching checks imposed by quickfix downstream from here and might be an oversight or lazy man's "quick fix" avoiding the work to validate correctly. The real issue is what sortedTags more than likely should be doing and how quickfix is not supporting an aspect of the FIX protocol, instead just opting to take a heavy hand and sort everything on write. What should be happening here is the sortedTags should be enforcing the repeating group sort order that the server expects in the agreed repeating group template. During integrations of a client to a server the client will format the repeating groups according to the tag order imposed by the server, not just numerical order by default.
This test also exposes an issue with enforcing arbitrary sort order in the header, body, or trailer that might cause message failure. Example: many servers expect headers to have 8,9, then 35 as the first three with no preference for the remaining header tags, or when looking at the body being numerically sorted might land a non repeating group tag inside a repeating group entry destroying it, etc...
This issue has been overlooked, maybe by design for work effort or opinion,but it is a hard coded rule opposite of the FIX protocol on this point that needs to be reviewed for correction. I think the other reason this has been overlooked is because integrations go through a process of alignment until the messages flow and agree, so we move on none the wiser. We do this assuming that during our work our implementation is what needs to change or be corrected to get things talking, especially on first use, because normally it is our fault things are not working.

from quickfix.

Hi @steelkorbin , I couldn't agree more. Totally understand how this might have slipped through. I mean, if I hadn't called ParseMessage, but instead sent the NOS message directly after it was constructed using fix44nos.New(), I wouldn't have encountered this problem at all. But, I had a special situation where the process that constructed the message was different from the quickfix initiator process and so had to transmit the message as bytes over the network to the initiator in order to send it to the counterparty. The other aspect that made this bug particularly difficult was that, in the quickfix.Send() code path, we log the message that we're sending by calling ToApp() before build() is called and unfortunately the output of ToApp() is perfectly fine. So, I was very perplexed when I got back a business message reject from the counterparty. I should have checked the messages table in the database itself.

from quickfix.

I'm seeing this type of behavior badly break processing of Fenics and Bloomberg FIX messages. Leading to missing groups or out of order groups that don't validate.

from quickfix.

Thanks for adding the reference to my reported bug. I did not do any changes in the received message and just sent it back to the sender using the same connection and the repeating group tags were missed up.

from quickfix.