Repeating Group Messages in Quickfix C++

Question

We cannot seem to process some group messages with QuickFix.

I am trying to set up a connection with the ICE exchange using QuickFix (C++). I have created a custom data dictionary to handle ICE's non-standard messages. The first message to handle is a SecurityDefinition. The message contains about 13000 entries broken into blocks of 100. I attached the message below (the first two entries with CR/LF added for clarity).

My question is in the data dictionary, I defined a group as part of the entry with all the fields they specify in the group. Yet the message is rejected before it gets to the cracker as having an invalid tag (tag = 305).

Message 2 Rejected: Tag not defined for this message type:305

Does this dictionary entry look correct? Is there any documentation anywhere on how to handle group messages?

Dictionary entry:

<message name='SecurityDefinition' msgcat='app' msgtype='d'>
    <field name='SecurityResponseID' required='Y' />
    <field name='SecurityResponseType' required='Y' />
    <field name='SecurityReqID' required='Y' />
    <field name='TotNoRelatedSym' required='N' />
    <field name='NoRpts' required='N' />
    <field name='ListSeqNo' required='N' />
    <group name='NoUnderlyings' required='N'>
        <field name='UnderlyingSymbol' required='N' />
        <field name='UnderlyingSecurityID' required='N' />
        <field name='UnderlyingSecurityIDSource' required='N' />
        <field name='UnderlyingCFICode' required='N' />
        <field name='UnderlyingSecurityDesc' required='N' />
        <field name='UnderlyingMaturityDate' required='N' />
        <field name='UnderlyingContractMultiplier' required='N' />
        <field name='IncrementPrice' required='N' />
        <field name='IncrementQty' required='N' />
        <field name='LotSize' required='N' />
        <field name='NumofCycles' required='N' />
        <field name='LotSizeMultiplier' required='N' />
        <field name='Clearable' required='N' />
        <field name='StripId' required='N' />
        <field name='StripType' required='N' />
        <field name='StripName' required='N' />
        <field name='HubId' required='N' />
        <field name='HubName' required='N' />
        <field name='HubAlias' required='N' />
        <field name='UnderlyingUnitOfMeasure' required='N' />
        <field name='PriceDenomination' required='N' />
        <field name='PriceUnit' required='N' />
        <field name='Granularity' required='N' />
        <field name='NumOfDecimalPrice' required='N' />
        <field name='NumOfDecimalQty' required='N' />
        <field name='ProductId' required='N' />
        <field name='ProductName' required='N' />
        <field name='ProductDescription' required='N' />
        <field name='TickValue' required='N' />
        <field name='ImpliedType' required='N' />
        <field name='PrimaryLegSymbol' required='N' />
        <field name='SecondaryLegSymbol' required='N' />
        <field name='IncrementStrike' required='N' />
        <field name='MinStrike' required='N' />
        <field name='MaxStrike' required='N' />
    </group>
</message>

Actual message:

8=FIX.4.49=5004335=d49=ICE34=252=20121017-00:39:41.38556=600357=23322=3924323=4320=1393=1310382=13267=1711=100 311=1705282309=TEB SMG0013-TFL SMG0013305=8463=FXXXXX307=NG Basis Futures Spr - TETCO-ELA/TGP-500L - Feb13542=20130131436=1.09013=0.00059014=2500.09017=25009022=289024=19025=Y916=20130201917=201302289201=11969200=129202=Feb139300=60589301=Texas Eastern Transmission Corp. - East Louisiana Zone/Tennessee Gas Pipeline Co. - Zone L, 500 Leg Pool9302=TETCO-ELA/TGP-500L998=MMBtus9100=USD9101=USD / MMBtu9085=daily9083=49084=09061=4909062=NG Basis Futures Spr9063=Natural Gas Basis Futures Spread9032=1.259004=17051939005=1353778 311=1714677309=PGE SQF0014.H0014-SCB SQF0014.H0014305=8463=FXXXXX307=NG Basis Futures Spr - PG&E-Citygate/Socal-Citygate - Q1 14542=20131231436=1.09013=0.00059014=2500.09017=25009022=909024=19025=Y916=20140101917=201403319201=12339200=159202=Q1 149300=59979301=PG&E - Citygate/Socal - Citygate9302=PG&E-Citygate/Socal-Citygate998=MMBtus9100=USD9101=USD / MMBtu9085=daily9083=49084=09061=4909062=NG Basis Futures Spr9063=Natural Gas Basis Futures Spread9032=1.259004=13430529005=1344660

Answer 1

We got around this by creating a custom component called SecDefICE with all the custom fields and redefining the SecurityDefinition message in a custom data dictionary.

We are now able to read the group information. Only drawback has been that we have had to use generic groups and grab fields by number rather than doing it in a typesafe manner. When I have time I will create a new namespace (FIXICE or the like) that will have the correct header files (Messages, Fields, custom cracker, etc)

Listed below is the definition from our dictionary.

New Component

  <component name='SecDefICE' required='N'>
   <field name='UnderlyingSymbol' required='N' />
   <field name='UnderlyingSecurityID' required='N' />
   <field name='UnderlyingSecurityIDSource' required='N' />
   <field name='UnderlyingCFICode' required='N' />
   <field name='UnderlyingSecurityDesc' required='N' />
   <field name='UnderlyingMaturityDate' required='N' />
   <field name='UnderlyingContractMultiplier' required='N' />
   <field name='IncrementPrice' required='N' />
   <field name='IncrementQty' required='N' />
   <field name='LotSize' required='N' />
   <field name='NumofCycles' required='N' />
   <field name='LotSizeMultiplier' required='N' />
   <field name='Clearable' required='N' />
   <field name='StartDate' required='N' />
   <field name='EndDate' required='N' />
   <field name='StripId' required='N' />
   <field name='StripType' required='N' />
   <field name='StripName' required='N' />
   <field name='HubId' required='N' />
   <field name='HubName' required='N' />
   <field name='HubAlias' required='N' />
   <field name='UnderlyingUnitOfMeasure' required='N' />
   <field name='PriceDenomination' required='N' />
   <field name='PriceUnit' required='N' />
   <field name='Granularity' required='N' />
   <field name='NumOfDecimalPrice' required='N' />
   <field name='NumOfDecimalQty' required='N' />
   <field name='ProductId' required='N' />
   <field name='ProductName' required='N' />
   <field name='ProductDescription' required='N' />
   <field name='TickValue' required='N' />
   <field name='ImpliedType' required='N' />
   <field name='PrimaryLegSymbol' required='N' />
   <field name='SecondaryLegSymbol' required='N' />
   <field name='IncrementStrike' required='N' />
   <field name='MinStrike' required='N' />
   <field name='MaxStrike' required='N' />
  </component> 

New message definition

  <message name='SecurityDefinition' msgcat='app' msgtype='d'>
   <field name='SecurityResponseID' required='Y' />
   <field name='SecurityResponseType' required='Y' />
   <field name='SecurityReqID' required='Y' />
   <field name='TotNoRelatedSym' required='N' />
   <field name='NoRpts' required='N' />
   <field name='ListSeqNo' required ='N' />
   <group name='NoUnderlyings' required='N'>
    <component name='SecDefICE' required='N' />
   </group>
   <field name='Text' required='N' />
    <!-- these are the original fields in 4.4 spec -->
    <!-- group name='NoUnderlyings' required='N' -->
    <!-- component name='UnderlyingInst' required='N' -->
    <!-- /group -->
    <!-- component name='Instrument' required='N' -->
    <!-- component name='InstrumentExtension' required='N' -->
    <!-- field name='Currency' required='N' -->
    <!-- field name='TradingSessionID' required='N' -->
    <!-- field name='TradingSessionSubID' required='N' -->
    <!-- field name='EncodedTextLen' required='N' -->
    <!-- field name='EncodedText' required='N' -->
    <!-- group name='NoLegs' required='N' -->
    <!-- component name='InstrumentLeg' required='N' -->
    <!-- /group -->
    <!-- field name='ExpirationCycle' required='N' -->
    <!-- field name='RoundLot' required='N' -->
    <!-- field name='MinTradeVol' required='N' -->
  </message>

Answer 2

If these tags aren't in your config file add them to it. I am assuming your data dictionary isn't being read.

UseDataDictionary=Y

ValidateFieldsOutOfOrder=N   /* This caused a lot of grief for me, 
because of the underlying data structure used in QuickFix, 
hence I always put it to N to save me the pain */

But I checked the specs for FIX4.4, this message does exist in the version, so why are you adding it ? This should be there in the default xml config file available with QuickFix.

Your first group is being read properly it seems, but the second group(two 305 tags) is causing the error. Quickfix cannot figure out how many and what groups are there. And you are unnecessarily replicating information. You use a group like this, in your message. And Quickfix isn't able to parse your message definition properly. This is how QuickFix works so you cannot do anything about it.

 <group name="NoUnderlyings" required="N">
   <component name="UnderlyingInstrument" required="N" /> 
 </group>

Else you would need to replicate the UnderlyingInstrument at multiple places, if it is used in multiple messages, and it would increase your XML size without any benefits. This is how it is done in the default Quickfix config file, try following it and it would save you a lot of grief.