CODEWITHSUNDEEP
hyperledger-fabric
aldred
aldred

Reputation: 853

New RAFT orderer cannot detect that it belongs to an application channel

I have an initial Hyperledger Fabric network as the following:

initial network

org1.example.com has 2 RAFT nodes. The system channel name is system-channel (there is only 1 consortium defined). The application channel name is channel1 in which the peer from org1.example.com runs chaincode1. Note that there is no separate orderer organization.

The system channel profile is:

    OrdererGenesis:
        <<: *ChannelDefaults
        Orderer:
            <<: *OrdererDefaults
            Organizations:
                - *Org1
        Consortiums:
            SampleConsortium:
                Organizations:
                    - *Org1

I want to add another organization, org2.example.com so that the final network will be:

Final network

Steps that I took for system-channel (I followed https://hyperledger-fabric.readthedocs.io/en/release-1.4/raft_configuration.html#reconfiguration):

  1. Add Org2MSP definition to Orderer Channel Group in system-channel
  2. Add Org2MSP definition to Consortium Channel Group in system-channel
  3. Add Org2's orderer information (tls certs, etc) to the consenters list i.e. channel_group.groups.Orderer.values.ConsensusType.value.metadata.consenters in system-channel
  4. Start Org2's orderer i.e. orderer0.org2.example.com
  5. Add orderer0.org2.example.com to the list of orderer addresses in system-channel i.e. .channel_group.values.OrdererAddresses.value.addresses

All the above steps work well and orderer0.org2.example.com serves system-channel

Moving on, the steps that I took for channel1:

  1. Add Org2MSP definition to Orderer Channel Group in channel1
  2. Add Org2MSP definition to Application Channel Group in channel1
  3. Add orderer0.org2.example.com to the list of orderer addresses in channel1 i.e. .channel_group.values.OrdererAddresses.value.addresses
  4. Add orderer0.org2.example.com's information (tls certs, etc) to the consenters list i.e. channel_group.groups.Orderer.values.ConsensusType.value.metadata.consenters in channel1

Up to step number 3, everything is OK. Once I completed step 4, I begin to see channel1 does not exist errors the following in the newly added orderer orderer0.org2.example.com:

2019-05-09 09:38:03.360 UTC [comm.grpc.server] 1 -> INFO 05a streaming call completed grpc.service=orderer.Cluster grpc.method=Step grpc.peer_address=192.168.224.6:43116 grpc.peer_subject="CN=orderer0.org1.example.com,OU=peer+OU=org1,O=org1.example.com,L=Singapore,ST=Singapore,C=SG" error="channel channel1 doesn't exist" grpc.code=Unknown grpc.call_duration=711.5µs

In the current RAFT leader, there are error messages as well (3 refers to orderer0.org2.example.com:

2019-05-09 09:38:02.859 UTC [orderer.consensus.etcdraft] logSendFailure -> ERRO 0c5 Failed to send StepRequest to 3, because: aborted channel=channel1 node=1

It seems that orderer0.org2.example.com is not aware that it is supposed to serve channel1. I also cannot see channel1 folder in /var/hyperledger/production/orderer/chains in orderer0.org2.example.com

As part of my troubleshooting, I tried to persist all the orderers' /var/hyperledger/production/orderer folder which contains the chain. shut down orderer0.org1.example.com and orderer0.org2.example.com and copy the channel1 folder from orderer0.org1.example.com to orderer0.org2.example.com and finally start both orderers.

Copy ledger manually

Now orderer0.org2.example.com knows that it needs to serve channel1 as evidenced in the logs

2019-05-10 02:36:04.161 UTC [orderer.consensus.etcdraft] apply -> INFO 044 Applied config change to add node 1, current nodes in channel: [1] channel=channel1 node=3
2019-05-10 02:36:04.161 UTC [orderer.consensus.etcdraft] apply -> INFO 045 Applied config change to add node 2, current nodes in channel: [1 2] channel=channel1 node=3
2019-05-10 02:36:04.161 UTC [orderer.consensus.etcdraft] writeBlock -> INFO 046 Got block [6], expect block [7], this node was forced to catch up channel=channel1 node=3
2019-05-10 02:36:04.161 UTC [orderer.consensus.etcdraft] apply -> INFO 047 Applied config change to add node 3, current nodes in channel: [1 2 3] channel=channel1 node=3

Based on above, it is obvious orderer0.org2.example,com has no way to receive channel1's block to start serving the channel. For peers, peer can receive the block and issue peer channel join block_name.block but orderers can't do that. I am wondering what step am I missing.

To simulate the environment and reproduce the issue, refer to: https://github.com/aldredb/bring-your-own-orderer

Upvotes: 2

Views: 2222

Answers (1)

aldred
aldred

Reputation: 853

This is resolved. The issue is that I didn't use the latest config block as the bootstrap block in the new orderer, orderer0.org2.example.com. Once I did the aforementioned, orderer0.org2.example.com is able to detect channel1:

2019-05-10 14:06:04.778 UTC [orderer.common.server] replicateDisabledChains -> INFO 072 Successfully replicated 0 chains: []
2019-05-10 14:06:44.680 UTC [orderer.common.server] replicateDisabledChains -> INFO 073 Found 1 inactive chains: [channel1]
2019-05-10 14:06:44.689 UTC [orderer.common.cluster] ReplicateChains -> INFO 074 Will now replicate chains [channel1]

Upvotes: 4

Related Questions