书接上回，之前利用fabric-go-sdk的项目进行修改，成功创建起了多组织多对等节点的Fabric网络，但这远远不能满足正常开发使用的需求。本文将基于之前的工作内容，逐步实现多通道多排序节点的Fabric网络，并对创建以及踩坑过程做一个简单记录。

多通道搭建

组织结构

在之前已经搭建出3个组织，每个组织有2个peer节点的Fabric网络的基础上，多通道的方案计划使用3条通道进行划分，其中Org1的peer0、peer1和Org3的peer0加入Channel1，Org2的peer0、peer1和Org3的peer1加入Channel2，Org3的peer0、peer1加入Channel3，具体示意图如下

多通道示意图

配置文件修改

fixtures

网络结构发生了变化，只需对生成通道以及相关锚节点的配置文件configtx.yaml进行修改。对于crypto-config.yaml和docker-compose.yaml这两个文件与通道的配置没有关系，故不用作任何改动。

在configtx.yaml中只需要按照网络结构修改Profiles选项即可，这里直接贴出修改后的相关配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
Profiles:
    OrdererGenesis:
        <<: *ChannelDefaults
        Orderer:
            <<: *OrdererDefaults
            Organizations:
                - *OrdererOrg
            Capabilities:
                <<: *OrdererCapabilities
        Consortiums:
            SampleConsortium:
                Organizations:
                    - *Org1
                    - *Org2
                    - *Org3
    ChannelOne:
        Consortium: SampleConsortium
        <<: *ChannelDefaults
        Application:
            <<: *ApplicationDefaults
            Organizations:
                - *Org1
                - *Org3
            Capabilities:
                <<: *ApplicationCapabilities
    ChannelTwo:
        Consortium: SampleConsortium
        <<: *ChannelDefaults
        Application:
            <<: *ApplicationDefaults
            Organizations:
                - *Org2
                - *Org3
            Capabilities:
                <<: *ApplicationCapabilities
    ChannelThree:
        Consortium: SampleConsortium
        <<: *ChannelDefaults
        Application:
            <<: *ApplicationDefaults
            Organizations:
                - *Org3
            Capabilities:
                <<: *ApplicationCapabilities

修改完成后需要重新生成通道文件和对应的锚节点文件，直接修改脚本文件（脚本相关操作可以参考之前的介绍）。修改后的脚本会生成三个通道文件和五个锚节点文件，其中三个通道对应的channelID分别为mychannel1、mychannel2和mychannel3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# create.sh
rm -rf crypto-config channel-artifacts && mkdir crypto-config
../../fabric-samples/bin/cryptogen generate --config=crypto-config.yaml
../../fabric-samples/bin/configtxgen -profile OrdererGenesis -outputBlock ./channel-artifacts/genesis.block -channelID fabric-channel
../../fabric-samples/bin/configtxgen -profile ChannelOne -outputCreateChannelTx ./channel-artifacts/channel1.tx -channelID mychannel1
../../fabric-samples/bin/configtxgen -profile ChannelTwo -outputCreateChannelTx ./channel-artifacts/channel2.tx -channelID mychannel2
../../fabric-samples/bin/configtxgen -profile ChannelThree -outputCreateChannelTx ./channel-artifacts/channel3.tx -channelID mychannel3

../../fabric-samples/bin/configtxgen -profile ChannelOne -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors_ChannelOne.tx -channelID mychannel1 -asOrg Org1MSP
../../fabric-samples/bin/configtxgen -profile ChannelOne -outputAnchorPeersUpdate ./channel-artifacts/Org3MSPanchors_ChannelOne.tx -channelID mychannel1 -asOrg Org3MSP
../../fabric-samples/bin/configtxgen -profile ChannelTwo -outputAnchorPeersUpdate ./channel-artifacts/Org2MSPanchors_ChannelTwo.tx -channelID mychannel2 -asOrg Org2MSP
../../fabric-samples/bin/configtxgen -profile ChannelTwo -outputAnchorPeersUpdate ./channel-artifacts/Org3MSPanchors_ChannelTwo.tx -channelID mychannel2 -asOrg Org3MSP
../../fabric-samples/bin/configtxgen -profile ChannelThree -outputAnchorPeersUpdate ./channel-artifacts/Org3MSPanchors_ChannelThree.tx -channelID mychannel3 -asOrg Org3MSP
./ccp-generate.sh

config.yaml和main.go

回到项目根目录，main.go和config.yaml就是实现通道创建、安装链码、执行智能合约的客户端代码及其配置文件，每个通道由一个go文件和yaml配置文件来控制操作，所以实现三个通道的测试需要有3份go代码和yaml配置文件，对应通道123分别将go文件命名为main1.go、main2.go、main3.go，yaml文件命名为config1.yaml、config2.yaml、config3.yaml，这三套文件的起始状态都是原先main.go和config.yaml的副本

config.yaml

**client:**这里我将所有的client组织都改为org3，即将原先的org1整体替换为org3

Channels: config1.yaml对应mychannel1，相应的peers节点分别为peer0.org1.example.com、peer1.org1.example.com、peer0.org3.example.com，另外两个配置文件中的对应部分以此类推

Organizations: 只需要保留通道1所对应的Org1和Org3两个组织，并且Org3中的节点只保留peer0，其他配置文件以此类推

Peers: 同上，只保留通道1中所加入的3各节点的信息

entityMatchers: 只保留org1和org3的相关信息

至此config1.yaml就修改完成了，别忘了参照上述步骤继续修改2和3的配置文件

main.go

以下步骤没有特别说明均指对main1.go的操作，其他go文件以此类推

orgs： 只保留对应通道的组织，并更改组织锚节点文件的文件名(OrgAnchorFile)和加入对应通道的节点数量(OrgPeerNum) ，比如main1.go中的Org3改为

1
2
3
4
5
6
7
8
{
		OrgAdminUser:  "Admin",
		OrgName:       "Org3",
		OrgMspId:      "Org3MSP",
		OrgUser:       "User1",
		OrgPeerNum:    1,
		OrgAnchorFile: "/home/baymrx/go/src/fabric-go-fabcar/fixtures/channel-artifacts/Org3MSPanchors_ChannelOne.tx",
	},

info： 修改ChannelID和ChannelConfig为对应的通道号（以main1.go为例）

1
2
ChannelID:        "mychannel1",
ChannelConfig:    "/home/baymrx/go/src/fabric-go-fabcar/fixtures/channel-artifacts/channel1.tx",

sdk： 修改sdk, err := sdkInit.Setup("config.yaml", &info)配置文件的文件名，即sdk, err := sdkInit.Setup("config1.yaml", &info)

ccpPath： 全部修改为org3的连接配置文件路径，由于默认生成的ccp文件使用的都是peer0，而在channel2中并没有org3的peer0，所以可以拷贝一份connection-org3.yaml的副本，并修改其中的节点名和对应端口供main2.go使用

network： network, err := gw.GetNetwork("mychannel")改为network, err := gw.GetNetwork("mychannel1")

populateWallet()函数： 将函数中所有的org1都改为 org3

报错踩坑记录

以下列出的报错及解决方案只作为我自己在操作时的踩坑记录，如遇同样报错，不保证解决方案对所有类似问题通用

>> 开始创建通道…
>> Create channel and join error: Create channel error: error should be nil for SaveChannel of orgchannel: create channel failed: create channel failed: SendEnvelope failed: calling orderer ‘orderer.example.com:7050’ failed: Orderer Server Status Code: (400) BAD_REQUEST. Description: Failing initial channel config creation: mismatched channel IDs: ‘mychannel1’ != ‘channel1’

这里是由于对channelID和channelName没做好区分造成的，只需注意配置文件和代码中用到的通道名都是channelID就行了（如mychannel1），只有在文件路径中才会出现channel*（如channel1.tx）

>>>> 使用每个org的管理员身份更新锚节点配置…
>> Create channel and join error: Create channel error: SaveChannel for anchor org Org3 error: create channel failed: create channel failed: SendEnvelope failed: calling orderer ‘orderer.example.com:7050’ failed: Orderer Server Status Code: (400) BAD_REQUEST. Description: error applying config update to existing channel ‘mychannel1’: error authorizing update: ConfigUpdate for channel ‘mychannel3’ but envelope for channel ‘mychannel1’

这里是由于把所有的通道配置都放在了一个config.yaml中，将其他不必要的配置删掉即可

>> 开始安装链码…
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
fabric-go-fabcar/sdkInit.installCC({0xc000132180?, 0xc00012e008?}, {0xc0011d0000?, 0x1?, 0x1?}, {0xc0001fbaa0, 0x2, 0x1?})
/home/baymrx/go/src/fabric-go-fabcar/sdkInit/sdkSetting.go:216 +0x305
fabric-go-fabcar/sdkInit.CreateCCLifecycle(0xc000198840, 0xc00022dd10?, 0x40?, 0x1271920?)
/home/baymrx/go/src/fabric-go-fabcar/sdkInit/sdkSetting.go:136 +0x233
main.main()
/home/baymrx/go/src/fabric-go-fabcar/main1.go:76 +0x36a
exit status 2

报错问题在于数组出界，跟踪一下代码可以发现问题出在main.go中的orgs数组变量中，由于Org3只有一个peer节点加入Channel1，而数组中对应Org3的OrgPeerNum设置为2，造成运行的时候数组越界，数量改成1就好了

>> 组织认可智能合约定义…
>>> chaincode approved by Org1 peers:
peer0.org1.example.com:7051
peer1.org1.example.com:8051
>>> chaincode approved by Org3 peers:
peer0.org3.example.com:11051
[fabsdk/util] 2022/08/11 07:40:17 UTC - lazyref.(*Reference).refreshValue -> WARN Error - initializer returned error: QueryBlockConfig failed: QueryBlockConfig failed: target(s) required. Will retry again later
>> create chaincode lifecycle error: %v queryApprovedCC error: Org Org3 Peer peer0.org3.example.com:11051 NewInvoker error: Test status Code: (12) UNKNOWN. Description: LifecycleQueryApprovedCC returned error: querying for installed chaincode failed: Transaction processing for endorser [peer0.org3.example.com:11051]: Chaincode status Code: (500) UNKNOWN. Description: failed to invoke backing implementation of ‘QueryApprovedChaincodeDefinition’: could not fetch approved chaincode definition (name: ‘fabcar’, sequence: ‘1’) on channel ‘mychannel1’

exit status 255

>> 组织认可智能合约定义…
>>> chaincode approved by Org3 peers:
peer0.org3.example.com:11051
peer1.org3.example.com:12051
[fabsdk/util] 2022/08/11 07:44:48 UTC - lazyref.(*Reference).refreshValue -> WARN Error - initializer returned error: QueryBlockConfig failed: no channel peers configured for channel [mychannel3]. Will retry again later
>> create chaincode lifecycle error: %v queryApprovedCC error: Org Org3 Peer peer0.org3.example.com:11051 NewInvoker error: Test status Code: (12) UNKNOWN. Description: LifecycleQueryApprovedCC returned error: querying for installed chaincode failed: Transaction processing for endorser [peer0.org3.example.com:11051]: Chaincode status Code: (500) UNKNOWN. Description: failed to invoke backing implementation of ‘QueryApprovedChaincodeDefinition’: could not fetch approved chaincode definition (name: ‘fabcar’, sequence: ‘1’) on channel ‘mychannel3’
exit status 255

这两个错误都是搞混了channelID与channel名造成的，在config.yaml中channels:下面的选项错误地使用了channel1，改成mychannel1报错消失

>> 设置链码状态完成
Failed to get network: Failed to create new channel client: event service creation failed: could not get chConfig cache reference: QueryBlockConfig failed: QueryBlockConfig failed: queryChaincode failed: Transaction processing for endorser [localhost:7051]: Chaincode status Code: (500) UNKNOWN. Description: channel ‘mychannel’ not found
exit status 1

还有未更改的channelID，检索全文，在main.go中有一行代码

1
network, err := gw.GetNetwork("mychannel")

将mychannel改为mychannel1之后解决问题

1
network, err := gw.GetNetwork("mychannel1")

>> 设置链码状态完成
Failed to get network: Failed to create new channel client: event service creation failed: could not get chConfig cache reference: QueryBlockConfig failed: QueryBlockConfig failed: queryChaincode failed: Transaction processing for endorser [localhost:7051]: Chaincode status Code: (500) UNKNOWN. Description: channel ‘mychannel3’ not found
exit status 1

报错是在执行main3.go，报错信息分析一下，localhost:7051对应的节点找不到mychannel3，7051端口在配置中对应的是Org1的peer0节点，而通道3中只有Org3的peer节点，所以找不到是必然。排查config.yaml，其中的ccpPath路径没有修改，连接配置依然是使用的org1，将其改为org3的路径。同样，将populateWallet()函数中的组织也都改为org3，问题解决

>> 设置链码状态完成
Failed to get network: Failed to create new channel client: event service creation failed: could not get chConfig cache reference: QueryBlockConfig failed: QueryBlockConfig failed: queryChaincode failed: Transaction processing for endorser [peer0.org3.example:11051]: Endorser Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [peer0.org3.example:11051]: connection is in TRANSIENT_FAILURE
exit status 1

重新生成证书及公私钥文件后，没有清除钱包文件夹，使用rm -rf wallet keystore删除之前的钱包文件即可

多排序节点搭建

多排序节点方案计划使用三个orderer节点和EtcdRaft排序共识算法进行搭建，对应的三个orderer节点的地址及端口号如下

- orderer0.example.com:7050

- orderer1.example.com:8050

- orderer2.example.com:9050

配置文件修改

configtx.yaml

Organizations选项：

需要修改OrdererOrg中的OrdererEndpoints

1
2
3
4
OrdererEndpoints:
    - orderer0.example.com:7050
    - orderer1.example.com:8050
    - orderer2.example.com:9050
Orderer选项：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Orderer Type: The orderer implementation to start
# Available types are "solo", "kafka" and "etcdraft".
OrdererType: etcdraft
Addresses:
    - orderer0.example.com:7050
    - orderer1.example.com:8050
    - orderer2.example.com:9050
EtcdRaft:
    Consenters:
        - Host: orderer0.example.com
          Port: 7050
          ClientTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer0.example.com/tls/server.crt
          ServerTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer0.example.com/tls/server.crt
        - Host: orderer1.example.com
          Port: 8050
          ClientTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/server.crt
          ServerTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/server.crt
        - Host: orderer2.example.com
          Port: 9050
          ClientTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer2.example.com/tls/server.crt
          ServerTLSCert: crypto-config/ordererOrganizations/example.com/orderers/orderer2.example.com/tls/server.crt

crypto-config.yaml

OrdererOrgs选项：
1
2
3
4
5
6
7
8
OrdererOrgs:
  - Name: Orderer
    Domain: example.com
    EnableNodeOUs: true
    Specs:
      - Hostname: orderer0
      - Hostname: orderer1
      - Hostname: orderer2

docker-compose.yaml

在volumes中添加新的orderer节点
1
2
3
4
5
6
7
8
9
10
volumes:
  orderer0.example.com:
  orderer1.example.com:
  orderer2.example.com:
  peer0.org1.example.com:
  peer0.org2.example.com:
  peer1.org1.example.com:
  peer1.org2.example.com:
  peer0.org3.example.com:
  peer1.org3.example.com:
service中同样添加新节点配置

添加后记得修改节点端口和文件路径的映射，以orderer1为例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
orderer1.example.com:
  container_name: orderer1.example.com
  image: hyperledger/fabric-orderer:latest
  environment:
    - FABRIC_LOGGING_SPEC=DEBUG
    - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
    - ORDERER_GENERAL_LISTENPORT=8050
    - ORDERER_GENERAL_GENESISMETHOD=file
    - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
    - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
    - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
    # enabled TLS
    - ORDERER_GENERAL_TLS_ENABLED=true
    - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
    - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
    - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
    # - ORDERER_KAFKA_TOPIC_REPLICATIONFACTOR=1
    # - ORDERER_KAFKA_VERBOSE=true
    - ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/var/hyperledger/orderer/tls/server.crt
    - ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/var/hyperledger/orderer/tls/server.key
    - ORDERER_GENERAL_CLUSTER_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
  working_dir: /opt/gopath/src/github.com/hyperledger/fabric
  command: orderer
  volumes:
      - ./channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
      - ./crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/msp:/var/hyperledger/orderer/msp
      - ./crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/:/var/hyperledger/orderer/tls
      - orderer1.example.com:/var/hyperledger/production/orderer
  ports:
    - 8050:8050
  networks:
    - test

config.yaml

orderers选项

三个config文件中将3个排序节点的配置文件都添加上

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
orderers:
  orderer0.example.com:
    url: orderer0.example.com:7050
    grpcOptions:
      ssl-target-name-override: orderer0.example.com
      keep-alive-time: 0s
      keep-alive-timeout: 20s
      keep-alive-permit: false
      fail-fast: false
      allow-insecure: false
    tlsCACerts:
      path: /home/baymrx/go/src/fabric-go-fabcar/fixtures/crypto-config/ordererOrganizations/example.com/tlsca/tlsca.example.com-cert.pem
  orderer1.example.com:
    url: orderer1.example.com:8050
    grpcOptions:
      ssl-target-name-override: orderer1.example.com
      keep-alive-time: 0s
      keep-alive-timeout: 20s
      keep-alive-permit: false
      fail-fast: false
      allow-insecure: false
    tlsCACerts:
      path: /home/baymrx/go/src/fabric-go-fabcar/fixtures/crypto-config/ordererOrganizations/example.com/tlsca/tlsca.example.com-cert.pem
  orderer2.example.com:
    url: orderer2.example.com:9050
    grpcOptions:
      ssl-target-name-override: orderer2.example.com
client parameters are disabled
      keep-alive-time: 0s
      keep-alive-timeout: 20s
      keep-alive-permit: false
      fail-fast: false
      allow-insecure: false
    tlsCACerts:
      path: /home/baymrx/go/src/fabric-go-fabcar/fixtures/crypto-config/ordererOrganizations/example.com/tlsca/tlsca.example.com-cert.pem

main.go

info参数

修改其中的OrdererEndpoint为orderer0.example.com

sdkInit/sdkSetting.go

与上一条相对应，在sdkInit/sdkSetting.go中有几处函数参数原先使用的是固定字符串，需要对应修改为变量

将代码中的resmgmt.WithOrdererEndpoint("orderer.example.com")改为resmgmt.WithOrdererEndpoint(info.OrdererEndpoint)（直接搜索全部替换）

报错踩坑记录

>> 开始创建通道…
>> Create channel and join error: Create channel error: error should be nil for SaveChannel of orgchannel: create channel failed: create channel failed: SendEnvelope failed: calling orderer ‘orderer0.example.com:7050’ failed: Orderer Client Status Code: (2) CONNECTION_FAILED. Description: dialing connection on target [orderer0.example.com:7050]: connection is in TRANSIENT_FAILURE

>> exit status 255

报错原因是之前误将crypto-config.yaml中的OrdererOrgs改成了下面的配置，按照上面所使用的正确配置即可解决

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
OrdererOrgs:
  - Name: Orderer0
    Domain: example.com
    EnableNodeOUs: true
    Specs:
      - Hostname: orderer0
  - Name: Orderer1
    Domain: example.com
    EnableNodeOUs: true
    Specs:
      - Hostname: orderer1
  - Name: Ordere2
    Domain: example.com
    EnableNodeOUs: true
    Specs:
      - Hostname: orderer2

源码

最后把整个项目源码附上，下载链接

为了减少压缩包大小，项目中移除了Tape性能测试工具，需要时自行配置。代码使用时需要将所有文件中与描述文件路径相关的源码全部替换为自己的工作目录，并且fabric-samples需要与本项目在统一根目录下。

本文主要资料参考：

Fantasy大佬的博客 (Github)FabricLearn-实现一个工作室联盟链

hyperledger-fabric生产网络生成身份文件-yunqihao

触不可及` 大佬的博客

进阶篇（一）Fabric 2.0 手动生成CA证书搭建Fabric网络-Raft协议-多orderer节点