Would like to upload JSON format datafile into Hyperledger Composer Asset Registry

Question

For my development / testing purposes I have configured a separate Transaction (using Online Playground) to upload few Master records for the defined Participants. I would next like to build in a feature to demonstrate that these records can in fact be extracted from legacy application and imported into the Assets/Participants Registry.

I have already converted extracted (.csv format) datafile to .json format file using one of the converters available online.

I have been advised that REST APIs be leveraged for building this code. Can someone direct me to relevant documentation, sample piece of code which reads a few hundred (JSON formatted) records and posts them to the Participants/Assets Registry?

Thanks!

Answer 1

Sure, in fact I've just done this for a Blockchain tutorial I'm writing. The inbound data is structured to identify participants and assets. In my example, there are 5 types of Participants and 1 type of Asset. The code will handle multiple types of assets. The inbound file is a text file using JSON notation:

{ "members":
    [
        {"type": "Seller", "pw": "bob", "companyName": "PC Hardware, Inc", "id": "bob@pchardwareinc.com"},
        {"type": "Seller", "pw": "jojo", "companyName": "Inovative Solutions, Inc", "id": "jojo@innovativesolutionsinc.com"},
        {"type": "Seller", "pw": "mary", "companyName": "The i-Series Experts, Inc", "id": "mary@iseriesexpertsinc.com"},
        {"type": "Seller", "pw": "wilber", "companyName": "Cooling Systems R Us, Inc", "id": "wilber@coolingsystemsrusinc.com"},
        {"type": "Seller", "pw": "erin", "companyName": "2nd Life Systems, Inc", "id": "Erin@2ndlifesystemsinc.com"},
        {"type": "Seller", "pw": "abdul", "companyName": "The Cognitive Advantage, Inc", "id": "abdul@cognitiveadvantageinc.com"},
        {"type": "Seller", "pw": "aiesha", "companyName": "Cloud Nine Software, Inc", "id": "aiesha@cloudninesoftwareinc.com"},
        {"type": "Seller", "pw": "joshua", "companyName": "Office Experts, Inc", "id": "joshua@officeexpertsinc.com"},
        {"type": "Seller", "pw": "alwayson", "companyName": "NonStop, Inc", "id": "alwayson@nonstopinc.com"},
        {"type": "Seller", "pw": "anytime", "companyName": "Virtual Paper, Inc", "id": "anytime@virtualpaperinc.com"},
        {"type": "Buyer", "pw": "yes", "companyName": "Software Solutions, Inc", "id": "yes@softwaresolutionsinc.com"},
        {"type": "Buyer", "pw": "buyer1", "companyName": "Hybrid Cloud Designs, Inc", "id": "buyer1@hybridclouddesignsinc.com"},
        {"type": "Buyer", "pw": "eric", "companyName": "Born On The Cloud, Inc", "id": "eric@bornonthecloudinc.com"},
        {"type": "Buyer", "pw": "abby", "companyName": "Kid Friendly Learning, Inc", "id": "abby@kidfriendlyinc.com"},
        {"type": "Buyer", "pw": "kristen", "companyName": "The Education Game, Inc", "id": "kristen@gameify.com"},
        {"type": "Shipper", "pw": "speedmatters", "companyName": "Fast Eddy, Inc", "id": "speedmatters@fasteddyinc.com"},
        {"type": "Shipper", "pw": "wantitnow", "companyName": "The Overnight Experts, Inc", "id": "wantitnow@overnightexpressinc.com"},
        {"type": "Provider", "pw": "info", "companyName": "PC Hardware Now, Inc", "id": "info@pchardwarenowinc.com"},
        {"type": "Provider", "pw": "power", "companyName": "UPS Systems, Inc", "id": "power@upssystemsinc.com"},
        {"type": "Provider", "pw": "online", "companyName": "The App Store, Inc", "id": "online@theappstoreinc.com"},
        {"type": "FinanceCo", "pw": "easymoney", "companyName": "The Global Financier", "id": "easymoney@easymoneyinc.com"}
    ],
    "items":
    [
        {"itemNo": 1, "itemDescription": "Macbook Pro 16Gb, 1Tb", "quantity": 2000, "unitPrice": 1285},
        {"itemNo": 2, "itemDescription": "Macbook Pro 8Gb, .5Tb", "quantity": 2000, "unitPrice": 985},
        {"itemNo": 3, "itemDescription": "Lenovo Thinkpad W520 16Gb, .25Tb", "quantity": 1000, "unitPrice": 500},
        {"itemNo": 4, "itemDescription": "Lenovo Thinkpad W520 32Gb, 1Tb", "quantity": 4000, "unitPrice": 1565},
        {"itemNo": 5, "itemDescription": "4K Monitor - 25", "quantity": 4000, "unitPrice": 265},
        {"itemNo": 6, "itemDescription": "4K Monitor - 32", "quantity": 4000, "unitPrice": 365},
        {"itemNo": 7, "itemDescription": "4K Monitor - 37", "quantity": 4000, "unitPrice": 465},
        {"itemNo": 8, "itemDescription": "4K Monitor - 42", "quantity": 4000, "unitPrice": 565}
    ],
    "assets":
    [   
        {"type": "Order", "id": "001", "buyer": "yes@softwaresolutionsinc.com", "seller": "bob@pchardwareinc.com", "items": [{"itemNo": 1, "quantity": 5}, {"itemNo": 2, "quantity": 2}, {"itemNo": 6, "quantity": 2}] },
        {"type": "Order", "id": "002", "buyer": "yes@softwaresolutionsinc.com", "seller": "jojo@innovativesolutionsinc.com", "items": [{"itemNo": 2, "quantity": 5}, {"itemNo": 8, "quantity": 2}] },
        {"type": "Order", "id": "003", "buyer": "yes@softwaresolutionsinc.com", "seller": "Erin@2ndlifesystemsinc.com", "items": [{"itemNo": 3, "quantity": 15}, {"itemNo": 4, "quantity": 20}] },
        {"type": "Order", "id": "004", "buyer": "yes@softwaresolutionsinc.com", "seller": "aiesha@cloudninesoftwareinc.com", "items": [{"itemNo": 1, "quantity": 5}, {"itemNo": 2, "quantity": 2}, {"itemNo": 6, "quantity": 2}] },
        {"type": "Order", "id": "005", "buyer": "yes@softwaresolutionsinc.com", "seller": "joshua@officeexpertsinc.com", "items": [{"itemNo": 2, "quantity": 5}, {"itemNo": 8, "quantity": 2}] },
        {"type": "Order", "id": "006", "buyer": "eric@bornonthecloudinc.com", "seller": "Erin@2ndlifesystemsinc.com", "items": [{"itemNo": 3, "quantity": 15}, {"itemNo": 4, "quantity": 20}] },
        {"type": "Order", "id": "007", "buyer": "eric@bornonthecloudinc.com", "seller": "bob@pchardwareinc.com", "items": [{"itemNo": 1, "quantity": 5}, {"itemNo": 2, "quantity": 2}, {"itemNo": 6, "quantity": 2}] },
        {"type": "Order", "id": "008", "buyer": "eric@bornonthecloudinc.com", "seller": "jojo@innovativesolutionsinc.com", "items": [{"itemNo": 2, "quantity": 5}, {"itemNo": 8, "quantity": 2}] },
        {"type": "Order", "id": "009", "buyer": "eric@bornonthecloudinc.com", "seller": "Erin@2ndlifesystemsinc.com", "items": [{"itemNo": 3, "quantity": 15}, {"itemNo": 4, "quantity": 20}] }
    ]
}

I then open an admin connection and iterate through the members (participants) and assets. I've included the first part of the code segment that iterates through the members and adds them to the appropriate registry and the creates an identity for them. The code for doing this for the assets is similar. You'll note that in the identity process, the member id's and secrets are written into a local file. This is an awful idea for a production system, but makes the tutorial simpler to manage. You'd obviously want to deal with identity creation in a different manner.

exports.autoLoad = function(req, res, next) {
    // get the autoload file
    let newFile = path.join(path.dirname(require.main.filename),'startup','memberList.json');
    let startupFile = JSON.parse(fs.readFileSync(newFile));
    // connect to the network
    let businessNetworkConnection;
    let factory;
    let adminConnection = new AdminConnection();
        adminConnection.connect(config.composer.connectionProfile, config.composer.adminID, config.composer.adminPW)
        .then(() => {
            businessNetworkConnection = new BusinessNetworkConnection();
            return businessNetworkConnection.connect(config.composer.connectionProfile, network, adminID, adminPW)
            .then(() => {
                factory = businessNetworkConnection.getBusinessNetwork().getFactory();
                for (let each in startupFile.members)
                    {(function(_idx, _arr)
                        {
                            return businessNetworkConnection.getParticipantRegistry(NS+'.'+_arr[_idx].type)
                            .then(function(participantRegistry){
                                return participantRegistry.get(_arr[_idx].id)
                                .then((_res) => {console.log('['+_idx+'] member with id: '+_arr[_idx].id+' already exists in Registry '+NS+'.'+_arr[_idx].type)})
                                .catch((error) => {
                                    let participant = factory.newResource(NS, _arr[_idx].type, _arr[_idx].id);
                                    participant.companyName = _arr[_idx].companyName;
                                    participantRegistry.add(participant)
                                    .then(() => {
                                        console.log('['+_idx+'] '+_arr[_idx].companyName+" successfully added");
                                    })
                                    .then(() => {
                                        return businessNetworkConnection.issueIdentity(NS+'.'+_arr[_idx].type+'#'+_arr[_idx].id, _arr[_idx].pw)
                                            .then((result) => {
                                                let _mem = _arr[_idx];
                                                _mem.secret = result.userSecret;
                                                _mem.userID = result.userID;
                                                memberTable.push(_mem);
                                                svc.saveMemberTable(memberTable);
                                            })
                                            .catch((error) => {
                                                console.error('create id for '+_arr[_idx].id+'failed.',error.message);
                                            });
                                        })
                                    .catch((error) => {console.log(_arr[_idx].companyName+" add failed",error);});
                                    });
                                })
                            .catch((error) => {console.log('error with getParticipantRegistry', error)});
                        })(each, startupFile.members)
                    }

All of the code referenced here is in Chapter 5 of the ZeroToBlockchain tutorial and git repo. The tutorial is still under development, but is complete through this section and is available here: https://github.com/rddill-IBM/ZeroToBlockchain

Answer 2

The JSON has to conform to the format expected by Composer - specifically it must have $class attributes to indicate the type information.

If your data is in the expected format then you can create the assets by doing an HTTP POST to the composer-rest-server (for example, using bash and curl), or you could use the Node.js SDK to create a command line application that reads the JSON files from disk and then creates the assets using the asset registry APIs.