CODEWITHSUNDEEP
javascriptnode.jsexpressmulter
Mile Tusevljak
Mile Tusevljak

Reputation: 13

parse multipart JSON with Multer

I want to consume multipart JSON data sent from a IoT camera via POST. Express Body-parser does not support multi-part data.

The data can be sent with, or without a image attached. Content-Disposition: form-data; name="event"; is the JSON data and Content-Disposition: form-data; name="image"; appears to be the image. The name of the image changes with each event.

The data sent looks like:

POST { 
host: '192.168.1.1:3000',
  accept: '*/*',
  'content-length': '1178',
  'content-type': 'multipart/form-data; boundary=------------------------f519f81dc2e1d562'
}
--------------------------f519f81dc2e1d562
Content-Disposition: form-data; name="event"; filename="20200921204027_795989event_3913.json"
Content-Type: application/octet-stream

{"packetCounter":"3913",
"capture_timestamp":"1600677267347",
"frame_timestamp":"0",
"capture_ts":"1600677267347000000",
"plateUTF8":"RAR463",
"plateASCII":"RAR463",
 ......
"GEOtarget":"Camera",
"imagesURI":["/local/fflprapp/tools.cgi?action=getImage&name=14/20200921203427_524035lp_ABC123_3912.jpg","/local/fflprapp/tools.cgi?action=getImage&name=15/20200921203427_525568roi_ABC123_3912.jpg"],
"sensorProviderID":"testID"
}
-------------------------f519f81dc2e1d562--

My attempt at consuming the JSON data with multer ( I am not worried yet about the image and in the above request the image part is turned off) is:

var express = require('express')
var multer = require('multer')
var upload = multer()
var app = express()

app.post('/', (req, res) => {
console.log('hello');
res.send(200);
});

app.listen(3000, () => { console.log('server running')});

When a request comes in the error is:

dal@dal-ubuntu20lts-vm:~/test$ node index.js
server running
MulterError: Unexpected field
at wrappedFileFilter (/home/al/test/node_modules/multer/index.js:40:19)
at Busboy. (/home/al/test/node_modules/multer/lib/make-middleware.js:114:7)
at Busboy.emit (events.js:198:13)
at Busboy.emit (/home/al/test/node_modules/busboy/lib/main.js:38:33)
at PartStream. (/home/al/test/node_modules/busboy/lib/types/multipart.js:213:13)
at PartStream.emit (events.js:198:13)
at HeaderParser. (/home/al/test/node_modules/dicer/lib/Dicer.js:51:16)
at HeaderParser.emit (events.js:198:13)
at HeaderParser._finish (/home/al/test/node_modules/dicer/lib/HeaderParser.js:68:8)
at SBMH. (/home/al/test/node_modules/dicer/lib/HeaderParser.js:40:12)
MulterError: Unexpected field
at wrappedFileFilter (/home/al/test/node_modules/multer/index.js:40:19)
at Busboy. (/home/al/test/node_modules/multer/lib/make-middleware.js:114:7)
at Busboy.emit (events.js:198:13)
at Busboy.emit (/home/al/test/node_modules/busboy/lib/main.js:38:33)
at PartStream. (/home/al/test/node_modules/busboy/lib/types/multipart.js:213:13)
at PartStream.emit (events.js:198:13)
at HeaderParser. (/home/al/test/node_modules/dicer/lib/Dicer.js:51:16)
at HeaderParser.emit (events.js:198:13)
at HeaderParser._finish (/home/al/test/node_modules/dicer/lib/HeaderParser.js:68:8)
at SBMH. (/home/al/test/node_modules/dicer/lib/HeaderParser.js:40:12)

What is the unexpected field and how do I fix this so that I can access the data like body.plateASCII etc.

UPDATE

Even with the code below on sole I get output :

[nodemon] starting 'node index.js'
server running
MulterError: Unexpected field

Same error - do not even get the console Hello statement.

If I change it to accept any files:

app.post('/test', upload.any(), (req, res) => {
console.log('hello')
console.log('req.files: ', req.files)

I get output:

    hello
req.files:  [ { fieldname: 'event',
    originalname: '20200924191124_784695event_21006.json',
    encoding: '7bit',
    mimetype: 'application/octet-stream',
    buffer:
     <Buffer 7b 22 70 61 63 6b 65 74 43 6f 75 6e 74 65 72 22 3a 22 32 31 30 30 36 22 2c 0d 0a 22 63 61 70 74 75 72 65 5f 74 69 6d 65 73 74 61 6d 70 22 3a 22 31 36 ... >,
    size: 963 },
  { fieldname: 'image',
    originalname: '20200924185429_410383roi_ABC123_15179.jpg',
    encoding: '7bit',
    mimetype: 'image/jpeg',
    buffer:
     <Buffer ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff db 00 43 00 06 04 05 06 05 04 06 06 05 06 07 07 06 08 0a 10 0a 0a 09 09 0a 14 0e 0f 0c ... >,
    size: 3182 } ]

Upvotes: 1

Views: 7815

Answers (2)

Al Grant
Al Grant

Reputation: 2354

The IoT camera is sending twp parts to the response. The first is fieldname events and the second fieldname image.

app.post('/test', upload.any(), (req, res) => { // 1.accept any files
   console.log('POST recd')
   var data = JSON.parse(req.files[0].buffer.toString()) //2. decode the binary JSON
   ....

This prints the response.

Upvotes: 0

Linus Unneb&#228;ck
Linus Unneb&#228;ck

Reputation: 24291

You don't show in your example where you are actually using the upload variable, but here is a complete example:

var express = require('express')
var multer = require('multer')
var upload = multer({ storage: multer.memoryStorage() })  // 1
var app = express()

app.post('/', upload.single('event'), (req, res) => { // 2
  var data = JSON.parse(req.file.buffer.toString()) // 3

  console.log('hello')
  res.send(200)
})

app.listen(3000, () => {
  console.log('server running')
})

  1. Specifying multer.memoryStorage() as the storage engine makes the data of the uploaded file available in memory.

  2. The upload.single('event') adds a middleware that expects a single file, with the field name event as can be seen in your data sent to the server.

  3. req.file.buffer.toString() turns the Buffer of the data into a string, assuming it's UTF-8 (which all JSON should be). JSON.parse then turns that string into a JavaScript object.

Upvotes: 2

Related Questions