How to do a follow on post of a file uploaded to a Node Express server onward to another API

Question

We have an Express server in Node, on one route we upload a file and as a follow on need to send this file onwards to another end point for processing, which is not accessible to the initial client. Most use cases seem to depend on getting a path from the local server, but in our case the file doesn't exist on the local server, and we have it as an uploaded object.

I've tried using multer and also with file-upload package, but run into issues with both of them on trying to do the follow on upload to the 3rd party API. Both post to the 3rd Party API which then fails. However if I upload a file locally using fs from the Express server then it gets uploaded to the API fine.

I could therefore save it locally first, and then use the local file to send on and remove it, but that feels like it shouldn't be necessary.

The code using Multer, which puts the file onto req.file:

    .post('/update', upload.single('file'),  async (req, res, next) => {
        try {
            const fileBuffer = req.file.buffer;
            const form = new FormData();
            form.append('file', fileBuffer);
            const boundary = form.getBoundary();
            const config = {
                headers: {
                    "Content-Type": `multipart/form-data; boundary=${boundary}`
            }};
            const axiosInst = axios.create(config);
            const url = `${baseAPI}/translate`;
            const { status, data } = await axiosInst.post(url, form);
            return res.status(status).json(data);
        } catch (error) {
            next(error);
        }
    })

The code using the file-upload package which puts the uploaded file onto req.files:

    .post('/update', async (req, res, next) => {
        try {
            let { file } = req.files;
            fileBuffer = Buffer.from(JSON.stringify(file));
            const form = new FormData();
            form.append('file', fileBuffer);
            const boundary = form.getBoundary();
            const config = {
                headers: {
                    "Content-Type": `multipart/form-data; boundary=${boundary}`
            }};
            const axiosInst = axios.create(config);
            const url = `${baseAPI}/translate`;
            const { status, data } = await axiosInst.post(url, form);
            return res.status(status).json(data);
        } catch (error) {
            next(error);
        }
    })

I've logged out the file object that gets posted to the API. In the successful post, taking from the file system on the node server it is:

FormData {
  _overheadLength: 237,
  _valueLength: 0,
  _valuesToMeasure: [
    ReadStream {
      _readableState: [ReadableState],
      readable: true,
      _events: [Object: null prototype],
      _eventsCount: 3,
      _maxListeners: undefined,
      path: 'c:\\work\\Test Template Upload Facility.docx',
      fd: null,
      flags: 'r',
      mode: 438,
      start: undefined,
      end: Infinity,
      autoClose: true,
      pos: undefined,
      bytesRead: 0,
      closed: false,
      emit: [Function],
      [Symbol(kFs)]: [Object],
      [Symbol(kCapture)]: false,
      [Symbol(kIsPerformingIO)]: false
    }
  ],
  writable: false,
  readable: true,
  dataSize: 0,
  maxDataSize: 2097152,
  pauseStreams: true,
  _released: false,
  _streams: [
    '----------------------------344728646415746168257760\r\n' +
      'Content-Disposition: form-data; name="file"; filename="Test Template Upload Facility.docx"\r\n' +
      'Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document\r\n' +
      '\r\n',
    DelayedStream {
      source: [ReadStream],
      dataSize: 0,
      maxDataSize: Infinity,
      pauseStream: true,
      _maxDataSizeExceeded: false,
      _released: false,
      _bufferedEvents: [Array],
      _events: [Object: null prototype],
      _eventsCount: 1
    },
    [Function: bound ]
  ],
  _currentStream: null,
  _insideLoop: false,
  _pendingNext: false,
  _boundary: '--------------------------344728646415746168257760'
}

In the posts to API using either the Multer or file-data approach, both which failed, the file objects between the Multer or file-data are fairly similar and logs out as this:

FormData {
  _overheadLength: 143,
  _valueLength: 12024,
  _valuesToMeasure: [],
  writable: false,
  readable: true,
  dataSize: 0,
  maxDataSize: 2097152,
  pauseStreams: true,
  _released: false,
  _streams: [
    '----------------------------486237832288086608829884\r\n' +
      'Content-Disposition: form-data; name="file"\r\n' +
      'Content-Type: application/octet-stream\r\n' +
      '\r\n',
    <Buffer 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 df a4 d2 6c 5a 01 00 00 20 05 00 00 13 00 08 02 5b 43 6f 6e 74 65 6e 74 5f 54 79 70 65 73 5d 2e 78 6d 6c 20 ... 11974 more bytes>,
    [Function: bound ]
  ],
  _currentStream: null,
  _insideLoop: false,
  _pendingNext: false,
  _boundary: '--------------------------486237832288086608829884'
}

I could of course save the file locally, and may have to do just that, but would appreciate if anyone can see a way to avoid having to do that, thanks.

Answer 1

The issue was the missing filename parameter, as correctly pointed out by @Maximorlov in his second point.

In case anyone wants to see the correct solution without reading thru comments, I've put it below.

    .post('/update', upload.single('file'),  async (req, res, next) => {
        try {
            const fileBuffer = req.file.buffer;
            const form = new FormData();
            form.append('file', fileBuffer, req.file.originalname);
            const boundary = form.getBoundary();
            const config = {
                headers: {
                    "Content-Type": `multipart/form-data; boundary=${boundary}`
            }};
            const axiosInst = axios.create(config);
            const url = `${baseAPI}/translate`;
            const { status, data } = await axiosInst.post(url, form);
            return res.status(status).json(data);
        } catch (error) {
            next(error);
        }
    })

Answer 2

I see two things:

1. You're constructing the form but eventually you're sending the file, not the form. Change your code to: await axiosInst.post(url, form)

2. When appending a file to a form in Node.js, you have to use the 3rd argument (file name). With multer as example, you would do: form.append('file', fileBuffer, req.file.originalname);

I wrote an article on how to Send a File With Axios in Node.js. It covers a few commong pitfalls to avoid and I think you'll learn something useful.