CODEWITHSUNDEEP
google-chromemeteornpmsend
Matyas
Matyas

Reputation: 13702

Why does meteor hang while using WebApp & send npm package to serve custom files?

I've created a custom upload / serve mechanism to manage media files for a meteor app.

The problem

The whole application hangs after ~800 - 4000 (it varies from system to system) files are served to a browser

Simplified serve code

var idx = 0;
var send = Meteor.npmRequire('send');

WebApp.connectHandlers.use(function(req, res /*, next*/ ) {
    sendFile(req, res, '/penken.jpg');
});

var sendFile = function(req, res, urlPath) {
    var lidx = idx++;
    console.log(lidx, 'requesting', urlPath);

    send(req, urlPath, {
        root: process.env.PWD + '/.uploads',
        maxAge: 20 * 60 * 1000
    }).pipe(res);
};

Notes

Sample project to exemplify the issue

Find it at: https://github.com/albertmatyi/meteor-hangs

To try it out

  1. Clone the project git clone https://github.com/albertmatyi/meteor-hangs.git
  2. Enter its folder cd meteor-hangs
  3. Run the server meteor
  4. Open up chrome and navigate to (you should see a picture) http://localhost:3000/
  5. Open up the Inspector and disable the cache Disable cache
  6. Refresh the page ~810-4500 times (may vary) - Simply keep your finger on the refesh button (F5, Ctrl + R or Cmd + R) The console will show how many time the resource was requested and served
  7. Result: Meteor hangs and will not serve any other client

GitHub meteor issue ref

Upvotes: 0

Views: 197

Answers (1)

Tarang
Tarang

Reputation: 75945

This probably happens because you are using middleware that has blocking file I/O ops. The thing is for every page load, Meteor initially has to go through each middleware handler and run it before a page can be served.

The reason you are probably having this issue is that the operation you are doing is I/O intensive and its blocking the rest of the middleware from operating while it is awaiting the previous tasks to complete. (Keep in mind javascript is asynchronous but file I/O is blocking).

There isn't an easy way passed this. Immediate ideas that come to mind are to somehow use a memory buffer or something for this, or ensure that it only runs for the specific path (such as /upload instead of for all paths. This way you would not be inconvenienced so much.

The reason the wget operation works is there is no other javascript called. For each page load meteor has a couple of dozen javascript files for each module it has that the browser will call (which in turn each file request will call the middlware too). It will wait for the js before the page is served.

What I would suggest, to keep things simple, is to only do the sendFile operation if the route matches a specific pattern, such as /upload. This way it is not run if you use http://localhost:3000.

Something a bit like this (which will only work on /upload

var url = Npm.require('url');

WebApp.connectHandlers.use(function(req, res, next) {
    var path = url.parse(req.url).pathname;
    if(path != '/upload') next()

    sendFile(req, res, '/penken.jpg');
});

Upvotes: 1

Related Questions