CODEWITHSUNDEEP
meteorbuild-process
sami
sami

Reputation: 733

Super slow build time - EMFILE, too many open files

I am making a Meteor web application that may list thousands of items - each item may have 1-5 images

I am using OSX 10.11.4 and Meteor 1.3.2.4

So far, there haven't been any problems with the build process

Today, I moved a folder that contains 20K images into the public/ directory and the project build time increased tremendously - I looked up some advice and did a repair disk but that did not solve the problem

I cancelled the build process and it showed an error with the text "EMFILE, too many open files” before exiting

Once I removed the images folder, the build time went back to the < 3s it was before

Is this normal? If yes, how can I keep it to a minimum while having 10Ks of images

Upvotes: 1

Views: 291

Answers (1)

Philip Pryde
Philip Pryde

Reputation: 940

Meteor cmd creates a checksum of each file so it can tell If something has been modified when serving to the client so I think that's where the open files are coming from as it scans each one tk create the checksums.

MAC has a low open file limit of about 256 max which is the EMFILE error you are getting. You can check the current limit with ulimit -n and you can increase it by running ulimit -n 512 where 512 is the new limit, as per here.

Just increase incrementally to see how much you need to set it to or pop in 20k and see what happens...

Still gonna take a while to load, might be better serving images from an nginx server or s3 buckets or similar rather than on meteor itself. If you want to scale up it means you need to deploy a whole bunch of images everytime which can be slow going over network depending on size

Upvotes: 1

Related Questions