CODEWITHSUNDEEP
phpsymfonycomposer-php
davedev
davedev

Reputation: 121

composer install keeps consuming all RAM memory

In the Symfony 2.7 project I'm working on, I use a lot of packages via Composer 1.6.5.

Since a few days I can no longer add new packages or rebuild the composer.lock. it keeps running out of memory.

I started noticing the problem when I tried to add a new package. but the issue is not that package. because as a test I am now trying to rebuild my composer.lock by deleting that composer.lock file and executing the following command:

php -d memory_limit=-1 C:/ProgramData/ComposerSetup/bin/composer.phar install -vvv

when I track my ram consumption when running that command, it never passes the 1.5G limit, while my Windows machine still has plenty of ram available.

I noticed that the error is usually thrown when/after composer is trying to resolve one particular package and writing it into the cache. That one particular package does consume a lot of memory (up to 0.5G) Some research learned me there is nothing wrong with this package and it has been in our project & thus in composer.json for more than three years now. I also run the same composer install command on a Mac and there it did manage to resolve this package and write it to cache, but then it failed later on when it tried to remove another unrelated package, because by then the memory on the mac was fully consumed.

this is the error I get:

...
Downloading http://packagist.org/p/kriswallsmith/spork%24dafe3f5808d858c7b15bd464923710bf24d5f7457b7b6c50a298ca1bea1a35a0.json
Writing C:/Users/david/AppData/Local/Composer/repo/https---packagist.org/provider-kriswallsmith$spork.json into cache

Fatal error: Out of memory (allocated 1424228352) (tried to allocate 268435456 bytes) in phar://C:/ProgramData/ComposerSetup/bin/composer.phar/src/Composer/DependencyResolver/Solver.php on line 220
PHP Fatal error:  Out of memory (allocated 1424228352) (tried to allocate 268435456 bytes) in phar://C:/ProgramData/ComposerSetup/bin/composer.phar/src/Composer/DependencyResolver/Solver.php on line 220
/cygdrive/c/ProgramData/ComposerSetup/bin/composer: line 10: 10284 Segmentation fault      php "${dir}/composer.phar" "$@"

The strange thing is that until a few days ago, when a used composer to require a package I'm working on, it worked just fine.

These are my 3 questions:

  1. why does composer consume so much memory?
  2. why does composer ignore my memory_limit=-1 setting and refuses to use more than 1.5G?
  3. I read somewhere that I can configure swap space on/for composer. Would this be an option?

here is my composer.json file:

{
  "name": "kulsol",
  "license": "proprietary",
  "type": "project",
  "autoload": {
    "psr-4": {
      "": "src/",
      "SymfonyStandard\\": "app/SymfonyStandard/"
    },
    "files": [
      "src/helpers.php",
      "app/AppKernel.php"
    ]
  },
  "autoload-dev": {
    "psr-4": {
      "Tests\\": "tests/"
    }
  },
  "require": {
    "php": ">=5.3.9",
    "symfony/symfony": "2.7.*",
    "doctrine/orm": "2.5.*",
    "doctrine/dbal": "<2.6",
    "doctrine/doctrine-bundle": "~1.4",
    "twig/extensions": "~1.0",
    "symfony/assetic-bundle": "~2.3",
    "symfony/swiftmailer-bundle": "^2.3",
    "symfony/monolog-bundle": "~2.4",
    "sensio/distribution-bundle": "~4.0",
    "sensio/framework-extra-bundle": "~3.0,>=3.0.2",
    "incenteev/composer-parameter-handler": "~2.0",
    "jms/serializer-bundle": "~2.0.0",
    "gedmo/doctrine-extensions": "2.4.3",
    "symfony/stopwatch": "^2.7",
    "vich/uploader-bundle": "^0.14.0",
    "nesbot/carbon": "^1.21",
    "swiftmailer/swiftmailer": "^5.4",
    "mtdowling/cron-expression": "^1.0",
    "cron/cron-bundle": "^1.0",
    "mustache/mustache": "^2.9",
    "leafo/lessphp": "<0.5",
    "friendsofsymfony/elastica-bundle": "^3.1",
    "guzzlehttp/guzzle": "^6.1",
    "oro/doctrine-extensions": "^1.0",
    "willdurand/geocoder-bundle": "^4.0",
    "rlerdorf/opcache-status": "dev-master",
    "henrikbjorn/stampie-bundle": "^1.0",
    "stof/stampie-extra": "^0.2.0",
    "isinlor/buzz-bundle": "^1.0",
    "ocramius/proxy-manager": "~1.0",
    "beberlei/DoctrineExtensions": "^1.0",
    "doctrine/doctrine-migrations-bundle": "^1.0",
    "ramsey/uuid": "3.*",
    "phpoffice/phpexcel": "1.8.1",
    "misd/guzzle-bundle": "v1.1.5",
    "php-http/guzzle6-adapter": "^1.1",
    "php-http/httplug-bundle": "^1.3",
    "egeloen/google-map": "2.0.x-dev",
    "egeloen/google-map-bundle": "dev-master#432438624d74f0384cc727b3609af3cc4314b152",
    "ezyang/htmlpurifier": "^4.8",
    "tfox/mpdf-port-bundle": "1.3.*",
    "ramsey/uuid-doctrine": "^1.2",
    "oneup/uploader-bundle": "^1.7",
    "webmozart/assert": "^1.2",
    "league/tactician-bundle": "~0.4",
    "kuleuven-gbiomed/message": "^0.1.0",
    "danielstjules/stringy": "^3.0",
    "html2text/html2text": "^4.0",
    "kuleuven-gbiomed/collection-utilities": "^0.1",
    "kuleuven/distribution-api": "*",
    "jms/serializer": "^1",
    "friendsofsymfony/rest-bundle": "^2.3"
  },
  "require-dev": {
    "sensio/generator-bundle": "~2.3",
    "kahlan/kahlan": "^3.1",
    "phpunit/phpunit": "^5.7",
    "doctrine/doctrine-fixtures-bundle": "^2.3",
    "friendsofphp/php-cs-fixer": "^2.2",
    "liip/functional-test-bundle": "^1.7",
    "vimeo/psalm": "^0.3.51",
    "phpmd/phpmd": "^2.6"
  },
  "scripts": {
    "post-root-package-install": [
      "SymfonyStandard\\Composer::hookRootPackageInstall"
    ],
    "post-install-cmd": [
      "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::removeSymfonyStandardFiles",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::prepareDeploymentTarget"
    ],
    "post-update-cmd": [
      "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::removeSymfonyStandardFiles",
      "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::prepareDeploymentTarget"
    ]
  },
  "extra": {
    "symfony-app-dir": "app",
    "symfony-web-dir": "web",
    "symfony-assets-install": "relative",
    "incenteev-parameters": {
      "file": "app/config/parameters.yml"
    },
    "symfony-var-dir": "var",
    "symfony-bin-dir": "bin",
    "symfony-tests-dir": "tests"
  },
  "repositories": [
    {
      "type": "vcs",
      "url": "https://github.com/Isinlor/SensioBuzzBundle.git"
    },
    {
      "type": "vcs",
      "url": "https://github.com/kuleuven-gbiomed/collection-utilities.git"
    },
    {
      "type": "path",
      "url": "distribution/API/"
    }
  ],
  "config": {
    "bin-dir": "bin"
  }
}

Upvotes: 2

Views: 2163

Answers (1)

davedev
davedev

Reputation: 121

In case it could help someone else, here's a quick solution

as it turned out, it's a windows-xampp related issue. xampp only supports 32 bit versions of php. which in turn means it only allows memory_limit to 1.5G (or 2G of which 0.5G is assigned elsewhere) at all times. Which effectively ignores any memory_limit settings in php.ini or set in runtime.

Solved it by downloading a 64 bit php and running composer from that 64 bit php version. that gives me up to 3G, enough to run composer

Upvotes: 6

Related Questions