CODEWITHSUNDEEP
pythonmemoryerlangrabbitmqcelery
horriblyUnpythonic
horriblyUnpythonic

Reputation: 853

Celery back-end failure?

I messed something up somewhere.

I'm using celery to run a task that runs other tasks. I'm using RabbitMQ as my back-end.

When I start-up my computer, I immediately have erl.exe taking up 50% of my CPU resources and several gigs of RAM. If I let it go, the RAM usage will continue to climb pretty steadily. (the drop is when I ended the process)

taskmgr

If I end the process, celery beat/workers error out out with 

[2014-04-17 12:35:07,869: ERROR/MainProcess] consumer: Cannot connect to amqp://[email protected]:5672//: [Errno 10061] No connection could be made because the target machine actively refused it.

If I don't end the process, the workers and the beat just spit out the config info and then hang.

If I try to use discard_all() it also hangs while the RAM usage steadily climbs.

Is there a way I can reset erlang without going through celery? I don't care about anything in the queue, it's all dev stuff.

EDIT:

Here is the end of my rabbitmq log file.

=INFO REPORT==== 17-Apr-2014::13:20:41 ===
Starting RabbitMQ 3.2.4 on Erlang R16B03-1
Copyright (C) 2007-2013 GoPivotal, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

=INFO REPORT==== 17-Apr-2014::13:20:41 ===
node           : rabbit@myComputer
home dir       : C:\Windows
config file(s) : (none)
cookie hash    : +z4Hn5WvfV2DO2xSryp0/Q==
log            : C:/Users/myUser/AppData/Roaming/RabbitMQ/log/[email protected]
sasl log       : C:/Users/myUser/AppData/Roaming/RabbitMQ/log/[email protected]
database dir   : c:/Users/myUser/AppData/Roaming/RabbitMQ/db/rabbit@myComputer-mnesia

=INFO REPORT==== 17-Apr-2014::13:20:44 ===
Limiting to approx 8092 file handles (7280 sockets)

=INFO REPORT==== 17-Apr-2014::13:20:45 ===
Memory limit set to 9829MB of 24573MB total.

=INFO REPORT==== 17-Apr-2014::13:20:45 ===
Disk free limit set to 50MB

=INFO REPORT==== 17-Apr-2014::13:24:57 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 17-Apr-2014::13:24:57 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index

=WARNING REPORT==== 17-Apr-2014::13:24:57 ===
msg_store_persistent: rebuilding indices from scratch

=INFO REPORT==== 17-Apr-2014::13:26:48 ===
started TCP Listener on [::]:5672

=INFO REPORT==== 17-Apr-2014::13:26:49 ===
started TCP Listener on 0.0.0.0:5672

=WARNING REPORT==== 17-Apr-2014::13:35:04 ===
file descriptor limit alarm set.

********************************************************************
*** New connections will not be accepted until this alarm clears ***
********************************************************************

=WARNING REPORT==== 17-Apr-2014::13:35:05 ===
file descriptor limit alarm cleared

=INFO REPORT==== 17-Apr-2014::13:35:05 ===
Server startup complete; 0 plugins started.

=WARNING REPORT==== 17-Apr-2014::13:35:06 ===
file descriptor limit alarm set.

********************************************************************
*** New connections will not be accepted until this alarm clears ***
********************************************************************

=WARNING REPORT==== 17-Apr-2014::13:35:07 ===
file descriptor limit alarm cleared

=INFO REPORT==== 17-Apr-2014::13:37:40 ===
vm_memory_high_watermark set. Memory used:10558805712 allowed:10306889318

=WARNING REPORT==== 17-Apr-2014::13:37:40 ===
memory resource limit alarm set on node 'rabbit@myComputer'.

**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************

=INFO REPORT==== 17-Apr-2014::13:37:41 ===
vm_memory_high_watermark clear. Memory used:10087183696 allowed:10306889318

=WARNING REPORT==== 17-Apr-2014::13:37:41 ===
memory resource limit alarm cleared on node 'rabbit@myComputer'

=INFO REPORT==== 17-Apr-2014::13:38:04 ===
vm_memory_high_watermark set. Memory used:10323681952 allowed:10306889318

=WARNING REPORT==== 17-Apr-2014::13:38:04 ===
memory resource limit alarm set on node 'rabbit@myComputer'.

**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************

Upvotes: 0

Views: 1592

Answers (1)

horriblyUnpythonic
horriblyUnpythonic

Reputation: 853

I ended up:

  1. Uninstalling RabbitMQ and Erlang.
  2. Removed C:\Users\username\AppData\RabbitMQ
  3. Restarted because at least one piece of Erlang (epmd.exe) was still running.
  4. Reinstalled Erlang and RabbitMQ.

Hello, IT. Have you tried turning it off and on again?

Upvotes: 1

Related Questions