CODEWITHSUNDEEP
pythonbashperforcecron
jackr
jackr

Reputation: 1437

crontab job occasionally quits without explanation

I have a crontab job that runs hourly. Sometimes it quits early without explanation, or reason I can discern. What might be the problem? What might help me dig deeper?

The context is Ubuntu 12.04.3 LTS. SHELL is /bin/bash.

The job definition is presently:

59 * * * * set -x; set +e; cd ~/OPEN/staging-site ; /usr/local/bin/p4 sync -q ; bin/publish.py ...

(the "set"s are part of my current debugging work).

The resulting email, when it fails, with some headers pruned, is:

From: Cron Daemon <[email protected]>
To: "Repenning, Jack" <[email protected]>
Content-Class: urn:content-classes:message
Date: Fri, 17 Jan 2014 14:57:01 -0600
Subject: Cron <jrepenni@prod-akiary-app01> set -x; set +e; cd  ~/OPEN/staging-site ; /usr/local/bin/p4 sync  -q ; bin/staging
x-cron-env: <LOGNAME=jrepenni>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

+ set +e
+ cd /u0/jrepenni/OPEN/staging-site
+ /usr/local/bin/p4 sync -q

Note that there's no mention of launching bin/staging. Confirming this, there are side effects that would be found, such as creating a tmp directory for its work, if even the most minimal execution of bin/staging happened. No such traces.

I happen to know that the Perforce API returns an error for some things that no rational being would consider errors (such as "you requested a sync but were already in sync"). This does not seem to extend to the command line, which returns no-error (0) in this case. But, what the heck, I tried the set +e in case such a spurious error return was halting the script. No help.

The script bin/staging takes care of $PATH and such cron-required niceties -- but, of course, it doesn't even get launched, so that's neither here nor there.

UPDATE: The script is executable:

> ll bin/staging 
-r-xr-xr-x 1 jrepenni users 251 Jan 17 11:55 bin/staging

... and shabanged:

> head -1 bin/staging
#!/bin/bash

Upvotes: 0

Views: 214

Answers (2)

jackr
jackr

Reputation: 1437

Hmmm: either I misspoke, or behavior is variable.

I now see those external traces, indicating that the bin/staging script does run. Cron job email sill stops at the p4 line, but this seems to be an I/O problem, not an actual execution one. Possibly ssh is at fault (in our config, p4 talks to its repository via ssh tunnel).

Whatever's going on, it changed last night; I had been getting regular output from the bin/staging process, and suddenly not.

Thanks, all, for the labor of your "little grey cells."

Upvotes: 0

chicks
chicks

Reputation: 2463

It seems likely that things are breaking in perforce since that is where it stops. It could help to merge STDERR into your output by adding 2>&1 to you p4 sync.

Putting your code into a script and running it repeatedly could you trip over the issue interactively.

Upvotes: 1

Related Questions