Bash scripts extremely slow to start on OSX

Question

I'm having a problem where my bash scripts take an extremely long time to start. At first I thought that it was something in the script itself, but a quick experiment disproved that.

This is the example script

#!/bin/bash
echo 'ping'

When I run it

$ time ./script.sh

I get

ping

real    0m12.018s
user    0m0.002s
sys    0m0.002s

(I've even seen it go up to 17 seconds...) What's incredibly strange is that when I run it the second time, it goes instantly.

ping

real    0m0.004s
user    0m0.002s
sys 0m0.002s

But when I edit the file again, it's back to the long wait.

$ echo 'echo test' >> gbr.sh
$ time ./gbr.sh
ping
test

real    0m13.021s
user    0m0.003s
sys 0m0.003s

It's almost like bash is compiling my script or something. Is there any way to debug this? It also doesn't seem to be anything with my .bash_profile - if I just put an echo on the first line of .bash_profile I still only see that after 10+ seconds.

I've tried this in iTerm2 and in native Terminal - both have exactly the same issue. This is on macOS Sierra - 10.12.3

Answer 1

Starting with 10.15, macOS might actually be talking to Apple every time a new script is executed. I found this in a recent changelog for TextMate:

Fixed: Delay when re-executing (bundle) commands on macOS 10.15. Starting with macOS 10.15 Apple will “call home” each time a new script/binary is executed, this can have a delay of more than a second, depending on internet connectivity/location. The result seems to be cached per inode, previously TextMate would use temporary files when executing scripts or shell commands (giving them a new inode on each run), it now re-uses these files to avoid the delay on repeated executions.

Answer 2

I had the same issue on my work machine, running macOS Sierra 10.12.5 and 10.12.6, and I confirmed that @charles-duffy is right. My company's anti-malware is the problem. They are using a daemon called Confer by Carbon Black for endpoint security.

Run this command to see if the problem stops:

launchctl unload /Library/LaunchDaemons/com.confer.sensor.daemon.plist

I think Confer is taking a fingerprint of every new program it sees and tries to compare it to a blacklist before allowing it to run. It looks like it times out after a second (trying to register the fingerprint online?), and something is making it try over and over again, yielding longer delays of integer seconds. This makes newly modified programs launch slowly, although speeds return to normal after the fingerprint has been taken.

---

Example runs

Here's how the problem manifested on my work laptop. As time went on, sometimes the delay increased, but always by an integer number of seconds at a time. If Confer remembers the program from before, there will be no delay.

bash scripts

$ bash --norc --noprofile -l  # make sure rc scripts don't interfere

bash-3.2$ echo exit > exit.sh && chmod +x exit.sh

bash-3.2$ time ./exit.sh

real    0m1.004s
user    0m0.001s
sys 0m0.002s

bash-3.2$ time ./exit.sh

real    0m0.002s
user    0m0.001s
sys 0m0.001s

There were no problems running commands provided to bash -c:

bash-3.2$ time bash -c exit

real    0m0.008s
user    0m0.002s
sys 0m0.003s

awk scripts

But the problem did affect awk scripts:

bash-3.2$ printf '%s\n' '#!/usr/bin/env awk -f' 'BEGIN { exit 0; }' > test.awk && chmod +x test.awk

bash-3.2$ time ./test.awk

real    0m4.010s
user    0m0.002s
sys 0m0.001s

bash-3.2$ time ./test.awk

real    0m0.005s
user    0m0.002s
sys 0m0.001s

Compiled binaries

It even affected compiled C and Golang code:

bash-3.2$ printf '%s\n' '#include "stdio.h"' 'int main() {' 'printf("Hello, world!\n");' 'return 0;' '}' > hello.c && gcc -o hello hello.c

bash-3.2$ time ./hello
Hello, world!

real    0m4.006s
user    0m0.001s
sys 0m0.001s

bash-3.2$ time ./hello
Hello, world!

real    0m0.004s
user    0m0.001s
sys 0m0.001s

bash-3.2$ printf '%s\n' 'package main' 'import "fmt"' 'func main() {' 'fmt.Println("test")' '}' > test.go && go build -o test ./test.go

bash-3.2$ time ./test
test

real    0m4.018s
user    0m0.001s
sys 0m0.003s

bash-3.2$ time ./test
test

real    0m0.005s
user    0m0.001s
sys 0m0.002s