CODEWITHSUNDEEP
linuxmacosbashsed
Jim from Princeton
Jim from Princeton

Reputation: 741

Standard linux sed not working in mac

found this way to get unread email count from gmail, but it does not work on my mac:

I think that the sed implementation on mac may not be 'standard' linux.

this is the line I am struggling with:

MAILCOUNTER=`curl -u $USERID:$PASSWORD --silent "https://mail.google.com/mail/feed/atom" \
   | sed -n 's|<fullcount>\(.*\)</fullcount>|\1|p'`


[email protected]
PASSWORD=yoursecretpassword

WAIT=10

# Loop to check for new mail every X minutes:
while [ "1" -eq "1" ]; do

    # Command line to fetch the number of unread emails:
    MAILCOUNTER=`curl -u $USERID:$PASSWORD --silent "https://mail.google.com/mail/feed/atom" \
| sed -n 's|<fullcount>\(.*\)</fullcount>|\1|p'`

    if [[ "$MAILCOUNTER" = "" ]]; then
        echo "ERROR: The program coulndn't fetch the account for user \"$USERID\"."
        echo "- Are you connected to the Internet?"
        echo -e "- Is the userid and password correct for \"$USERID\"?\n"
        curl -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET 192.168.1.50/?emailCount=-1&

    elif [[ "$MAILCOUNTER" -eq "0" ]]; then
        echo "* There is 0 new email for user $USERID."
        curl -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET 192.168.1.50/?emailCount=0&

    elif [[ "$MAILCOUNTER" -gt "0" ]]; then
        echo "* There is $MAILCOUNTER new email for user $USERID."
        curl -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET 192.168.1.50/?emailCount=$MAILCOUNTER&
    fi

    echo "* Waiting $WAIT seconds before checking for emails again."
    echo "* (^C to quit the program)"
    sleep $WAIT

done

the curl errors like this, which is driving me nuts because entering the URL into my browser returns correctly as follows the error

<HTML>
<HEAD>
<TITLE>Unauthorized</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Unauthorized</H1>
<H2>Error 401</H2>
</BODY>
</HTML>

URL RETURN:

<feed xmlns="http://purl.org/atom/ns#" version="0.3">
<title>Gmail - Inbox for [email protected]</title>
<tagline>New messages in your Gmail Inbox</tagline>
<fullcount>1</fullcount>
<link rel="alternate" href="http://mail.google.com/mail" type="text/html"/>
<modified>2015-01-02T00:14:36Z</modified>
<entry>

I corrected the google mail permissions and now curl returns

I get from this:

curl -u MYUSRNAME:MYPASSWORD --silent "https://mail.google.com/mail/feed/atom"

this:

<?xml version="1.0" encoding="UTF-8"?><feed version="0.3" xmlns="http://purl.org/atom/ns#"><title>Gmail - Inbox for [email protected]</title><tagline>New messages in your Gmail Inbox</tagline><fullcount>1</fullcount><link rel="alternate" href="http://mail.google.com/mail" type="text/html" /><modified>2015-01-02T02:03:50Z</modified><entry><title>test</title><summary>test</summary><link rel="alternate" href="http://mail.google.com/[email protected]&amp;message_id=14aa8623548638bc&amp;view=conv&amp;extsrc=atom" type="text/html" /><modified>2015-01-02T02:03:36Z</modified><issued>2015-01-02T02:03:36Z</issued><id>tag:gmail.google.com,2004:1489150113099430076</id><author><name>the author</name><email>[email protected]</email></author></entry></feed>MacBookPro-2:arduino_gmail_checker admin$

I just want the number in here:

<fullcount>1</fullcount>

Upvotes: 0

Views: 335

Answers (2)

David W.
David W.

Reputation: 107030

There are differences between Mac's sed which is based upon Unix standards and BSD, and the Linux version of sed which is based upon gnu and contains many different extentions that aren't compatible with other versions. However, looking at the code, there's really nothing about them that shouldn't be working on the Mac.

If would have been helpful to see the error messages you're getting to see what's going on. I suspect this isn't a Mac/sed issue as something you're not setting up correctly. There is absolutely no error checking with the curl commands. If one of these commands fail, it'll simply continue merrily along, and pipe the output to sed. Are you sure that these curl commands are fetching data?

Take the line that has the sed command, and try manually running it without the sed:

For example:

MAILCOUNTER=`curl -u $USERID:$PASSWORD --silent "https://mail.google.com/mail/feed/atom" \
| sed -n 's|<fullcount>\(.*\)</fullcount>|\1|p'`

Try these commands from the command line:

$ USERID=xxxxxx
$ PASSWORD=xxxxxx
$ curl -u $USERID:$PASSWORD --silent "https://mail.google.com/mail/feed/atom"

Does that curl command produce anything? If not, the problem isn't sed, but with your curl.

You can also try adding set -xv to the beginning of your script and set +xv to the end. This turns on shell debugging. Each line will print before being executed, and then print again with all the variables interpolated. This will help you track down errors in the code.

OK so it was a permissions thing with GMAIL, which I solved by loosening the security, and now curl returns as above in the edited post

Glad we got that settled. Here's the command I used to get the count:

$ curl -s  -u$username:$password "https://mail.google.com/mail/feed/atom" \
| xmllint --format - \
| sed -n 's#<fullcount>\(.*\)</fullcount>#\1#p'

The -s in curl prevents the progress report. The xmllint command reformats the output to make it easier for sed to use. To see what that does, try:

$ curl -s  -u$username:$password "https://mail.google.com/mail/feed/atom"

vs.

$ curl -s  -u$username:$password "https://mail.google.com/mail/feed/atom" \
| xmllint --format -

Upvotes: 3

Gerard Rozsavolgyi
Gerard Rozsavolgyi

Reputation: 5064

Mac sed implementation is based on BSD sed and differs slightly from Linux sed. The easyest solution is to install gsed (GNU sed, available from Mac Ports for example)

Upvotes: 1

Related Questions