Reputation: 2665
I'm having trouble with the Gmail PHP API.
I want to retrieve the body content of emails, but I can retrieve it only for emails which have attachments! My question is why?
Here's my code so far:
// Authentication things above...
$client = getClient();
$gmail = new Google_Service_Gmail($client);
$list = $gmail->users_messages->listUsersMessages('me', ['maxResults' => 1000]);
while ($list->getMessages() != null) {
foreach ($list->getMessages() as $mlist) {
$message_id = $mlist->id;
$optParamsGet2['format'] = 'full';
$single_message = $gmail->users_messages->get('me', $message_id, $optParamsGet2);
$threadId = $single_message->getThreadId();
$payload = $single_message->getPayload();
$headers = $payload->getHeaders();
$parts = $payload->getParts();
//print_r($parts); PRINTS SOMETHING ONLY IF I HAVE ATTACHMENTS...
$body = $parts[0]['body'];
$rawData = $body->data;
$sanitizedData = strtr($rawData,'-_', '+/');
$decodedMessage = base64_decode($sanitizedData); //should display my body content
}
if ($list->getNextPageToken() != null) {
$pageToken = $list->getNextPageToken();
$list = $gmail->users_messages->listUsersMessages('me', ['pageToken' => $pageToken, 'maxResults' => 1000]);
} else {
break;
}
}
The second option to retrieve content that I know is by using the snippet located in the Headers part, but it only retrieves the 50 first characters or so, which isn't very useful.
Upvotes: 31
Views: 19442
Reputation: 1
I just want to complement @F3L1X79 answer, before you break de foreach loop, you have to check the $FOUND_BODY variable is not FALSE, so I added an If condition before every break; in the body search. If you don't do this, the code wil break even if the body was not found.
if($FOUND_BODY !== false) break;
Upvotes: 0
Reputation: 1123
I wasn't satisfied with the other answers because they're all flawed (elaboration in spoiler), and some are long and mixed with features the asker (and I) didn't look for.
To warn you about potential problems in other answers:
no plain text fallback
or failing to deal with falsy message body - the string '0' (unlikely to happen, but not too unlikely)
or lacking a deep enough search through the payload tree structure
So I thought I'd save others the trouble and share my code (tested on my entire inbox).
// input: the message object (not the payload!)
// output: html or plain text
function msg_body($msg) {
$body = msg_body_recursive($msg->payload);
return array_key_exists('html', $body) ? $body['html'] : $body['plain'];
}
function msg_body_recursive($part) {
if($part->mimeType == 'text/html') {
return ['html' => decodeBody($part->body->data)];
} else if($part->mimeType == 'text/plain') {
return ['plain' => decodeBody($part->body->data)];
} else if($part->parts) {
$return = [];
foreach($part->parts as $sub_part) {
$result = msg_body_recursive($sub_part);
$return = array_merge($return, $result);
if(array_key_exists('html', $return))
break;
}
return $return;
}
return [];
}
function decodeBody($encoded) {
$sanitizedData = strtr($encoded,'-_', '+/');
return base64_decode($sanitizedData);
}
Upvotes: 13
Reputation: 714
As further improvement the code should be recursive, also you need to load the message in format "full" to extract the body. Below three functions you can put into your own class.
private function decodeBody($body) {
$rawData = $body;
$sanitizedData = strtr($rawData,'-_', '+/');
$decodedMessage = base64_decode($sanitizedData);
if(!$decodedMessage)
return false;
return $decodedMessage;
}
private function decodeParts($parts)
{
foreach ($parts as $part)
{
if ($part->getMimeType() === 'text/html' && $part->getBody())
if ($result = $this->decodeBody($part->getBody()->getData()))
return $result;
}
foreach ($parts as $part)
{
if ($result = $this->decodeParts($part->getParts()))
return $result;
}
}
/**
* @param Google_Service_Gmail_Message $message
* @return bool|null|string
*/
public function getMessageBody($message)
{
$payload = $message->getPayload();
if ($result = $this->decodeBody($payload->getBody()->getData()))
return $result;
return $this->decodeParts($payload->getParts());
}
Upvotes: 0
Reputation: 112777
Let's do a little experiment. I've sent two messages to myself. One with an attachment, and one without.
Request:
GET https://www.googleapis.com/gmail/v1/users/me/messages?maxResults=2
Response:
{
"messages": [
{
"id": "14fe21fd6b3fb46f",
"threadId": "14fe21fd6b3fb46f"
},
{
"id": "14fe21f9341ed73c",
"threadId": "14fe21f9341ed73c"
}
],
"nextPageToken": "08943597140129624594",
"resultSizeEstimate": 3
}
I only ask for the payload, since that is where all the relevant parts are:
fields = payload
GET https://www.googleapis.com/gmail/v1/users/me/messages/14fe21fd6b3fb46f?fields=payload
GET https://www.googleapis.com/gmail/v1/users/me/messages/14fe21f9341ed73c?fields=payload
Mail without attachment:
{
"payload": {
"parts": [
{
"partId": "0",
"mimeType": "text/plain",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/plain; charset=UTF-8"
}
],
"body": {
"size": 22,
"data": "aGVjaz8gTm8gYXR0YWNobWVudD8NCg=="
}
},
{
"partId": "1",
"mimeType": "text/html",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/html; charset=UTF-8"
}
],
"body": {
"size": 43,
"data": "PGRpdiBkaXI9Imx0ciI-aGVjaz8gTm8gYXR0YWNobWVudD88L2Rpdj4NCg=="
}
}
]
}
}
Mail with attachment:
{
"payload": {
"parts": [
{
"mimeType": "multipart/alternative",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "multipart/alternative; boundary=001a1142e23c551e8e05200b4be0"
}
],
"body": {
"size": 0
},
"parts": [
{
"partId": "0.0",
"mimeType": "text/plain",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/plain; charset=UTF-8"
}
],
"body": {
"size": 9,
"data": "V293IG1hbg0K"
}
},
{
"partId": "0.1",
"mimeType": "text/html",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/html; charset=UTF-8"
}
],
"body": {
"size": 30,
"data": "PGRpdiBkaXI9Imx0ciI-V293IG1hbjwvZGl2Pg0K"
}
}
]
},
{
"partId": "1",
"mimeType": "image/jpeg",
"filename": "feelthebern.jpg",
"headers": [
{
"name": "Content-Type",
"value": "image/jpeg; name=\"feelthebern.jpg\""
},
{
"name": "Content-Disposition",
"value": "attachment; filename=\"feelthebern.jpg\""
},
{
"name": "Content-Transfer-Encoding",
"value": "base64"
},
{
"name": "X-Attachment-Id",
"value": "f_ieq3ev0i0"
}
],
"body": {
"attachmentId": "ANGjdJ_2xG3WOiLh6MbUdYy4vo2VhV2kOso5AyuJW3333rbmk8BIE1GJHIOXkNIVGiphP3fGe7iuIl_MGzXBGNGvNslwlz8hOkvJZg2DaasVZsdVFT_5JGvJOLefgaSL4hqKJgtzOZG9K1XSMrRQAtz2V0NX7puPdXDU4gvalSuMRGwBhr_oDSfx2xljHEbGG6I4VLeLZfrzGGKW7BF-GO_FUxzJR8SizRYqIhgZNA6PfRGyOhf1s7bAPNW3M9KqWRgaK07WTOYl7DzW4hpNBPA4jrl7tgsssExHpfviFL7yL52lxsmbsiLe81Z5UoM",
"size": 100446
}
}
]
}
}
These responses corresponds to the $parts
in your code. As you can see, if you are lucky, $parts[0]['body']->data
will give you what you want, but most of the time it will not.
There are generally two approaches to this problem. You could implement the following algorithm (you are much better at PHP than me, but this is the general outline of it):
payload.parts
and check if it contains a part
that has the body you were looking for (either text/plain
or text/html
). If it has, you are done with your searching. If you were parsing a mail like the one above with no attachment, this would be enough.parts
found inside the parts
you just checked, recursively. You will eventually find your part
. If you were parsing a mail like the one above with an attachment, this would eventually find you your body
.The algorithm could look something like the following (example in JavaScript):
var response = {
"payload": {
"parts": [
{
"mimeType": "multipart/alternative",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "multipart/alternative; boundary=001a1142e23c551e8e05200b4be0"
}
],
"body": {
"size": 0
},
"parts": [
{
"partId": "0.0",
"mimeType": "text/plain",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/plain; charset=UTF-8"
}
],
"body": {
"size": 9,
"data": "V293IG1hbg0K"
}
},
{
"partId": "0.1",
"mimeType": "text/html",
"filename": "",
"headers": [
{
"name": "Content-Type",
"value": "text/html; charset=UTF-8"
}
],
"body": {
"size": 30,
"data": "PGRpdiBkaXI9Imx0ciI-V293IG1hbjwvZGl2Pg0K"
}
}
]
},
{
"partId": "1",
"mimeType": "image/jpeg",
"filename": "feelthebern.jpg",
"headers": [
{
"name": "Content-Type",
"value": "image/jpeg; name=\"feelthebern.jpg\""
},
{
"name": "Content-Disposition",
"value": "attachment; filename=\"feelthebern.jpg\""
},
{
"name": "Content-Transfer-Encoding",
"value": "base64"
},
{
"name": "X-Attachment-Id",
"value": "f_ieq3ev0i0"
}
],
"body": {
"attachmentId": "ANGjdJ_2xG3WOiLh6MbUdYy4vo2VhV2kOso5AyuJW3333rbmk8BIE1GJHIOXkNIVGiphP3fGe7iuIl_MGzXBGNGvNslwlz8hOkvJZg2DaasVZsdVFT_5JGvJOLefgaSL4hqKJgtzOZG9K1XSMrRQAtz2V0NX7puPdXDU4gvalSuMRGwBhr_oDSfx2xljHEbGG6I4VLeLZfrzGGKW7BF-GO_FUxzJR8SizRYqIhgZNA6PfRGyOhf1s7bAPNW3M9KqWRgaK07WTOYl7DzW4hpNBPA4jrl7tgsssExHpfviFL7yL52lxsmbsiLe81Z5UoM",
"size": 100446
}
}
]
}
};
// In e.g. a plain text message, the payload is the only part.
var parts = [response.payload];
while (parts.length) {
var part = parts.shift();
if (part.parts) {
parts = parts.concat(part.parts);
}
if(part.mimeType === 'text/html') {
var decodedPart = decodeURIComponent(escape(atob(part.body.data.replace(/\-/g, '+').replace(/\_/g, '/'))));
console.log(decodedPart);
}
}
The far easier option is to just get the raw data of the mail, and let a already written library do the work for you:
Request:
format = raw
fields = raw
GET https://www.googleapis.com/gmail/v1/users/me/messages/14fe21fd6b3fb46f?format=raw&fields=raw
Response:
{
"raw": "TUlNRS1WZXJzaW9uOiAxLjANClJlY2VpdmVkOiBieSAxMC4yOC45OS4xOTYgd2l0aCBIVFRQOyBGcmksIDE4IFNlcCAyMDE1IDEzOjIzOjAxIC0wNzAwIChQRFQpDQpEYXRlOiBGcmksIDE4IFNlcCAyMDE1IDIyOjIzOjAxICswMjAwDQpEZWxpdmVyZWQtVG86IGVtdGhvbGluQGdtYWlsLmNvbQ0KTWVzc2FnZS1JRDogPENBRHNaTFJ5eGk2UGt0MHZnUS1iZHd1N2FNLWNHRmZKcEwrRHYyb3ZKOGp4SGN4VWhfQUBtYWlsLmdtYWlsLmNvbT4NClN1YmplY3Q6IFdoYXQgZGENCkZyb206IEVtaWwgVGhvbGluIDxlbXRob2xpbkBnbWFpbC5jb20-DQpUbzogRW1pbCBUaG9saW4gPGVtdGhvbGluQGdtYWlsLmNvbT4NCkNvbnRlbnQtVHlwZTogbXVsdGlwYXJ0L2FsdGVybmF0aXZlOyBib3VuZGFyeT0wMDFhMTE0NjhmMTY1YzUwNDUwNTIwMGI0YzYxDQoNCi0tMDAxYTExNDY4ZjE2NWM1MDQ1MDUyMDBiNGM2MQ0KQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFyc2V0PVVURi04DQoNCmhlY2s_IE5vIGF0dGFjaG1lbnQ_DQoNCi0tMDAxYTExNDY4ZjE2NWM1MDQ1MDUyMDBiNGM2MQ0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9VVRGLTgNCg0KPGRpdiBkaXI9Imx0ciI-aGVjaz8gTm8gYXR0YWNobWVudD88L2Rpdj4NCg0KLS0wMDFhMTE0NjhmMTY1YzUwNDUwNTIwMGI0YzYxLS0="
}
The biggest drawback of the second method is that if you get the message raw, you will download all the attachment data right away, which might be far to much data for your use case.
I'm not good at PHP, but this looks promising if you want to go with the second solution! Good luck!
Upvotes: 27
Reputation: 2991
I wrote this code as an improvement of @F3L1X79's answer as this filters html response correctly.
<?php
ini_set("display_errors", 1);
ini_set("track_errors", 1);
ini_set("html_errors", 1);
error_reporting(E_ALL);
require_once __DIR__ . '/vendor/autoload.php';
session_start();
function decodeBody($body) {
$rawData = $body;
$sanitizedData = strtr($rawData,'-_', '+/');
$decodedMessage = base64_decode($sanitizedData);
if(!$decodedMessage){
$decodedMessage = FALSE;
}
return $decodedMessage;
}
function fetchMails($gmail, $q) {
try{
$list = $gmail->users_messages->listUsersMessages('me', array('q' => $q));
while ($list->getMessages() != null) {
foreach ($list->getMessages() as $mlist) {
$message_id = $mlist->id;
$optParamsGet2['format'] = 'full';
$single_message = $gmail->users_messages->get('me', $message_id, $optParamsGet2);
$payload = $single_message->getPayload();
// With no attachment, the payload might be directly in the body, encoded.
$body = $payload->getBody();
$FOUND_BODY = decodeBody($body['data']);
// If we didn't find a body, let's look for the parts
if(!$FOUND_BODY) {
$parts = $payload->getParts();
foreach ($parts as $part) {
if($part['body'] && $part['mimeType'] == 'text/html') {
$FOUND_BODY = decodeBody($part['body']->data);
break;
}
}
} if(!$FOUND_BODY) {
foreach ($parts as $part) {
// Last try: if we didn't find the body in the first parts,
// let's loop into the parts of the parts (as @Tholle suggested).
if($part['parts'] && !$FOUND_BODY) {
foreach ($part['parts'] as $p) {
// replace 'text/html' by 'text/plain' if you prefer
if($p['mimeType'] === 'text/html' && $p['body']) {
$FOUND_BODY = decodeBody($p['body']->data);
break;
}
}
}
if($FOUND_BODY) {
break;
}
}
}
// Finally, print the message ID and the body
print_r($message_id . " <br> <br> <br> *-*-*- " . $FOUND_BODY);
}
if ($list->getNextPageToken() != null) {
$pageToken = $list->getNextPageToken();
$list = $gmail->users_messages->listUsersMessages('me', array('pageToken' => $pageToken));
} else {
break;
}
}
} catch (Exception $e) {
echo $e->getMessage();
}
}
$client = new Google_Client();
$client->setAuthConfig('client_secrets.json');
$client->addScope(Google_Service_Gmail::GMAIL_READONLY);
if (isset($_SESSION['access_token']) && $_SESSION['access_token']) {
$client->setAccessToken($_SESSION['access_token']);
$gmail = new Google_Service_Gmail($client);
$q = ' after:2016/11/7';
fetchMails($gmail, $q);
} else {
$redirect_uri = 'http://' . $_SERVER['HTTP_HOST'] . '/gmail-api/oauth2callback.php';
header('Location: ' . filter_var($redirect_uri, FILTER_SANITIZE_URL));
}
Upvotes: 13
Reputation: 2665
UPDATE: You might want to check my second answer below this one for a more complete code.
Finally, I worked today, so here's the complete code answer for finding the body - thanks to @Tholle:
// Authentication things above
/*
* Decode the body.
* @param : encoded body - or null
* @return : the body if found, else FALSE;
*/
function decodeBody($body) {
$rawData = $body;
$sanitizedData = strtr($rawData,'-_', '+/');
$decodedMessage = base64_decode($sanitizedData);
if(!$decodedMessage){
$decodedMessage = FALSE;
}
return $decodedMessage;
}
$client = getClient();
$gmail = new Google_Service_Gmail($client);
$list = $gmail->users_messages->listUsersMessages('me', ['maxResults' => 1000]);
try{
while ($list->getMessages() != null) {
foreach ($list->getMessages() as $mlist) {
$message_id = $mlist->id;
$optParamsGet2['format'] = 'full';
$single_message = $gmail->users_messages->get('me', $message_id, $optParamsGet2);
$payload = $single_message->getPayload();
// With no attachment, the payload might be directly in the body, encoded.
$body = $payload->getBody();
$FOUND_BODY = decodeBody($body['data']);
// If we didn't find a body, let's look for the parts
if(!$FOUND_BODY) {
$parts = $payload->getParts();
foreach ($parts as $part) {
if($part['body']) {
$FOUND_BODY = decodeBody($part['body']->data);
break;
}
// Last try: if we didn't find the body in the first parts,
// let's loop into the parts of the parts (as @Tholle suggested).
if($part['parts'] && !$FOUND_BODY) {
foreach ($part['parts'] as $p) {
// replace 'text/html' by 'text/plain' if you prefer
if($p['mimeType'] === 'text/html' && $p['body']) {
$FOUND_BODY = decodeBody($p['body']->data);
break;
}
}
}
if($FOUND_BODY) {
break;
}
}
}
// Finally, print the message ID and the body
print_r($message_id . " : " . $FOUND_BODY);
}
if ($list->getNextPageToken() != null) {
$pageToken = $list->getNextPageToken();
$list = $gmail->users_messages->listUsersMessages('me', ['pageToken' => $pageToken, 'maxResults' => 1000]);
} else {
break;
}
}
} catch (Exception $e) {
echo $e->getMessage();
}
As you can see, my problem was, sometimes the body cannot be found in the payload->parts but directly in the payload->body! (plus I add the loop for multiple parts).
Hope this helps somebody else.
Upvotes: 29
Reputation: 2665
For those who are interested I greatly improved my last answer, making it working with text/html (and fallback to text/plain if necessary) and transforming the images as base64 attachments that will auto-load when printed as full HTML!
Code isn't perfect at all and is way too long to explain in details but it's working for me.
Feel free to take it and adapt it (maybe correct/improve it if necessary).
// Authentication things above
/*
* Decode the body.
* @param : encoded body - or null
* @return : the body if found, else FALSE;
*/
function decodeBody($body) {
$rawData = $body;
$sanitizedData = strtr($rawData,'-_', '+/');
$decodedMessage = base64_decode($sanitizedData);
if(!$decodedMessage){
$decodedMessage = FALSE;
}
return $decodedMessage;
}
$client = getClient();
$gmail = new Google_Service_Gmail($client);
$list = $gmail->users_messages->listUsersMessages('me', ['maxResults' => 1000]);
try{
while ($list->getMessages() != null) {
foreach ($list->getMessages() as $mlist) {
$message_id = $mlist->id;
$optParamsGet2['format'] = 'full';
$single_message = $gmail->users_messages->get('me', $message_id, $optParamsGet2);
$payload = $single_message->getPayload();
$parts = $payload->getParts();
// With no attachment, the payload might be directly in the body, encoded.
$body = $payload->getBody();
$FOUND_BODY = FALSE;
// If we didn't find a body, let's look for the parts
if(!$FOUND_BODY) {
foreach ($parts as $part) {
if($part['parts'] && !$FOUND_BODY) {
foreach ($part['parts'] as $p) {
if($p['parts'] && count($p['parts']) > 0){
foreach ($p['parts'] as $y) {
if(($y['mimeType'] === 'text/html') && $y['body']) {
$FOUND_BODY = decodeBody($y['body']->data);
break;
}
}
} else if(($p['mimeType'] === 'text/html') && $p['body']) {
$FOUND_BODY = decodeBody($p['body']->data);
break;
}
}
}
if($FOUND_BODY) {
break;
}
}
}
// let's save all the images linked to the mail's body:
if($FOUND_BODY && count($parts) > 1){
$images_linked = array();
foreach ($parts as $part) {
if($part['filename']){
array_push($images_linked, $part);
} else{
if($part['parts']) {
foreach ($part['parts'] as $p) {
if($p['parts'] && count($p['parts']) > 0){
foreach ($p['parts'] as $y) {
if(($y['mimeType'] === 'text/html') && $y['body']) {
array_push($images_linked, $y);
}
}
} else if(($p['mimeType'] !== 'text/html') && $p['body']) {
array_push($images_linked, $p);
}
}
}
}
}
// special case for the wdcid...
preg_match_all('/wdcid(.*)"/Uims', $FOUND_BODY, $wdmatches);
if(count($wdmatches)) {
$z = 0;
foreach($wdmatches[0] as $match) {
$z++;
if($z > 9){
$FOUND_BODY = str_replace($match, 'image0' . $z . '@', $FOUND_BODY);
} else {
$FOUND_BODY = str_replace($match, 'image00' . $z . '@', $FOUND_BODY);
}
}
}
preg_match_all('/src="cid:(.*)"/Uims', $FOUND_BODY, $matches);
if(count($matches)) {
$search = array();
$replace = array();
// let's trasnform the CIDs as base64 attachements
foreach($matches[1] as $match) {
foreach($images_linked as $img_linked) {
foreach($img_linked['headers'] as $img_lnk) {
if( $img_lnk['name'] === 'Content-ID' || $img_lnk['name'] === 'Content-Id' || $img_lnk['name'] === 'X-Attachment-Id'){
if ($match === str_replace('>', '', str_replace('<', '', $img_lnk->value))
|| explode("@", $match)[0] === explode(".", $img_linked->filename)[0]
|| explode("@", $match)[0] === $img_linked->filename){
$search = "src=\"cid:$match\"";
$mimetype = $img_linked->mimeType;
$attachment = $gmail->users_messages_attachments->get('me', $mlist->id, $img_linked['body']->attachmentId);
$data64 = strtr($attachment->getData(), array('-' => '+', '_' => '/'));
$replace = "src=\"data:" . $mimetype . ";base64," . $data64 . "\"";
$FOUND_BODY = str_replace($search, $replace, $FOUND_BODY);
}
}
}
}
}
}
}
// If we didn't find the body in the last parts,
// let's loop for the first parts (text-html only)
if(!$FOUND_BODY) {
foreach ($parts as $part) {
if($part['body'] && $part['mimeType'] === 'text/html') {
$FOUND_BODY = decodeBody($part['body']->data);
break;
}
}
}
// With no attachment, the payload might be directly in the body, encoded.
if(!$FOUND_BODY) {
$FOUND_BODY = decodeBody($body['data']);
}
// Last try: if we didn't find the body in the last parts,
// let's loop for the first parts (text-plain only)
if(!$FOUND_BODY) {
foreach ($parts as $part) {
if($part['body']) {
$FOUND_BODY = decodeBody($part['body']->data);
break;
}
}
}
if(!$FOUND_BODY) {
$FOUND_BODY = '(No message)';
}
// Finally, print the message ID and the body
print_r($message_id . ": " . $FOUND_BODY);
}
if ($list->getNextPageToken() != null) {
$pageToken = $list->getNextPageToken();
$list = $gmail->users_messages->listUsersMessages('me', ['pageToken' => $pageToken, 'maxResults' => 1000]);
} else {
break;
}
}
} catch (Exception $e) {
echo $e->getMessage();
}
Cheers.
Upvotes: 22