php not able to access s3 objects unless I manually click make public

Question

I have a amazon SES setup to forward/save emails in a s3 bucket.

In s3 I have a bucket policy which reads as following and is support to automatically make all objects in the folder public (I know this is not ideal to do):

{
"Version": "2012-10-17",
"Id": "Policy1517940937030",
"Statement": [
    {
        "Sid": "Stmt1517940933454",
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::pipedemail/*"
    }
  ]
}

I then have a php script that is used to list out the objects in the s3 bucket and grab each object so we can put them into a mysql.

My issues is that I can list out the objects without any issue. However my script tries to parse and save the contents of each object into a mysql table and it only works if I manually login to the s3 and click "make public" on each of the objects? so my s3 bucket policy does not seem to be working? I know my other option is to use the $s3->getObject() tag to get the object but I am not sure how to use that tag with the php file_get_contents() method so that i can get the raw email object file to be parsed?

here is my code of course I ** out the logins/api keys.

<?php

//mysql connection 
$servername = "*****";
$username = "***";
$password ="****";
$databasename ="***";

$con = mysqli_connect("$servername","$username","$password","$databasename");

if (mysqli_connect_errno())
{
  echo "Failed to connect to MySQL: " . mysqli_connect_error();
}


//load the php mime parse library
require_once __DIR__.'/vendor/autoload.php';

$Parser = new PhpMimeMailParser\Parser();


//Include the AWS SDK using the Composer autoloader.
require 'awssdk/aws-autoloader.php';

use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
// AWS Info
$bucketName = 'pipedemail';
$IAM_KEY = '******';
$IAM_SECRET = '*****';
// Connect to AWS
try {
    // You may need to change the region. It will say in the URL when the bucket is open
    // and on creation. us-east-2 is Ohio, us-east-1 is North Virgina
    $s3 = S3Client::factory(
        array(
            'credentials' => array(
                'key' => $IAM_KEY,
                'secret' => $IAM_SECRET
            ),
            'version' => 'latest',
            'region'  => 'us-east-1'
        )
    );
} catch (Exception $e) {
    // We use a die, so if this fails. It stops here. Typically this is a REST call so this would
    // return a json object.
    die("Error: " . $e->getMessage());
}

// Use the high-level iterators (returns ALL of your objects).
$objects = $s3->getIterator('ListObjects', array('Bucket' => $bucketName));

foreach ($objects as $object) 
{
    $objectkey = $object['Key'];
    $path = "https://s3.amazonaws.com/pipedemail/$objectkey";

    //lets get the raw email file to parse it
    $Parser->setText(file_get_contents($path));

    // Once we've indicated where to find the mail, we can parse out the data
    //$to = $Parser->getHeader('to');             // "test" <test@example.com>, "test2" <test2@example.com>
    //$addressesTo = $Parser->getAddresses('to'); //Return an array : [[test, test@example.com, false],[test2, test2@example.com, false]]

    $from = $Parser->getHeader('from');             // "test" <test@example.com>

    $addressesFrom = $Parser->getAddresses('from'); //Return an array : test, test@example.com, false

    $subject = $Parser->getHeader('subject');

    //html of email body
    $html_emailbody = $Parser->getMessageBody('html');

    //insert the above pp variables data into a mysql table
    mysqli_query($con, "INSERT into emails(From_email,email_subject, email_body) VALUES('$from','$subject','$html_emailbody')");


    //now lets delete the object since we already took the email and saved it into mysql
    $s3->deleteObject(array('Bucket' => $bucketName, 'Key' => $objectkey)); 
}

?>

Answer 1

Yes, you should be using the getObject method as suggested by ceejayoz. You'll need to make sure the IAM user associated with the IAM key/secret you are using has s3:GetObject permissions or this call will fail.

Your code would then look something like:

    $objectkey = $object['Key'];

    // Get the object
    $result = $s3->getObject(array(
        'Bucket' => $bucketName,
        'Key'    => $objectkey
    ));

    //lets get the raw email file to parse it
    $Parser->setText($result['Body']);