AWS S3 make any new file added public

Question

I have an AWS S3 bucket that I have the AWS SES move raw emails to. my issue is that these emails are not public by default and I need a php script I have that grabs the raw files from this S3 bucket and puts it into a mysql to be able to do so without me having to manually mark the files as public in the folder.

So using my below code which I do use an IAM Key and IAM Secret how can I make my below script load in to eh AWS bucket with the correct permissions to be able top grab the raw files?

 //mysql connection 
  $servername = "***>rds.amazonaws.com";
  $username = "****";
  $password ="***";
  $databasename ="***";

  $rightnowdatetimeis = date('Y-m-d H:i:s');
  $rightnowdateis = date('Y-m-d');

$con = mysqli_connect("$servername","$username","$password","$databasename");

if (mysqli_connect_errno())
{
  echo "Failed to connect to MySQL: " . mysqli_connect_error();
}


//load the php mime parse library
require_once __DIR__.'/vendor/autoload.php';

$Parser = new PhpMimeMailParser\Parser();


//Include the AWS SDK using the Composer autoloader.
require 'awssdk/aws-autoloader.php';

use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
// AWS Info
$bucketName = 'pipedemail';
$IAM_KEY = '****';
$IAM_SECRET = '***';
// Connect to AWS
try {
    // You may need to change the region. It will say in the URL when the bucket is open
    // and on creation. us-east-2 is Ohio, us-east-1 is North Virgina
    $s3 = S3Client::factory(
        array(
            'credentials' => array(
                'key' => $IAM_KEY,
                'secret' => $IAM_SECRET
            ),
            'version' => 'latest',
            'region'  => 'us-east-1'
        )
    );
} catch (Exception $e) {
    // We use a die, so if this fails. It stops here. Typically this is a REST call so this would
    // return a json object.
    die("Error: " . $e->getMessage());
}

// Use the high-level iterators (returns ALL of your objects).
$objects = $s3->getIterator('ListObjects', array('Bucket' => $bucketName));

foreach ($objects as $object) 
{
    $objectkey = $object['Key'];

    // Get the object
    $result = $s3->getObject(array(
       'Bucket' => $bucketName,
       'Key'    => $objectkey
    ));

    //lets get the raw email file to parse it
    //$Parser->setText($result['Body']);

    //echo "$objectkey";
    $path = "*****";

    //lets get the raw email file to parse it
    $Parser->setText(file_get_contents($path));

    // Once we've indicated where to find the mail, we can parse out the data
    //$to = $Parser->getHeader('to');             // "test" <test@example.com>, "test2" <test2@example.com>
     $addressesTo = $Parser->getAddresses('to'); //Return an array : [[test, test@example.com, false],[test2, test2@example.com, false]]
     $tobrand = $addressesTo[0]['address'];

     $to_brand_domainname = explode("@", "$tobrand", 2)[1]; 
     //lets get the brandid based on the to domain name
     $brandsql = mysqli_query($con, "SELECT brandid FROM brands WHERE branddomainname='$to_brand_domainname' LIMIT 1");
     $brandrow = mysqli_fetch_array($brandsql);
     $brandid = $brandrow['brandid'];

     $from = $Parser->getHeader('from');             // John Smith
     //lets break the full name into a lastname and firstname veriable
     $namepieces = explode(" ", $from);
     $first_name = $namepieces[0];
     $last_name = $namepieces[1];

     $addressesFrom = $Parser->getAddresses('from'); //Return an array : test, test@example.com, false
     $fromname = $addressesFrom[0]['display']; //not sure what this returns yet

     $fromemail = $addressesFrom[0]['address'];
     $subject = $Parser->getHeader('subject');


    //html of email body
    $html_emailbody = $Parser->getMessageBody('html');
   // $htmlEmbedded = $Parser->getMessageBody('htmlEmbedded'); //HTML Body included data


   //First lets see if this email address exists within the database 
    $emailsql = mysqli_query($con, "SELECT cid FROM customer_profiles WHERE email_address='$fromemail' LIMIT 1");
    $erow = mysqli_fetch_assoc($emailsql);
    $cid = $erow['cid'];

   //if customer does not exists
     if($cid < 1)
     {
          $customsql = mysqli_query($con, "INSERT into customer_profiles(first_name, last_name, email_address, last_contact_date) 
                       VALUES('$first_name','$last_name','$fromemail','$rightnowdateis')");
          $cid = mysqli_insert_id($con);
     }

   //create the support issue
     $sql = mysqli_query($con, "INSERT into product_issues(cid, date_created, brandid) VALUES('$cid','$rightnowdatetimeis','$brandid')");
     $issueid = mysqli_insert_id($con);


     mysqli_query($con, "INSERT into customer_notes(cid, date_written, note_body, issueid, note_type, brandid, note_subject, note_status) 
                           VALUES('$cid','$rightnowdatetimeis','$html_emailbody','$issueid','email','$brandid','$subject','unread')");
     $noteid = mysqli_insert_id($con);

    //Pass in a writeable path to save attachments
     $attach_dir = 'email_attachments/';     // Be sure to include the trailing slash
    //$include_inline = false;             // Optional argument to include inline attachments (default: true)
    //$Parser->saveAttachments($attach_dir [,$include_inline]);
    // $Parser->saveAttachments($attach_dir);

    // Get an array of Attachment items from $Parser
    // $attachments = $Parser->getAttachments([$include_inline]);

    //  Loop through all the Attachments
     //   if(count($attachments) > 0) 
     //    {
      //       foreach ($attachments as $attachment) 
      //       {
       //        $fileattachmentname = $attachment->getFilename();
        //        $attachment_filetype = $attachment->getContentType();

            //save file attachement name to database
         //       mysqli_query($con, "INSERT into email_attachments(attachment_name, attachment_filetype, cid, noteid, issueid) 
         //                            VALUES('$fileattachmentname','$attachment_filetype','$cid','$noteid','$issueid')");

            //echo 'Filename : '.$attachment->getFilename().'<br />'; // logo.jpg
            //echo 'Filesize : '.filesize($attach_dir.$attachment->getFilename()).'<br />'; // 1000
            //echo 'Filetype : '.$attachment->getContentType().'<br />'; // image/jpeg
            //echo 'MIME part string : '.$attachment->getMimePartStr().'<br />'; // (the whole MIME part of the attachment)
         //   }
      //  }

    //now lets delete the object since we already took the email and saved it into mysql
    $s3->deleteObject(array('Bucket' => $bucketName, 'Key' => $objectkey)); 
}

Answer 1

if you want public read access to your bucket, then this policy will do it

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AddPerm",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::examplebucket/*"]
    }
  ]
}

But what you want to do with your emails , that is public read access to bucket objects is done with the help of ACL , i.e. Access Control Lists , and you need not give public read access to bucket because your case wants public read access to objects, i.e emails only.

While storing an object to s3 , you can set ACL of that object to public-read in your code to do your work

example in python to upload an image with public read access using ACL

import boto3    

s3client=boto3.client('s3')
s3client.upload_file('filename', 'bucket-name', ktr,ExtraArgs={'ACL': 'public-read'})

You can leran more about ACL's on this link

https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html

Answer 2

I know you already mentioned the security concerns, but I want to re-iterate that you aren't just giving public read access but it appears you are trying to allow any s3 action. You aren't just opening yourself up to someone reading your data (which you don't seem concerned about), but also writing their own data or manipulating your data. You really do not want a bucket that can scale to infinite size to be writable to the entire public and doubly so if you are then going to process whatever get's written there.

You should seriously review how bucket ACLs work and consider using IAM credentials or instance roles (if the server processing data is also on AWS) to achieve securely handling your email data. Nothing prevents you from using a IAM policies to allow whatever is processing this data to read and write securely to this bucket. There are many reasons why AWS pastes warnings about not doing this all over their documentation.