CODEWITHSUNDEEP
pythonpython-3.xamazon-web-servicesamazon-s3boto3
bharathi m
bharathi m

Reputation: 11

How to rename the s3 file name by using python boto3

Hi i was created a new file/folder in s3 bucket now i want to change the name of the folder/file name . how to rename the file/folder name in s3 bucket by using python boto3

Upvotes: 0

Views: 5292

Answers (2)

ViaTech
ViaTech

Reputation: 2843

There are a lot of 'issues' with folder structures in s3 it seems as the storage is flat.

I have a Django project where I needed the ability to rename a folder but still keep the directory structure in-tact, meaning empty folders would need to be copied and stored in the renamed directory as well.

aws cli is great but neither cp or sync or mv copied empty folders (i.e. files ending in '/') over to the new folder location, so I used a mixture of boto3 and the aws cli to accomplish the task.

More or less I find all folders in the renamed directory and then use boto3 to put them in the new location, then I cp the data with aws cli and finally remove it.

import threading

import os
from django.conf import settings
from django.contrib import messages
from django.core.files.storage import default_storage
from django.shortcuts import redirect
from django.urls import reverse

def rename_folder(request, client_url):
    """
    :param request:
    :param client_url:
    :return:
    """
    current_property = request.session.get('property')
    if request.POST:
        # name the change
        new_name = request.POST['name']
        # old full path with www.[].com?
        old_path = request.POST['old_path']
        # remove the query string
        old_path = ''.join(old_path.split('?')[0])
        # remove the .com prefix item so we have the path in the storage
        old_path = ''.join(old_path.split('.com/')[-1])
        # remove empty values, this will happen at end due to these being folders
        old_path_list = [x for x in old_path.split('/') if x != '']

        # remove the last folder element with split()
        base_path = '/'.join(old_path_list[:-1])
        # # now build the new path
        new_path = base_path + f'/{new_name}/'
        # remove empty variables
        # print(old_path_list[:-1], old_path.split('/'), old_path, base_path, new_path)
        endpoint = settings.AWS_S3_ENDPOINT_URL
        # # recursively add the files
        copy_command = f"aws s3 --endpoint={endpoint} cp s3://{old_path} s3://{new_path} --recursive"
        remove_command = f"aws s3 --endpoint={endpoint} rm s3://{old_path} --recursive"
        
        # get_creds() is nothing special it simply returns the elements needed via boto3
        client, resource, bucket, resource_bucket = get_creds()
        path_viewing = f'{"/".join(old_path.split("/")[1:])}'
        directory_content = default_storage.listdir(path_viewing)

        # loop over folders and add them by default, aws cli does not copy empty ones
        # so this is used to accommodate
        folders, files = directory_content
        for folder in folders:
            new_key = new_path+folder+'/'
            # we must remove bucket name for this to work
            new_key = new_key.split(f"{bucket}/")[-1]
            # push this to new thread
            threading.Thread(target=put_object, args=(client, bucket, new_key,)).start()
            print(f'{new_key} added')

        # # run command, which will copy all data
        os.system(copy_command)
        print('Copy Done...')
        os.system(remove_command)
        print('Remove Done...')

        # print(bucket)
        print(f'Folder renamed.')
        messages.success(request, f'Folder Renamed to: {new_name}')

    return redirect(request.META.get('HTTP_REFERER', f"{reverse('home', args=[client_url])}"))

Upvotes: 0

John Rotenstein
John Rotenstein

Reputation: 270154

There is no 'rename' function in Amazon S3. Object names are immutable.

You would need to:

  • Copy the object to a new Key (filename)
  • Delete the original object

Please note that folders do not actually exist in Amazon S3. Rather, the full path of the object is stored in its Key. Thus, it is not possible to rename folders, since that would involve renaming all objects within that path. (And objects can't be renamed, as mentioned above.)

If you wanted to "rename a folder", you could write a Python script that will:

  • Obtain a listing of all objects within the given Prefix
  • Loop through each object, then:
  • Copy the object to a new Key
  • Delete the original object

If you do not want to code this, then there are some tools (eg Cyberduck) that give a nice user interface and can do many of these operations for you.

Upvotes: 1

Related Questions