CODEWITHSUNDEEP
javascriptfirebasefirebase-storage
isuru
isuru

Reputation: 3565

How to download entire folder from Firebase Storage?

I want to download an entire folder from Firebase storage. I can download single files using DownloadURL as follows, but it does not work for folders.

var storageRef = firebase.storage().ref();

// Create a reference to the file we want to download
var starsRef = storageRef.child(path);

// Get the download URL
starsRef.getDownloadURL().then(function(url) {
  // Insert url into an <img> tag to "download"
  ImageUrl = url;

  console.log(ImageUrl);
}).catch(function(error) {
  switch (error.code) {
    case 'storage/object_not_found':
      // File doesn't exist
      break;

    case 'storage/unauthorized':
      // User doesn't have permission to access the object
      break;

    case 'storage/canceled':
      // User canceled the upload
      break;

    case 'storage/unknown':
      // Unknown error occurred, inspect the server response
      break;
  }
});

How to download entire folder from Firebase?

Upvotes: 49

Views: 43454

Answers (8)

John
John

Reputation: 11399

If you are working with Firebase Storage, it's recommended to use the gcloud storage commands (instead of gsutil)

Important: gsutil is not the recommended CLI for Cloud Storage. Use gcloud storage commands in the Google Cloud CLI instead. ref

Here's a page with some useful examples on how to use the CLI

In order to copy all files in a folder to a different folder, we can use the [--recursive, -R, -r] flag with the cp command:

gcloud storage cp --recursive fromDir toDir

For example:

gcloud storage cp --recursive gs://my-bucket-name.appspot.com/myFromFolder gs://my-bucket-name.appspot.com/myToFolder

Upvotes: 0

Harry Allen
Harry Allen

Reputation: 473

The gcloud CLI is now recommended over gsutil. Use the following command to download an entire bucket:

gcloud storage cp "bucket path" . -r

Docs: https://cloud.google.com/sdk/gcloud/reference/storage/cp

Upvotes: 0

Mr.Schtelbe
Mr.Schtelbe

Reputation: 145

Following the suggestions above, I made the .html script.


    <!DOCTYPE html>
<html>
<head>
  <title>Download Firebase Storage Folder</title>
</head>
<body>
  <h1>Download Firebase Storage Folder</h1>

  <button id="downloadButton">Download Folder</button>

  <script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-app.js"></script>
  <script src="https://www.gstatic.com/firebasejs/8.6.1/firebase-storage.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/jszip/3.7.1/jszip.min.js"></script>
  <script src="https://cdnjs.cloudflare.com/ajax/libs/FileSaver.js/2.0.5/FileSaver.min.js"></script>

  <script>
    // Initialize Firebase with your configuration
    const firebaseConfig = {
            apiKey: 'apiKey....', //replace
            authDomain: 'YOUR_AUTH_DOMAIN',
            projectId: 'YOUR_PROJECT_ID',
            storageBucket: 'YOUR_STORAGE_BUCKET',
            messagingSenderId: 'YOUR_MESSAGING_SENDER_ID',
            appId: 'YOUR_APP_ID',
    };

    // Initialize Firebase
    const app = firebase.initializeApp(firebaseConfig);

    // Define the downloadFolderAsZip function here
    const downloadFolderAsZip = async () => {
      console.log("Starting folder download...");

      const jszip = new JSZip();
      const storage = firebase.storage(app);
      const folderName = '/[Your folder name]'; //Your folder name 

      try {
        const folderRef = storage.ref(folderName);
        console.log("Folder reference created:", folderRef);

        const downloadFiles = async (folderRef, path) => {
          const folder = await storage.ref(folderRef.fullPath).listAll();
          console.log("Folder:", path);
          console.log("Files in the folder:", folder.items);

          const promises = folder.items.map(async (item) => {
            console.log("Downloading file:", item.name);
            const file = await item.getMetadata();
            const fileRef = storage.ref(item.fullPath);
            const fileBlob = await fileRef.getDownloadURL().then((url) => {
              return fetch(url).then((response) => {
                if (!response.ok) {
                  console.error("Failed to fetch file:", item.name);
                  throw new Error('Failed to fetch file');
                }
                return response.blob();
              });
            });
            jszip.folder(path).file(item.name, fileBlob);
          });

          await Promise.all(promises);

          for (const subfolder of folder.prefixes) {
            await downloadFiles(subfolder, `${path}${subfolder.name}/`);
          }
        };

        await downloadFiles(folderRef, '');

        console.log("Number of files added to the ZIP:", jszip.files.length);

        if (jszip.files.length === 0) {
          console.log("ZIP is empty.");
          return;
        }

        const blob = await jszip.generateAsync({ type: 'blob' });
        saveAs(blob, 'download.zip');
        console.log("Download completed.");
      } catch (error) {
        console.error('Error:', error);
        alert('An error occurred while downloading the folder.');
      }
    };

    // Add an event listener to the "Download Folder" button
    const downloadButton = document.getElementById('downloadButton');
    downloadButton.addEventListener('click', () => {
      downloadFolderAsZip();
    });
  </script>
</body>
</html>
  1. Fill the credentials
  2. Fill the folder name
  3. Save
  4. Open the .html file with the browser
  5. Be happy

Upvotes: 0

user3291025
user3291025

Reputation: 1117

For a recursive solution that includes subfolders in the zip file, see the following sample. You'll instantiate a jszip object, await promises from a function that zips files and traverses the directories, then save the zip. If the content is a file ("item"), it is zipped into the jszip object. If it is a folder ("prefix"), the function is called again with a new subpath, passing in the same jszip object. For further improvement, you may want to get contents with list and pagination if your contents are too many for listAll, since listAll limits retrievals.

import JSZip from 'jszip';
import { saveAs } from 'file-saver';
import {
  getStorage, ref, getBlob, listAll,
} from "firebase/storage";

const addFilesFromDirectoryToZip = async (directoryPath = "", zip) => {
  const storage = getStorage();
 
  const directoryContentsRef = ref(
    storage,
    directoryPath
  );
  const directoryContents = await listAll(directoryContentsRef);

  for (const file of directoryContents.items) {
    const fileRef = ref(storage, file.fullPath);
    const fileBlob = await getBlob(fileRef)
    zip.file(file.fullPath, fileBlob);
  }

  for (const folder of directoryContents.prefixes) {
    await addFilesFromDirectoryToZip(folder.fullPath, zip);
  };
};

export const downloadFolderAsZip = async (directoryPath = "") => {
  const zip = new JSZip();

  await addFilesFromDirectoryToZip(directoryPath, zip);

  const blob = await zip.generateAsync({ type: "blob" });
  const name = directoryPath.split('/').pop();
  saveAs(blob, name);
};

Upvotes: 0

geethsg7
geethsg7

Reputation: 285

You can download the folder by creating a zip file of it.

Here is a sample function:

import JSZip from 'jszip';
import { saveAs } from 'file-saver';
import {
  getStorage,
  listAll,
  ref,
  getDownloadURL,
  getMetadata,
} from 'firebase/storage';
import { auth } from '../../Firebase';

export const downloadFolderAsZip = async () => {
  const jszip = new JSZip();
  const storage = getStorage();
  const folderRef = ref(
    storage,
    'images'
  );
  const folder = await listAll(folderRef);
  const promises = folder.items
    .map(async (item) => {
      const file = await getMetadata(item);
      const fileRef = ref(storage, item.fullPath);
      const fileBlob = await getDownloadURL(fileRef).then((url) => {
        return fetch(url).then((response) => response.blob());
      });
      jszip.file(file.name, fileBlob);
    })
    .reduce((acc, curr) => acc.then(() => curr), Promise.resolve());
  await promises;
  const blob = await jszip.generateAsync({ type: 'blob' });
  saveAs(blob, 'download.zip');
};

Upvotes: 4

Flukedev8
Flukedev8

Reputation: 159

Command gustil for Windows !!!

gsutil cp -r gs://<bucket_name>.appspot.com/OBJECT_NAME "D:\path"

Use Cloud tools for PowerShell

REF for install windows >> https://cloud.google.com/storage/docs/gsutil_install

Upvotes: 8

Frank van Puffelen
Frank van Puffelen

Reputation: 598817

There is no API in Firebase Storage to download all files in a folder. You will have to download the files one by one, or create a zip file that contains all the files.

As Lahiru's answer shows it can be accomplished with gsutils, but that's a server-side operation - not something you'd run in your client-side application.

Related:

Upvotes: 13

Lahiru Chandima
Lahiru Chandima

Reputation: 24068

You can use gsutil to download the whole storage bucket

gsutil -m cp -R gs://<bucket_name> .

Upvotes: 100

Related Questions