CODEWITHSUNDEEP
htmlgoogle-app-engineuploadgoogle-cloud-platformgoogle-cloud-storage
Larry Cai
Larry Cai

Reputation: 1092

Monitor and automatically upload local files to Google Cloud Bucket

My goal is to make a website (hosted on Google's App Engine through a Bucket) that includes a upload button much like

<p>Directory: <input type="file" webkitdirectory mozdirectory /></p>

that prompts users to select a main directory.

The main directory will first generate a subfolder and have discrete files being written every few seconds, up to ~4000 per subfolder, upon which the machine software will create another subfolder and continue, and so on.

I want Google Bucket to automatically create a Bucket folder based on metadata (e.g. user login ID and time) in the background, and the website should monitor the main directory and subfolders, and automatically upload every file, sequentially from the time they are finished being written locally, into the Cloud Bucket folder. Each 'session' is expected to run for ~2-5 days.

Creating separate Cloud folders is meant to separate user data in case of multiple parallel users.

Does anyone know how this can be achieved? Would be good if there's sample code to adapt into existing HTML.

Thanks in advance!

Upvotes: 1

Views: 1077

Answers (1)

ZUKINI
ZUKINI

Reputation: 195

As per @JohnHanely, this is not really feasible using a application. I also do not understand the use case entirely but I can provide some insight into monitoring Cloud Buckets.

GCP provides Cloud Functions:

Respond to change notifications emerging from Google Cloud Storage. These notifications can be configured to trigger in response to various events inside a bucket—object creation, deletion, archiving and metadata updates.

The Cloud Storage Triggers will help you avoid having to monitor the buckets yourself and can instead leave that to GCF.

Maybe you could expand on what you are trying to achieve with that many folders? Are you trying to create ~4,000 sub-folders per user? There may be a better path forward should we know more about the intended use of the data? Is seems you want hold data and perhaps a DB is better suited? 

- Application
|--Accounts
|---- User1
|-------Metadata
|----User2
|------Meatadata

Upvotes: 0

Related Questions