CODEWITHSUNDEEP
pythonkubernetes
cglacet
cglacet

Reputation: 10952

How to properly administrate a database using a script

I use a python script to insert data in my database (using pandas and sqlalchemy). The script read from various sources, clean the data and insert it in the database. I plan on running this script once in a while to completely override the existing database with more recent data.

At first I wanted to have a single service and simply add an endpoint that would require higher privileges to run the script. But in the end that looks a bit odd and, more importantly, that python script is using quite a lot of memory (~700M) which makes me wonder how I should configure my deployment.

Increasing the memory limit of my pod for this (once in a while) operation looks like a bad idea to me, but I'm quite new to Kubernetes, so maybe I'm wrong. Thus this question.

So what would be a good (better) solution? Run another service just for that, simply connect to my machine and run the update manually using the python script directly?

Upvotes: 1

Views: 47

Answers (1)

Max Lobur
Max Lobur

Reputation: 6040

To run on demand https://kubernetes.io/docs/concepts/workloads/controllers/job/. This generates a Pod that runs till completion (exit) only once - a Job.

To run on schedule https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/. Every time when hitting a schedule, this generates the new, separate Job.

Upvotes: 2

Related Questions