J. Mini
J. Mini

Reputation: 1610

Do either Python or AWS Glue provide an alternative to .NET's SqlBulkCopy?

I am porting an old SSIS package to AWS Glue. The package runs daily. In several steps in this package, I take data from one table on one Microsoft SQL Server and copy all of it to an empty table of identical schema on another Microsoft SQL Server. I wish to replicate these steps in AWS Glue. This would be rather easy if I could write code in C# or PowerShell's dbatools because, like SSIS, they all use SqlBulkCopy either directly or indirectly. Unfortunately, I am under the following restrictions:

  1. If I am using a system where I can pick a language and one of the options is Python, then I must use either Python or SQL embedded in Python. I believe that IronPython is not an option.
  2. My copying must get the same or better performance as SSIS. In other words, I need SqlBulkCopy or an equivalent.
  3. AWS Glue must be involved. It is allowed to call other services, e.g. Lambda and Step Functions.

With these restrictions in mind, what options do I have? Do either Python or AWS Glue provide an alternative to .NET's SqlBulkCopy?

Upvotes: 3

Views: 202

Answers (0)

Related Questions