CODEWITHSUNDEEP
google-bigquery
shantanuo
shantanuo

Reputation: 32316

inserting quotes in big query

I can easily upload a file delimited by ^ It looks something like...

CN^others^2012-05-03 00:02:25^^^^^Mozilla/5.0^generic web browser^^^^^^^^

CN^others^2012-05-03 00:02:26^^^^^Mozilla/5.0^generic web browser^^^^^^^^

But if I have a double quote somewhere, it fails with an error message...

Line:1 / Field:, Data between close double quote (") and field separator: field starts with: Too many errors encountered. Limit is: 0.

CN^others^2012-05-03 00:02:25^^^^^"Mozilla/5.0^generic web browser^^^^^^^^

I do regulary get the files with "Mozilla as browser name, how do I insert data with double quotes?

Upvotes: 1

Views: 3872

Answers (1)

Jordan Tigani
Jordan Tigani

Reputation: 26617

Quotes can be escaped with another quote. For example, the field: This field has "internal quotes". would become This field has ""internal quotes"".

sed 's/\"/\"\"/g' should do the trick.

Note that in order to import data that contains quoted newlines, you need to set the allow_quoted_newlines flag to true on the import configuration. This means the import cannot be processed in parallel, and so may be slower than importing data without that flag set.

Upvotes: 3

Related Questions