App Engine Bulk Loader Download Data Corrupt File

Question

I am using Google App Engine and I have a high replication datastore on my current web application. I am attempting to test AE's bulkloader to download a 'kind' from my datastore. I am entering the following code in my CMD in order to download all entities of the inputted 'kind.'

python appcfg.py download_data --kind=TestClass --url=http://bulkloader-testing.appspot.com/remote_api --filename=success.csv

The download seems to occur successfully:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Jack Frost>cd..

C:\Users>cd..

C:\>cd "Program Files (x86)"

C:\Program Files (x86)>cd "Google App Engine SDK"

C:\Program Files (x86)\Google App Engine SDK>python appcfg.py download_data  --kind=TestClass --url=http://bulkloader-testing.appspot.com/remote_api --filename=success.csv
Downloading data records.
[INFO    ] Logging to bulkloader-log-20120125.155035
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10
[INFO    ] Opening database: bulkloader-progress-20120125.155035.sql3
[INFO    ] Opening database: bulkloader-results-20120125.155035.sql3
[INFO    ] Connecting to bulkloader-testing.appspot.com/remote_api
[INFO    ] Downloading kinds: ['TestClass']
.[INFO    ] TestClass: No descending index on __key__, performing serial download
.
[INFO    ] Have 14 entities, 0 previously transferred
[INFO    ] 14 entities (4808 bytes) transferred in 2.3 seconds

However; when I open the .csv file on my computer it opens and is un-comprehensible (I have opened it both with excel and notepad...Any ideas as to what I am doing wrong? Below is what the output looks like:

SQLite format 3 @
ü Õ fÏÕ GGtablebulkloader_database_signaturebulkloader_database_signatureCREATE TABLE bulkloader_database_signature ( value TEXT not null)g)tableresultresultCREATE TABLE result ( id BLOB primary key, value BLOB not null, sort_key BLOB)+? indexsqlite_autoindex_result_1result ö ûö „ ML‚\ :TestClass 00000000000000001003j(js~bulkloader-testingr TestClassërid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassëŠ! GAEGamer@gmail.com gmail.com )L‚ :TestClass 00000000000000001002j(js~bulkloader-testingr TestClassêrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassê)L‚ :TestClass 00000000000000001001j(js~bulkloader-testingr TestClassérid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassé'L‚ :TestClass 00000000000000000002j'js~bulkloader-testingr TestClassrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass'L‚ :TestClass 00000000000000000001j'js~bulkloader-testingr TestClassrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ Te

ú Û¶‘lG"ýسŽiDú $L:TestClass 00000000000000006001$L:TestClass 00000000000000005001 $L:TestClass 00000000000000004001$L:TestClass 00000000000000003003$L:TestClass 00000000000000003002 $L:TestClass 00000000000000003001 $L:TestClass 00000000000000002003$L:TestClass 00000000000000002002$L:TestClass 00000000000000002001$L:TestClass 00000000000000001003$L:TestClass 00000000000000001002$L:TestClass 00000000000000001001$L:TestClass 00000000000000000002$L:TestClass 00000000000000000001 {ƒ} app_id: s~bulkloader-testing url: http://bulkloader-testing.appspot.com/remote_api kind: TestClass download: False map: False dump: True restore: False progress_db: bulkloader-progress-20120125.155035.sql3 has_header: False

„ V¬ T „ ML‚\ :TestClass 00000000000000001003j(js~bulkloader-testingr TestClassërid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassëŠ! GAEGamer@gmail.com gmail.com )L‚ :TestClass 00000000000000001002j(js~bulkloader-testingr TestClassêrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassê)L‚ :TestClass 00000000000000001001j(js~bulkloader-testingr TestClassérid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassé'L‚ :TestClass 00000000000000000002j'js~bulkloader-testingr TestClassrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass'L‚ :TestClass 00000000000000000001j'js~bulkloader-testingr TestClassrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass ¤ T¨üP ¤ ) L‚ :TestClass 00000000000000003002j(js~bulkloader-testingr TestClassºrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassº) L‚ :TestClass 00000000000000003001j(js~bulkloader-testingr TestClass¹rid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass¹)L‚ :TestClass 00000000000000002003j(js~bulkloader-testingr TestClassÓrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassÓ)L‚ :TestClass 00000000000000002002j(js~bulkloader-testingr TestClassÒrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassÒ)L‚ :TestClass 00000000000000002001j(js~bulkloader-testingr TestClassÑrid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassÑ P T¨üP )L‚ :TestClass 00000000000000006001j(js~bulkloader-testingr TestClassñ.rid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClassñ.) L‚ :TestClass 00000000000000005001j(js~bulkloader-testingr TestClass‰'rid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass‰')L‚ :TestClass 00000000000000004001j(js~bulkloader-testingr TestClass¡rid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass¡)L‚ :TestClass 00000000000000003003j(js~bulkloader-testingr TestClass»rid *Ò…ØÌrmobile * 9786978770rname *Bulkloader Test‚ TestClass»

UPDATE: I assume the reason is because this is an SQLLite format and hence may not be a true CSV. I will check tomorrow when I come into work and update again...

Answer 1

The bulk downloader is downloading to SQLite, not CSV. If you want CSV you have to create and configure bulkloader.yaml: http://code.google.com/appengine/docs/python/tools/uploadingdata.html#Editing_the_Configuration_File

Specifying csv in the filename is not enough.

Answer 2

Alright, so I still do not know why when I view the .csv file it has unintelligible symbols in it BUT I have verified that the contents of the file are what they should be. After downloading the .csv file via the bulkloader I flushed the datastore and then re-uploaded the .csv file. After re-uploading the .csv file all of the datastore's prior content was restored. As a result, I assume the unintelligible symbols are due to my computer lacking some sort of configuration/software but since I only need the data itself and not a visual display of it I am moving on :)