How can I set Azure Blob Storage metadata to a numpy array?

Question

I would like to upload an image taken from a camera to Azure blob storage. I would also like to have an array of metadata (2KB array) attached to the image so that it can be retrieved at the same time as the image.

I'm uploading the image like this:

block_blob_service = BlockBlobService(
    account_name=settings.blob_name, account_key=settings.blob_key
)
block_blob_service.create_blob_from_stream(settings.blob_container, file_name, file_stream)

And setting metadata like this:

block_blob_service.set_blob_metadata(settings.blob_container,file_name,metadata={'f': np.array2string(array)})

But I get this error: azure.common.AzureException: Invalid return character or leading space in header: x-ms-meta-f

When I change the metadata to metadata={'f': "test"}, it works, so clearly the issue is the type of metadata. Is there a way to encode a numpy array so that it can be stored as metadata in Azure Blob Storage?

Sample array: [0.04955578638849593, 0.05177976606105041, -0.053318993917208075, -0.0011000404318764782, 0.014003503826903803, -0.032367835788941444, 0.07245417716935086, 0.05666663178602262, 0.07437731914085548, -0.014784160219013352, -0.03878502613543301, 0.02566167451220335, 0.031147309822621663, -0.060197377589914315, -0.060788198116395485, -0.01986763844100292, 0.017256696341404, -0.142380831362999, -0.037670310855975365, 0.00825975678190534, 0.14184623324812148, 0.007353056759508543, 0.037373733576363026, -0.1113637223282887, -0.024453949651278215, -0.05252461110378625, -0.07651801316588394, 0.03842642202584872, -0.004279922515586983, -0.02098751240179475, 0.046649385324818646, 0.007071314296201437, 0.06588686558856517, 0.004986403924302999, -0.06295243448714902, -0.03298373544313955, -0.001576552426238296, 0.09535429232060981, -0.09484623075111286, 0.015088723534152644, 0.038101839960783764, 0.007018969463338412, -0.06811956451481527, 0.06098000726782092, -0.07189545951363394, 0.14811996896388926, -0.0238130736822429, 0.030682159102404488, -0.13957872224153933, -0.024658783571205094, 0.06723701154021253, -0.020788778712732277, -0.09754204217733271, -0.06547405564280787, 0.021245563711261368, 0.02115207072888564, -0.06672521984728932, 0.06099327554051121, 0.1291634095622793, 0.14144203793944532, -7.664536720945415e-05, 0.04960947635653551, -0.08927941475160102, -0.08808275941072759, 0.0029037229685335993, -0.041537059973256504, 0.13771156819612315, -0.06596784606310341, 0.09234859926739268, 0.16737776649565964, -0.02107310863351277, -0.009757203828592971, 0.03505361845005009, -0.0632459309878387, 0.0368991673608436, -0.07959921776993119, 0.0637808177002734, -0.11299688147436203, -0.054904682372598546, -0.006692673850277836, -0.005304856447814152, 0.03618099955479746, -0.04096549251130717, -0.05061261982039628, -0.010057012498978102, 0.04458482333536819, -0.0942970079550264, 0.00596110880450805, -0.04747469136648984, 0.04556000169594045, 0.03894620787110517, -0.03252171239894806, -0.08798258719863841, -0.0035996908584291242, -0.0522677556704471, -0.03372864542032575, 0.03348910223294655, -0.0006987685826342861, 0.08049519053094173, 0.004998828951005972, -0.004650899794500701, -0.04334005878171669, 0.020525648086259698, 0.027288910193416022, 0.022200051430720208, -0.0032363654731832133, -0.025731832578348206, -0.06260364269450806, 0.08031192386720813, 0.017974097560188942, -0.08587876872647568, -0.06812048802699817, -0.07779380917186875, 0.020181454013451193, -0.02460146629261899, -0.014789425320696735, -0.013380280107825469, 0.1573464761558305, 0.08851829641411099, 0.07317678937777722, 0.08825644463556356, 0.07060225386001365, -0.006828706563512179, -0.06741543697992482, 0.02355000438275122, -0.02428296822480585, -0.004213341706349826, 0.08525777171792429, 0.06495591480362721, 0.09492815638265079, -0.08785229983145991, -0.016361755316660297, -0.04683850691149306, -0.046169801119274156, -0.08682097481614276, 0.00442597362245782, -0.08120127331428191, 0.06865276293154059, -0.021560376749016563, -0.07311616224595748, 0.0743518287616196, -0.042066852938807374, 0.06888799880037921, 0.0684758598439423, -0.013565642711317914, -0.05096792670289006, -0.03138654998256585, -0.010443392319704023, -0.03648175789151911, 0.07810045850622585, -0.033430094854916347, 0.037157299838373174, -0.00579948109542335, 0.06756868228277516, -0.08785273272779565, 0.005662767021393907, -0.05857409930270415, -0.07824816995091771, -0.0036479326457100067, 0.0308488674812962, -0.08775717086168204, -0.026447828687776738, 0.05580374312747066, -0.013028297508443192, 0.02333259843174069, 0.061699387183607124, 0.12583951605747987, 0.03097079273425608, -0.05953072084035122, 0.03491825537333509, 0.02438466459274376, -0.07853968955834088, 0.03834811107871418, -0.06875475330823985, -0.00828764252086563, 0.07365842262597752, -0.013856330797029519, 0.010999376415434253, 0.07652252250271452, 0.0698410561581954, 0.052842825985246436, -0.03554885185813142, 0.06403707790115092, -0.08833246123211871, -0.03193255491589001, -0.0551723277471059, -0.051780848301889754, 0.043976246844182935, -0.06905741278136927, 0.010857564987051524, -0.052156003481377, -0.004297235212480619, -0.0819276589357681, 0.04265997142333741, 0.05248355449381121, -0.028278098161653522, -0.01574660962358026, -0.09982292246575016, -0.06207531436155976, -0.05459793924403535, -0.040878657113828105, 0.04077808086515891, -0.05421289238327655, 0.053815031790979975, 0.033588347325383364, -0.087377527990181, 0.018380206631416002, -0.06845521068872773, -0.00876886361567962, -0.08313949450814799, -0.008825665026132305, -0.04529559943648165, 0.04512231464075622, -0.015645548170268093, 0.05521571838982445, 0.01233825984741356, 0.06946590097870815, 0.054318825724100475, -0.014672174446295429, 0.09591570114854722, 0.1544101114507814, -0.026979472285162877, 0.13926723889809983, 0.013684003784446907, -0.016103168297978206, 0.030936500129526905, 0.010095674651096107, 0.10653259922148137, -0.08178926919217241, -0.08876530706328163, -0.024578492123334565, 0.11689646938617816, 0.1311694656119551, 0.07089754688049664, -0.05646585085803906, -0.008091049864930316, -0.016533591713395742, -0.11047111894375797, 0.05019708623519331, -0.005151109255774569, -0.06357877055050783, -0.09657623602242912, 0.09058913540042962, 0.08560517102735196, -0.04054737237049818, 0.06818586258863314, -0.10438742471937805, 0.11737600308715014, -0.024694232369897626, 0.0004920981789164458, -0.027524203781766086, 0.004855913186033435, -0.03087021287811742, -0.1266331449097836, 0.011099928313684565, -0.007358964441690282]

They're all of shape (256,) and have dtype=float

Answer 1

The simplest solution would be to check that the leading character is not a space, and to replace newline values. If you replace them with something like a pipe character you can undo this later when you read the values back from storage.

What's the content of the numpy array? It's probably better to create your own metadata structure that does what you want rather than dumping it as a string under one tag.

metadata = np.array2string(array).replace('\n','|')
if metadata[0] != ' ':
     block_blob_service.set_blob_metadata(settings.blob_container,file_name,metadata={'f': metadata})