CODEWITHSUNDEEP
matlabparallel-processingparformat-file
casparjespersen
casparjespersen

Reputation: 3840

Warning when saving to matfile in parallel environment

I am trying to process a time signal, which is split up into (many) smaller segments, and each segment ("chunk") is analyzed independently. This output, I want to save in a combined file.

My current solution works:

configFileName = 'config.mat';
config = matfile(configFileName,'Writable',true);
% some irrelevant stuff saved in config
config.chnk(1,N) = struct('var1',[],'var2',[],'var3',[]); % pre-allocating
clearvars config;

parfor i = 1:N
    config = matfile(configFileName,'Writable',true);
    chunk = process(data(:,i)); % data has previously been sliced
    config.chnk(1,i) = chunk;
end

Note: Output of process(data(:,i)); is of same type that the config.chnk is pre-allocated with. Now, while this seems to work (update: it does not), I get a warning by Matlab:

Warning: Unable to read some of the variables due to unknown MAT-file error.

In matlab.io.MatFile/genericWho (line 209) In matlab.io.MatFile/whos (line 309) In matlab.io.MatFile (line 422) In matfile (line 75) In parallel_function>make_general_channel/channel_general (line 929) In remoteParallelFunction (line 38)

In my case, I do not read anything inside parfor so I could just omit the warning in my script by using evalc when creating the file object, but does anyone know why this warning appears, and how to avoid it properly?

Update: It does not work, actually. The exact variable I am writing to is invalid, and whenever I try to read from it, I get an error.

When the parallel process was finished, I got the following error:

Error using matlab.io.MatFile/whos (line 311) Could not open /Users/casparjespersen/ardetector/matlab/data/EMD/A0007_4/EEG.mat as a valid MAT-file.

Error in matlab.io.MatFile (line 422) varInfo = whos(obj);

Error in matfile (line 75) mf = matlab.io.MatFile(varargin{:});

Error in HHSA_BD_gen_emd (line 104) parfor windowIdx = 1:size(windowChunks,2)

Upvotes: 1

Views: 670

Answers (1)

casparjespersen
casparjespersen

Reputation: 3840

As @user3528438 pointed out, this is due to the workers writing to the file at the same time. I chose to re-design my script to avoid writing from within parfor.

Upvotes: 1

Related Questions