CODEWITHSUNDEEP
javaapache-flinkflink-streaming
monstereo
monstereo

Reputation: 940

Can not read _metadata for StateBackend Flink

I have just created simple flink application and enabled the checkpointing. I see that checkpointing files are created but I can not read them. File includes invalid characters. How can I convert these files to readable format? (Any solution for any language or online way is okey for me)

Here is the checkpoint folder structure

checkpoint_folder -> chk-23 -> _metadata
// _metadata file content:
L�$4c464d1a-1e37-47aa-957e-02f587343bbd��
V��������C�Corg.apache.flink.api.java.typeutils.runtime.TupleSerializerSnapshot���
������������������������������������������������������������������������������������������� �������

When I opened with sublime text:

4960 672d 0000 0002 0000 0000 0000 0122
0000 0000 0000 0004 20ba 6b65 f974 81d5
5700 70de 90e4 e791 0000 000c 0000 0080
0000 0001 0000 000c 0000 0000 ffff ffff
ffff ffff 0000 0000 0000 0001 0400 0000
0100 0673 706c 6974 7300 0000 0000 0100
2465 3165 3133 6662 342d 3838 3061 2d34
3063 392d 3965 6539 2d39 3465 3262 6465

public static void main(String[] args) throws Exception {
        final String checkpointPath="file:///to/path/checkpoint_folder";
        // set up the streaming execution environment
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironment();

        env.setStateBackend(new MemoryStateBackend(checkpointPath, null));
        env.enableCheckpointing(1000);
}

Upvotes: 2

Views: 1147

Answers (2)

정민철
정민철

Reputation: 21

You can use the deserialize method in the Metadata Serializer. Note that if "_metadata" data is saved as a file, additional HEADER_MAGIC_NUMBER and MetadataSerializers version information will be saved, so you need to add a code.

The sample code can be found in the link below.

https://ngela.tistory.com/49

Upvotes: 1

David Anderson
David Anderson

Reputation: 43499

If you want to either inspect or modify checkpoint files, the State Processor API is the recommended way to do this.

ReadRidesAndFaresSnapshot is a simple example that can read checkpoints written by RidesAndFaresSolution.

Upvotes: 2

Related Questions