Reputation: 185
I am trying to read a file, line by line in OCaml. Each line in the file represents a string I want to parse, in the correct format expected by the Parsing tool. I am saving each line in a list structure.
I an finding an issue parsing the string contained in each element of the list. I am using OCamllex and Menhir as parsing tools.
If I try to use print_string
to print the contents of the list at every element, I get the correct file contents.
If I try to pass a string
that I defined within the program to the function, then I get the desired output.
However, if I try to parse the string which I have just read from the file, I get an error: Fatal error: exception Failure ("lexing empty token")
Note: that all of this has been tested against the same string.
Here is a snippet of the code:
let parse_mon m = Parser.monitor Lexer.token (from_string m)
let parse_and_print (mon: string)=
print_endline (print_monitor (parse_mon mon) 0)
let get_file_contents file =
let m_list = ref [] in
let read_contents = open_in file in
try
while true; do
m_list := input_line read_contents :: !m_list
done; !m_list
with End_of_file -> close_in read_contents; List.rev !m_list
let rec print_file_contents cont_list = match cont_list with
| [] -> ()
| m::ms -> parse_and_print m
let pt = print_file_contents (get_file_contents filename)
Upvotes: 3
Views: 674
Reputation: 1586
I agree with kne, hard to say without seeing the file, but what you can do is trying to isolate the line that causes the trouble by doing :
let rec print_file_contents cont_list =
match cont_list with
| [] -> ()
| m::ms ->
try parse_and_print m
with Failure _ -> print_string m
Upvotes: 1
Reputation: 8390
Ocamllex throws an exception Failure "lexing: empty token"
when a text in the stream doesn't match any scanner pattern. Therefore, you will need to match with "catch-all" patterns such as .
, _
, or eof
.
{ }
rule scan = parse
| "hello" as w { print_string w; scan lexbuf }
(* need these two for catch-all *)
| _ as c { print_char c; scan lexbuf }
| eof { exit 0 }
Upvotes: 5
Reputation: 1418
Without seeing your grammar and file I can only offer a wild guess: Could it be that the file contains an empty line at the end? Depending on the .mll
that might result in the error you see. The reason being that get_file
appends new lines to the front of the list and print_file_contents
only looks at the head of that list.
Upvotes: 1