Rails Hash.from_xml not giving expected results

Question

Trying to process some XML that comes from an application called TeleForm. This is form scanning software and it grabs the data and puts it into XML. This is a snippet of the XML



  
    00000022000000
    3
    
  

  
    00000022000001
    3

I've dealt with this in an other system, probably using a custom parser we wrote. I figured it would be no problem in Rails, but I was wrong.

Parsing this with Hash.from_xml or from Nokogiri does not give me the results I expected, I get:

{"Records"=>{"Record"=>[{"Field"=>["", {"id"=>"Criterion_1", "type"=>"number", "length"=>"2", "Value"=>"3"}, ""]},
 {"Field"=>["", {"id"=>"Criterion_1", "type"=>"number", "length"=>"2", "Value"=>"3"}, ""]}]}}

After spending way too much time on this, I discovered if I gsub out the type and length attributes, I get what I expected (even if it is wrong! I only removed on the first record node).

{"Records"=>{"Record"=>[{"Field"=>[{"id"=>"ImageFilename", "Value"=>"00000022000000"}, 
{"id"=>"Criterion_1", "type"=>"number", "length"=>"2", "Value"=>"3"}, {"id"=>"Withdrew"}]}, 
{"Field"=>["", {"id"=>"Criterion_1", "type"=>"number", "length"=>"2", "Value"=>"3"}, ""]}]}}

Not being well versed in XML, I assume this style of XML using type and length attributes is trying to convert to the data types. In that case, I can understand why the "Withdrew" attribute showed up as empty, but don't understand why the "ImageFilename" was empty - it is a 14 character string.

I've got the work around with gsub, but is this invalid XML? Would adding a DTD (which TeleForm should have provided) give me different results?

EDIT

I'll provide a possible answer to my own question with some code as an edit. The code follows some of the features in the one answer I did receive from Mark Thomas, but I decided against Nokogiri for the following reasons:

The xml is consistent and alway contains the same tags (/Records/Record/Field) and attributes.
There can be several hundred records in each XML file and Nokogiri seems a little slow with only 26 records
I figured out how to get Hash.from_xml to give me what I expected (does not like type="string", but only use the hash to populate a class.

An expanded version of the XML with one complete record



  
    00000022000000
    731192AIINSC
    AIINSC
    M  T  W  H  F  S
    3
    3
    3
    3
    3
    3
    3
    3
    3
    3
    0
    0
    0
    0
    0
    Y
    
    731192
    
    
    
    2292

I am only experimenting with a workflow prototype to replace an aging system written in 4D and Active4D. This area of processing TeleForms data was implemented as a batch operation and it still may revert to that. I am just trying to merge some of the old viable concepts in a new Rails implementation. The XML files are on a shared server and will probably have to be moved into the web root and then some trigger set to process to files.

I am still in the defining stage, but my module/classes to handle the InterviewForm is looking like this and may change (with little error trapping, still trying to get into testing and my Ruby is not as good as it should be after playing with Rails for about 5 years!):

module Teleform::InterviewForm

  class Form < Prawn::Document
    # Not relevant to this question, but this class generates the forms from a Fillable PDF template and 
    # relavant Model(s) data.
    # These forms, when completed are what is processsed by TeleForms and produces the xml.
  end

  class RateForms
    attr_accessor  :records, :results

    def initialize(xml_path)
      fields = []
      xml = File.read(xml_path)
      # Hash.from_xml does not like a type of "string"
      hash = Hash.from_xml(xml.gsub(/type="string"/,'type="text"'))
      hash["Records"]["Record"].each do |record|
        #extract the field form each record
        fields << record["Field"]
      end
      @records = []
      fields.each do |field|
        #build the records for the form
        @records << Record.new(field)
      end
      @results = rate_records
    end

    def rate_records
      # not relevant to the qustions but this is where the data is processed and a bunch of stuff takes place
      return "Any errors"
    end
  end


  class Record
    attr_accessor(*[:image_filename, :doc_id, :form_id, :availability, :criterion_1, :criterion_2, 
      :criterion_3, :criterion_4, :criterion_5, :criterion_6, :criterion_7, :criterion_8, 
      :criterion_9, :criterion_10, :criterion_11, :criterion_12, :criterion_13, :criterion_14, :criterion_15, 
      :day_training, :saturday_training, :citizen_stage_id, :no_show, :night_training, :withdrew, :job_stage_id, :direct_hire])

    def initialize(fields)
      fields.each do |field|
        if field["type"] == "number"
          try("#{field["id"].underscore.to_sym}=", field["Value"].to_i)
        else
          try("#{field["id"].underscore.to_sym}=", field["Value"])
        end
      end
    end
  end

end

Mark Thomas · Accepted Answer

Thanks for adding the additional information that this is a rating for an interviewee. Using this domain information in your code will likely improve it. You haven't posted any code, but generally using domain objects leads to more concise and more readable code. I recommend creating a simple class representing a Rating, rather than transforming data from XML to a data structure.

class Rating
  attr_accessor :image_filename, :criterion_1, :withdrew
end

Using the above class, here's one way to extract the fields from the XML using Nokogiri.

doc = Nokogiri::XML(xml)
ratings = []

doc.xpath('//Record').each do |record|
    rating = Rating.new
    rating.image_filename = record.at('Field[@id="ImageFilename"]/Value/text()').to_s
    rating.criterion_1 = record.at('Field[@id="Criterion_1"]/Value/text()').to_s
    rating.withdrew = record.at('Field[@id="Withdrew"]/Value/text()').to_s
    ratings << rating
end

Now, ratings is a list of Rating objects, each with methods to retrieve the data. This is a lot cleaner than delving into a deep data structure. You could even improve on the Rating class further, for example creating a withdrew? method that returns a true or false.

Rails Hash.from_xml not giving expected results

Answers (2)

Related Questions