CODEWITHSUNDEEP
clojure
Dale
Dale

Reputation: 599

Clojure working with records

I have a set of values in Clojure that I want to structure similar to record. I'm trying to figure out the best way of handling a set of those records.

So I have a for example a record:

(defrecord Link [page url])

Whats the best datastructure to hold a collection of these records that I can step through recursively, while continuously updating the collection?

Previously I've done this on a single value using sequence, then concat'ing new links on the end as i process them recursively. But now I want to hold more information about each link.

Edit For Clarity

I had previously been using maps, however I think I've been confusing myself by trying to use a nested map with like 

#{:rootlink "http://www.google.co.uk" :links nestedmapoflinks}

which is confusing me when i try to re curse through it.

Below is the code that I've been using, below is what currently works with a sequence of links but no other info about the link.

(defn get-links
[url]
(map :href (map :attrs (html/select (fetch-url url) [:a])))))

(defn process-links
[links]
(if (not (empty? links))
(do
  (if (not (is-working (first links)))
(do
  (println (str (first links) " is not working"))
  (recur (rest links)))
(do
  (println (str (first links) " is working"))
  (recur (concat (rest links) (get-links (first links)))))))))

I think I have to add each item into the map with 

{:rootlink "http://www.google.co.uk" :link "http://someurlontherootlinkpage.com"}

instead of trying to work with the nested map.

However, the reason I mentioned records, because I was struggling to merge two maps together using the first method of map creation. I'm still a bit confused about the best structure to use to recurse through the map.

Final Update

Ok, so after much wrangling I finally came up with this below piece of code which returns a seq of vectors made up of:

["root link address" "link"]
["http://www.google.co.uk" "http://www.google.co.uk/examplelink"]

Code:

(defn get-links
   [url]
   (map #(vector url %)(map :href (map :attrs (html/select (fetch-url url) [:a])))))

The code is now on my github available in my profile.

Upvotes: 2

Views: 402

Answers (1)

Ankur
Ankur

Reputation: 33637

I think you are getting confused between using Tree type structure or a flat structure. Lets say you have a list of links as a vector of maps:

[ {:root nil :link "A.COM"} {:root nil :link "B.COM"} ]

Now you map over it and using your get-link method you get:

[ [ {:root nil :link "A.COM"} {:root "A.COM" :link "Aa.COM"} {:root "A.COM" :link "Ab.COM"} ] [ {:root nil :link "B.COM"} {:root "B.COM" :link "Ba.COM"} {:root "B.COM" :link "Bb.COM"}] ]

Now you can call flatten on this result to get a flat list of link instead of nested map in vector.

You can repeat this process recursively till you exit condition met.

Upvotes: 1

Related Questions