CODEWITHSUNDEEP
web-scrapingkimono
marchello
marchello

Reputation: 2136

Nested API structure with Kimono

There is a site that I want to convert to an API with Kimono and it has the following structure (I mean visually, not markup-vise):

CATEGORY 1:
 Product 1: PRICE
 Product 2: PRICE
 Product 3: PRICE
 ...

CATEGORY 2:
 Product 1: PRICE
 Product 2: PRICE
 Product 3: PRICE
 ...

etc... And I want the API to reflect that hierarchy, so it would be something like this:

{
  "CATEGORY 1": {
    "Product 1": {
      "price": "$"
    },
    "Product 2": {
      "price": "$"
    },
    "Product 3": {
      "price": "$"
    }
  },
  "CATEGORY 2": {
    "Product 1": {
      "price": "$"
    },
    "Product 2": {
      "price": "$"
    },
    "Product 3": {
      "price": "$"
    }
  }
}

The problem is that the site's markup doesn't show this hierarchy (Products aren't nested inside the Categories):

<h3>CATEGORY 1</h3>
<div class="product">
  <div>
    <div>
    <h4>
      <div>Product 1</div>
    </h4>
    <p>Price</p>
</div>
<div class="product">
  <div>
    <div>
    <h4>
      <div>Product 2</div>
    </h4>
    <p>Price</p>
</div>
<h3>CATEGORY 2</h3>
<div>
  <div>
    <div>
    <h4>
      <div>Product 1</div>
    </h4>
    <p>Price</p>
</div>
<div class="product">
  <div>
    <div>
    <h4>
      <div>Product 2</div>
    </h4>
    <p>Price</p>
</div>

No matter what I do, I always get something like this:

{
  "collection1": [
    {
      "property1": "Category 1",
      "property4": "Product 1",
      "property5": "price"
    },
    {
      "property1": "Category 2",
      "property4": "Product 1",
      "property5": "price"
    }
  ]
}

Is it possible to achieve this?

Upvotes: 0

Views: 67

Answers (1)

AndriuZ
AndriuZ

Reputation: 656

If CSS selectors won't work probably it's possible if you will try regexp for CATEGORY 1, CATEGORY 2 - can't help more without knowing page url for testing

Also You can try to put Cat1, Cat2 data in different Collections, separated with CSS:

...> h3

and regexp for middle part:

CATEGORY 1

Upvotes: 2

Related Questions