CODEWITHSUNDEEP
solr
Prathamesh Shetye
Prathamesh Shetye

Reputation: 11

solr data import handler not able to index data

Data config.xml is as follows

<dataConfig>  
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/solrdata" user="root" password="root"  />  
    <document name="cars">  

        <entity name="car" query="SELECT color FROM solrdata.car ">  

          <field column="color" name="color" />  

        </entity>  

    </document>  
</dataConfig>

schema.xml is as follsws

field name="color" type="string"  indexed="true" stored="true" />

i checked in debug mode its getting data but not able to process it

output of debug mode is as follows:

{
  "responseHeader": {
    "status": 0,
    "QTime": 312
  },
  "initArgs": [
    "defaults",
    [
      "config",
      "data-config.xml"
    ]
  ],
  "command": "full-import",
  "mode": "debug",
  "documents": [
    {
      "COLOR": [
        "red"
      ]
    },
    {
      "COLOR": [
        "silver"
      ]
    },
    {
      "COLOR": [
        "oii"
      ]
    }
  ],
  "verbose-output": [],
  "status": "idle",
  "importResponse": "",
  "statusMessages": {
    "Total Requests made to DataSource": "1",
    "Total Rows Fetched": "3",
    "Total Documents Skipped": "0",
    "Full Dump Started": "2013-03-07 15:49:14",
    "Total Documents Processed": "0",
    "Total Documents Failed": "3",
    "Time taken": "0:0:0.281"
  },
  "WARNING": "This response format is experimental.  It is likely to change in the future."
}

Upvotes: 1

Views: 5898

Answers (3)

hossein
hossein

Reputation: 9

Some specific fields, including version, should not be in the list of your fields. (fl="*" this is contains all fields) Try each field individually with the corresponding id

fl="id,color"

Upvotes: 0

Andreas Dolk
Andreas Dolk

Reputation: 114837

I can read from the import handler response:

"Total Documents Failed": "3"

Looks to me as if your query has some problems or if the loaded schema doesn't "match" the DIH ouput. A <uniqueKey> field is not required although highly recommended. But the missing unique key declaration should lead to that error.

Have a look at the "logging" page on the admin console. If the data import handler has problems with the query then you'll find a log entry there.

And don't forget to refresh the schema and the DIH config file if you've applied any changes while the solr instance is running.

Upvotes: 3

Mavellin
Mavellin

Reputation: 665

You will have a uniqueKey for each document to identify it uniquely (Can be considered similiar to the PrimaryKey in Databases).

Modify your entity in data-config.xml as follows:

<entity name="car" query="SELECT color,id FROM solrdata.car ">  

  <field column="id" name="id" />  
  <field column="color" name="color" />  

</entity>

Note: The field id is your primaryKey for the table car.

In your schema.xml file, add the following line,

<field name="id" type="string" indexed="true" stored="true" required="true" />

Also, make sure that the following text,

<uniqueKey>id</uniqueKey>

in your schema.xml is not commented out.

Now, restart your Solr Web-application and do a Full-import.

Upvotes: 3

Related Questions