CODEWITHSUNDEEP
solr6
poornima
poornima

Reputation: 11

How to add custom delimiter to a csv file while indexing in apache solr

I am new to apache solr and I am currently working with the version solr-6.4.0 I am trying to index a csv file in solr. Instead of using the default delimiter "," I would like to use a custom delimiter for example "_1ComPaaS1_". To achieve this what do I need to do?

I am working on a project and in need of some quick help.

This is the kind of response I am getting while I try to index a csv file with my custom delimiter.

{
  "responseHeader":{
    "status":0,
    "QTime":0,
    "params":{
      "q":"*:*",
      "indent":"on",
      "wt":"json",
      "_":"1561032630455"}},
  "response":{"numFound":9,"start":0,"docs":[
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_330i_sedan__1CoMPaaS1__2019 330i Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC_MANUAL__1CoMPaaS1__REAR_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"8e7337e9-3b93-4e5e-8c20-b7e6a8545042",
        "_version_":1636916417971355648},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_330i_xdrive_sedan__1CoMPaaS1__2019 330i xDrive Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC_MANUAL__1CoMPaaS1__ALL_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"e371eacf-e1da-4e9b-b02e-61ed972d2c97",
        "_version_":1636916418053144576},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_530i_sedan__1CoMPaaS1__2019 530i Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__REAR_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"6297428a-3cb5-43ff-90a3-57e86d35a7c9",
        "_version_":1636916418069921792},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_530i_xdrive_sedan__1CoMPaaS1__2019 530i xDrive Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__ALL_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"b1af091c-8c77-4190-8d88-bec13888583a",
        "_version_":1636916418069921793},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_530e_iperformance__1CoMPaaS1__2019 530e iPerformance__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__REAR_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"8cb1f622-cd59-43d4-8edc-0eccde6b4cf4",
        "_version_":1636916418069921794},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_530e_xdrive_iperformance__1CoMPaaS1__2019 530e xDrive iPerformance__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__ALL_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"4c5ffc24-a792-472f-ab3d-31defc6586d9",
        "_version_":1636916418069921795},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_540i_sedan__1CoMPaaS1__2019 540i Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__REAR_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"f7483583-e1cf-49d6-8125-0c5709da5a95",
        "_version_":1636916418069921796},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_540i_xdrive_sedan__1CoMPaaS1__2019 540i xDrive Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__ALL_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"ad1f7b3e-f692-4f9f-aecf-f9b88dd49edc",
        "_version_":1636916418069921797},
      {
        "productId__1CoMPaaS1__displayName__1CoMPaaS1__productType__1CoMPaaS1__partType__1CoMPaaS1__fuelType__1CoMPaaS1__transmissionType__1CoMPaaS1__driveTrainType__1CoMPaaS1__bodyType":["2019_m550i_xdrive_sedan__1CoMPaaS1__2019 M550i xDrive Sedan__1CoMPaaS1__AUTO__1CoMPaaS1__AUTO__1CoMPaaS1__PREMIUM_GAS__1CoMPaaS1__AUTOMATIC__1CoMPaaS1__ALL_WHEEL_DRIVE__1CoMPaaS1__SEDAN"],
        "id":"d52fda78-babc-4c79-9e16-1cd0401b976e",
        "_version_":1636916418069921798}]
  }}

Upvotes: 1

Views: 4198

Answers (1)

radhika garg
radhika garg

Reputation: 23

curl 'http://localhost:8983/solr/mycollection/update/csv?commit=true&separator=_1ComPaaS1_&escape=\&stream.file=csvFile.csv'

The newly introduced parameters:

  • commit – set to true. You’ll be glad you did.
  • separator – (reminds me of the Sade song Smooth Separator) this is where you can add tabs or pipes or anything else that strikes your fancy as a delimiter. Yes, you must encode it so a tab becomes 1ComPaaS1. No need to ask.
  • escape – if you are using a separator that is NOT a regular character (how boring) you have to tell the CSV handler to escape the separator so it is used properly. Without it the \t will be treated as a t and that would ruin the after-party.

Upvotes: 0

Related Questions