Django Haystack - unable to build solr schema

Question

I got following error when I tried to build solr schema:

(my_env) pecan@tux ~/Documents/Django/mysite $ python manage.py build_solr_schema
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/haystack/management/commands/build_solr_schema.py", line 29, in handle
    schema_xml = self.build_template(using=using)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/haystack/management/commands/build_solr_schema.py", line 57, in build_template
    return t.render(c)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/template/backends/django.py", line 64, in render
    context = make_context(context, request, autoescape=self.backend.engine.autoescape)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/template/context.py", line 287, in make_context
    raise TypeError('context must be a dict rather than %s.' % context.__class__.__name__)
TypeError: context must be a dict rather than Context.

Maybe these informations will be useful:

mysite/settings.py file:

"""
Django settings for mysite project.

Generated by 'django-admin startproject' using Django 1.11.5.

For more information on this file, see
https://docs.djangoproject.com/en/1.11/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/1.11/ref/settings/
"""

import os

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '****'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = []

SITE_ID = 1

# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sites',
    'django.contrib.sitemaps',
    'blog',
    'taggit',
    'haystack',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'mysite.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'mysite.wsgi.application'


# Database
# https://docs.djangoproject.com/en/1.11/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}


# Password validation
# https://docs.djangoproject.com/en/1.11/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


# Internationalization
# https://docs.djangoproject.com/en/1.11/topics/i18n/

LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_L10N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.11/howto/static-files/

STATIC_URL = '/static/'

HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
        'URL': 'http://127.0.0.1:8983/solr/blog'
    },
}

blog/search_indexes.py file:

from haystack import indexes
from .models import Post

class PostIndex(indexes.SearchIndex, indexes.Indexable):
    text = indexes.CharField(document=True, use_template=True)
    publish = indexes.DateTimeField(model_attr='publish')

    def get_model(self):
        return Post

    def index_queryset(self, using=None):
        return self.get_model().published.all()

blog/templates/search/indexes/blog/post_text.txt file:

{{ object.title }}
{{ object.tags.all|join:", " }}
{{ object.body }}

I'm using Apache Solr 4.10.4, Python 3.4.5 and Django 1.11.5. When I tried to import haystack in Python console I was getting below error:

>>> import haystack
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/haystack/__init__.py", line 10, in <module>
    from haystack.constants import DEFAULT_ALIAS
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/haystack/constants.py", line 10, in <module>
    ID = getattr(settings, 'HAYSTACK_ID_FIELD', 'id')
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/conf/__init__.py", line 56, in __getattr__
    self._setup(name)
  File "/home/pecan/Documents/Django/my_env/lib/python3.4/site-packages/django/conf/__init__.py", line 39, in _setup
    % (desc, ENVIRONMENT_VARIABLE))
django.core.exceptions.ImproperlyConfigured: Requested setting HAYSTACK_ID_FIELD, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.in Python console

I'm counting for help.

Answer 1

The version of Haystack in my Django project was wrong. I used django-haystack 2.6.1 package but it has a problem with django template context passing. This version passed in context a Context object instead of dictionary. More details: https://github.com/django-haystack/django-haystack/pull/1504/commits/295584314e19a191a59450e053b21809adceca2a.

haystack/management/commands/build_solr_schema.py in django-haystack 2.6.1:

     content_field_name, fields = backend.build_schema(
         connections[using].get_unified_index().all_searchfields()
     )
     return Context({
         'content_field_name': content_field_name,
         'fields': fields,
         'default_operator': constants.DEFAULT_OPERATOR,
         'ID': constants.ID,
         'DJANGO_CT': constants.DJANGO_CT,
         'DJANGO_ID': constants.DJANGO_ID,
     })

 def build_template(self, using):
     t = loader.get_template('search_configuration/solr.xml')

haystack/management/commands/build_solr_schema.py in django-haystack 2.7.dev0:

     content_field_name, fields = backend.build_schema(
         connections[using].get_unified_index().all_searchfields()
     )

     return {
         'content_field_name': content_field_name,
         'fields': fields,
         'default_operator': constants.DEFAULT_OPERATOR,
         'ID': constants.ID,
         'DJANGO_CT': constants.DJANGO_CT,
         'DJANGO_ID': constants.DJANGO_ID,
     }

 def build_template(self, using):
     t = loader.get_template('search_configuration/solr.xml')

I had to uninstall django-haystack 2.6.1 and install a newer version using commands:

pip uninstall django-haystack
pip install django-haystack==2.7.dev0

I also solved import error. In this case I just added line HAYSTACK_ID_FIELD = 1 to settings.py file and I set required environment variable DJANGO_SETTINGS_MODULE to value mysite.settings.

I executed following commands after editing settings.py:

(my_env) pecan@tux ~/Documents/Django/mysite $ DJANGO_SETTINGS_MODULE="mysite.settings"
(my_env) pecan@tux ~/Documents/Django/mysite $ echo $DJANGO_SETTINGS_MODULE 
mysite.settings
(my_env) pecan@tux ~/Documents/Django/mysite $ DJANGO_SETTINGS_MODULE python
bash: DJANGO_SETTINGS_MODULE: command not found
(my_env) pecan@tux ~/Documents/Django/mysite $ python
Python 3.4.5 (default, Sep 17 2017, 18:19:56) 
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import haystack
>>> 

Now django-haystack is working properly!

Answer 2

Debugging Haystack

There are some common problems people run into when using Haystack for the first time.

“No module named haystack.”

This problem usually occurs when first adding Haystack to your project.

• Are you using the haystack directory within your django-haystack checkout/install?

• Is the haystack directory on your PYTHONPATH? Alternatively, is haystack symlinked into your project?

• Start a Django shell (./manage.py shell) and try import haystack. You may receive a different, more descriptive error message.

• Double-check to ensure you have no circular imports. (i.e. module A tries importing from module B which is trying to import from module A.)

“No results found.” (On the web page)

Several issues can cause no results to be found. Most commonly it is either not running a rebuild_index to populate your index or having a blank document=True field, resulting in no content for the engine to search on.

• Do you have a search_indexes.py located within an installed app?

• Do you have data in your database?

• Have you run a ./manage.py rebuild_index to index all of your content?

• Try running ./manage.py rebuild_index -v2 for more verbose output to ensure data is being processed/inserted.

• Start a Django shell (./manage.py shell) and try:

from haystack.query import SearchQuerySet

sqs = SearchQuerySet().all()

sqs.count()

• You should get back an integer > 0. If not, check the above and reindex.

sqs[0] # Should get back a SearchResult object.

sqs[0].id # Should get something back like 'myapp.mymodel.1'.

sqs[0].text # ... or whatever your document=True field is.

• If you get back either u'' or None, it means that your data isn’t making it into the main field that gets searched. You need to check that the field either has a template that uses the model data, a model_attr that pulls data directly from the model or a prepare/prepare_FOO method that populates the data at index time.
• Check the template for your search page and ensure it is looping over the results properly. Also ensure that it’s either accessing valid fields coming back from the search engine or that it’s trying to access the associated model via the {{ result.object.foo }} lookup.

Source : http://django-haystack.readthedocs.io/en/v2.4.1/toc.html