jeremywat
jeremywat

Reputation: 1045

Custom jinja filter using python pandas is returning only sometimes returning correct information?

I'm building a site with the Material theme for MkDocs. I've added the following custom filter:

import pandas
def csv_to_html(csv_path):
    return pandas.read_csv(csv_path).to_html()

I'm trying to find a CSV file based on the metadata of my MD document. This is what my document (doc_12345.md) looks like:

---
title: myDocument
doc_number: doc_12345
---

I want to parse the file doc_12345.csv. The Jinja filters seem to be acting very erratically. I'm having a few different issues:

First problem

The filter only seems to be valid if I use the pipe (|). This fails:

{{ csv_to_html("data/doc_12345.csv") }}

Error:

jinja2.exceptions.UndefinedError: 'csv_to_html' is undefined

This works:

{{ "data/doc_12345.csv" | csv_to_html }}

Second problem

The page metadata is only found in some situations and not others. Referencing this documentation, I'm accessing the doc_number attribute with page.meta.doc_number. Using it on its own seems to work, see examples below:

{{ page.meta }}
# { 'title': 'myDocument', 'doc_number': 'doc_12345' }
{{ page.meta.doc_number }}
# doc_12345

But concatenating strings doesn't:

{{ "data/" + page.meta.doc_number + ".csv" }}

jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'doc_number'

Strangely enough, if I use default() it's able to find the doc_number attribute:

{{ "data/" + (page.meta.doc_number | default("random text")) + ".csv" }}
# data/doc_12345.csv

Why would it work when sent to a filter but not by itself?

Third problem

Once I pass that value to my csv_to_html filter, the doc_number disappears:

{{ "data/" + (page.meta.doc_number | default("random text")) + ".csv" | csv_to_html }}

FileNotFoundError: [Errno 2] No such file or directory: 'data/.csv'

I also tried setting it as a variable:

{% set csv_path = "data/" + (page.meta.doc_number | default("random text")) + ".csv" %}
{{ csv_path }}
# data/doc_12345.csv
{{ csv_path | csv_to_html }}
# FileNotFoundError: [Errno 2] No such file or directory: 'data/.csv'

What's even weirder is if I modify csv_to_html to simply return the input, the path is fine:

import pandas
def csv_to_html(csv_path):
    return csv_path
{{ csv_path | csv_to_html }}
# data/doc_12345.csv

So the csv_path string works fine until it gets to pandas.read_csv(), but it only has a problem if it was assembled using page.meta.doc_number. When I pass the string literal "data/doc_12345.csv" to csv_to_html, it converts the CSV to HTML just fine.

This makes no sense to me whatsoever. What's going on here?

Edit to add minimum reproducible example:

mkdocs.yml

theme:
  name: material
  custom_dir: templates
plugins:
  - mkdocs-simple-hooks:
      hooks:
        on_env: "modules.hooks:on_env"

modules/hooks.py

import pandas

def csv_to_html(csv_path):
    return pandas.read_csv(csv_path).to_html(index=False)

def on_env(env, config, **kwargs):
    # Add the filters
    env.filters['csv_to_html'] = csv_to_html

data/doc_12345.csv

ColumnA,ColumnB,ColumnC
ValueA,ValueB,ValueC

docs/doc_12345.md

---
title: myDocument
doc_number: doc_12345
---
# Hello world

templates/main.html

{% extends "base.html" %}
{% block content %}
{{ ("data/" + page.meta.doc_number + ".csv") | csv_to_html }}
{{ page.content }}
{% endblock %}

Upvotes: 0

Views: 35

Answers (0)

Related Questions