How to map multiple yaml files to frontmatter in Gatsby

Question

The problem:

I am having trouble mapping a second .yaml file to a second markdown frontmatter field in Gatsby. The project is for a magazine, with the first yaml file storing details about each author and the second file storing details about each issue.

The author.yaml file worked like it was supposed to, and I can query it across every template and page, but the issue.yaml file only works in two places: templates where I specifically passed its data as page context in gatsby-node, and (inexplicably) for markdown files with the frontmatter "featured" field === true. For mapped arrays in which "featured" is not present, or when it's false, every query that should pull data from the issue.yaml gives "TypeError: Cannot read property 'id' of null."

My suspicion is that it's because the issue field on my markdown files isn't the first field (which is author, and already mapped to the author.yaml file). As far as I can tell, I've implemented both yaml files exactly the same.

Things I've tried:

I've tried to query the allAuthorYaml and allIssueYaml nodes that I think are supposed to be automatically generated, but I couldn't get either of them to work. I also tried to create a schema customization in gatsby-node, but I couldn't figure out how to adapt the tutorial to define the nodes I needed globally (way over my head). I read somewhere that I could import the yaml data directly with import YAMLData from "../data/issue.yaml" and then create an array directly, but it also gave me null/undefined errors.

I am a hobbyist, and don't have a background in programming at all, which is probably a big part of the problem. Still, I'd be grateful for any help anyone might have. And if anyone happens to spot any other places my code needs improving, definitely let me know!

The starter I used was https://www.gatsbyjs.org/starters/JugglerX/gatsby-serif-theme/.
My repository is at https://github.com/ljpernic/HQ3.1/tree/HQhelp.

Thanks again!

---

My gatsby-config:

module.exports = {
  siteMetadata: {
    title: 'Haven Quarterly',
    description: 'a magazine of science fiction and fantasy',
    submit: {
      phone: 'XXX XXX XXX',
      email: 'havenquarterly@gmail.com',
    },
    menuLinks: [
      {
        name: 'Fiction',
        link: '/fiction',
      },
      {
        name: 'Non-fiction',
        link: '/non-fiction',
      },
      {
        name: 'Letters from the Future',
        link: '/future',
      },
      {
        name: 'Full Issues',
        link: '/fullissues',
      },
      {
        name: 'Contributors',
        link: '/contributors',
      },
      {
        name: 'About',
        link: '/about',
      },
      {
        name: 'Support',
        link: '/support',
      },
      {
        name: 'Submit',
        link: '/submit',
      },
    ],
  },
  plugins: [
    'gatsby-plugin-sass',
    'gatsby-transformer-json',
    'gatsby-transformer-remark',
    'gatsby-plugin-react-helmet',
    `gatsby-transformer-sharp`, 
    `gatsby-plugin-sharp`,
    `gatsby-plugin-catch-links`,
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages`,
        name: 'pages',
      },
    },
/*    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/posts`,
        name: 'posts',
      },
    },*/
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/data`,
        name: 'data',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/images`,
        name: 'images',
      },
    },
    {
      resolve: "gatsby-transformer-remark",
      options: {
        plugins: [
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 1200,
              quality: 95, 
            },
          },
        ], 
      },
    },
    {
      resolve: 'gatsby-plugin-google-analytics',
      options: {
        trackingId: guid ? guid : 'UA-XXX-1',
        // Puts tracking script in the head instead of the body
        head: false,
      },
    },
  `gatsby-transformer-yaml`,
  ],
  mapping: {
    // 3. map author to author.yaml
    "MarkdownRemark.frontmatter.author": `AuthorYaml`,
    "MarkdownRemark.frontmatter.issue": `IssueYaml`,
  },
};

My gatsby-node:

const _ = require('lodash');
const fs = require("fs")
const yaml = require("js-yaml")

// Create pages from markdown files
exports.createPages = ({ graphql, actions }) => {
  const { createPage } = actions;
  const ymlDoc = yaml.safeLoad(fs.readFileSync("./src/data/author.yaml", "utf-8"))
  const ymlIssueDoc = yaml.safeLoad(fs.readFileSync("./src/data/issue.yaml", "utf-8"))
  return new Promise((resolve, reject) => {
    resolve(
      graphql(
        `
          query {
            fictionarchive: allMarkdownRemark(
              filter: { fileAbsolutePath: { regex: "/fiction/" } }
              sort: { fields: [frontmatter___date], order: DESC }
            ) {
              edges {
                node {
                  id
                  frontmatter {
                    category
                    featured
                    path
                    title
                    date(formatString: "DD MMMM YYYY")
                  }
                  excerpt
                }
              }
            }

            nonfictionarchive: allMarkdownRemark(
              filter: { fileAbsolutePath: { regex: "/non-fiction/" } }
              sort: { fields: [frontmatter___date], order: DESC }
            ) {
              edges {
                node {
                  id
                  frontmatter {
                    category
                    featured
                    path
                    title
                    date(formatString: "DD MMMM YYYY")
                  }
                  excerpt
                }
              }
            }

            futurearchive: allMarkdownRemark(
              filter: { fileAbsolutePath: { regex: "/letters/" } }
              sort: { fields: [frontmatter___date], order: DESC }
            ) {
              edges {
                node {
                  id
                  frontmatter {
                    category
                    featured
                    path
                    title
                    date(formatString: "DD MMMM YYYY")
                  }
                  excerpt
                }
              }
            }

            issuesarchive: allMarkdownRemark(
              filter: { fileAbsolutePath: { regex: "/" } }
              sort: { fields: [frontmatter___date], order: DESC }
            ) {
              edges {
                node {
                  id
                  frontmatter {
                    category
                    featured
                    path
                    title
                    date(formatString: "DD MMMM YYYY")
                  }
                  excerpt
                }
              }
            }

            authorarchive: allMarkdownRemark(
              filter: { fileAbsolutePath: { regex: "/" } }
              sort: { fields: [frontmatter___date], order: DESC }
            ) {
              edges {
                node {
                  id
                  frontmatter {
                    category
                    featured
                    path
                    title
                    date(formatString: "DD MMMM YYYY")
                  }
                  excerpt
                }
              }
            }

          }
        `,
      ).then((result) => {
        ymlDoc.forEach(element => {
          createPage({
            path: element.idpath,
            component: require.resolve("./src/templates/eachauthor.js"),                     /*creates INDIVIDUAL AUTHOR PAGES*/
            context: {
              idname: element.id,
              bio: element.bio,
              twitter: element.twitter,
              picture: element.picture,
              stories: element.stories,
            },
          });
        });
        ymlIssueDoc.forEach(element => {
          createPage({
            path: element.idpath,
            component: require.resolve("./src/templates/eachissue.js"),                      /*creates INDIVIDUAL ISSUE PAGES*/
            context: {
              issueidname: element.id,
              text: element.text,
              currentcover: element.currentcover,
              artist: element.artist,
              artistbio: element.artistbio,
              artistimage: element.artistimage,
            },
          });
        });
        result.data.fictionarchive.edges.forEach(({ node }) => {
          const component = path.resolve('src/templates/eachpost.js');                      /*creates INDIVIDUAL FICTION PAGES*/
          createPage({
            path: node.frontmatter.path,
            component,
            context: {
              id: node.id,
            },
          });
        });
        result.data.nonfictionarchive.edges.forEach(({ node }) => {
          const component = path.resolve('src/templates/eachpost.js');                      /*creates INDIVIDUAL NON-FICTION PAGES*/
        createPage({
          path: node.frontmatter.path,
          component,
          context: {
            id: node.id,
          },
        });
      });
        result.data.futurearchive.edges.forEach(({ node }) => {
          const component = path.resolve('src/templates/eachpost.js');                      /*creates INDIVIDUAL LETTER PAGES*/
          createPage({
            path: node.frontmatter.path,
            component,
            context: {
              id: node.id,
            },
          });
        });
        result.data.issuesarchive.edges.forEach(({ node }) => {
          const component = path.resolve('src/templates/eachpost.js');                      /*creates INDIVIDUAL ISSUE PAGES; change template to change every issue page*/
          createPage({
            path: node.frontmatter.path,
            component,
            context: {
              id: node.id,
            },
          });
        });
        const FICposts = result.data.fictionarchive.edges                                   /*creates FICTION LIST PAGES*/
        const FICpostsPerPage = 10
        const FICnumPages = Math.ceil(FICposts.length / FICpostsPerPage)
        Array.from({ length: FICnumPages }).forEach((_, i) => {
          createPage({
            path: i === 0 ? `/fiction` : `/fiction/${i + 1}`,
            component: path.resolve('src/templates/fictionarchive.js'),
            context: {
              limit: FICpostsPerPage,
              skip: i * FICpostsPerPage,
              FICnumPages,
              FICcurrentPage: i + 1,
            },
          });
        });
        const NONFICposts = result.data.nonfictionarchive.edges                             /*creates NON-FICTION LIST PAGES*/
        const NONFICpostsPerPage = 10
        const NONFICnumPages = Math.ceil(NONFICposts.length / NONFICpostsPerPage)
        Array.from({ length: NONFICnumPages }).forEach((_, i) => {
          createPage({
            path: i === 0 ? `/non-fiction` : `/non-fiction/${i + 1}`,
            component: path.resolve('src/templates/nonfictionarchive.js'),
            context: {
              limit: NONFICpostsPerPage,
              skip: i * NONFICpostsPerPage,
              NONFICnumPages,
              NONFICcurrentPage: i + 1,
            },
          });
        });
        const FUTposts = result.data.futurearchive.edges                                   /*creates LETTERS FROM THE FUTURE LIST PAGES*/
        const FUTpostsPerPage = 10
        const FUTnumPages = Math.ceil(FUTposts.length / FUTpostsPerPage)
        Array.from({ length: FUTnumPages }).forEach((_, i) => {
          createPage({
            path: i === 0 ? `/future` : `/future/${i + 1}`,
            component: path.resolve('src/templates/futurearchive.js'),
            context: {
              limit: FUTpostsPerPage,
              skip: i * FUTpostsPerPage,
              FUTnumPages,
              FUTcurrentPage: i + 1,
            },
          });
        });
        const FULLposts = result.data.issuesarchive.edges                                   /*creates ISSUES LIST PAGES*/
        const FULLpostsPerPage = 10
        const FULLnumPages = Math.ceil(FULLposts.length / FULLpostsPerPage)
        Array.from({ length: FULLnumPages }).forEach((_, i) => {
          createPage({
            path: i === 0 ? `/fullissues` : `/fullissues/${i + 1}`,
            component: path.resolve('src/templates/issuesarchive.js'),
            context: {
              limit: FULLpostsPerPage,
              skip: i * FULLpostsPerPage,
              FULLnumPages,
              FULLcurrentPage: i + 1,
            },
          });
        });
        const AUTposts = result.data.authorarchive.edges
        const AUTpostsPerPage = 10
        const AUTnumPages = Math.ceil(AUTposts.length / AUTpostsPerPage)
        Array.from({ length: AUTnumPages }).forEach((_, i) => {
          createPage({
            path: i === 0 ? `/contributors` : `/contributors/${i + 1}`,
            component: path.resolve('src/templates/authorarchive.js'),
            context: {
              limit: AUTpostsPerPage,
              skip: i * AUTpostsPerPage,
              AUTnumPages,
              AUTcurrentPage: i + 1,
            },
          });
        });
        resolve();
      }),
    );
  });
};

My front page (shortened):

import { graphql, withPrefix, Link } from 'gatsby';
import Image from "gatsby-image";
import Helmet from 'react-helmet';
import SEO from '../components/SEO';
import Layout from '../layouts/index';

const Home = (props) => {                                                     //THIS SETS THE FRONT PAGE, including featured story, latest stories, and latest issues
  const json = props.data.allFeaturesJson.edges;
  const posts = props.data.allMarkdownRemark.edges;

  return (
    <Layout bodyClass="page-home">
      <SEO title="Home" />
      <Helmet>
        <meta
          name="Haven Quarterly"
          content="A Magazine of Science Fiction and Fantasy"
        />
      </Helmet>

                                                                                          {/*FEATURED*/}
      <div className="intro pb-1">
        <div className="container">
          <div className="row2 justify-content-start">
            <div className="grid-container pt-2">
              <div className="wide">
                <div className="col-12">
                  <Link to="/featured">
                      <h4>Featured Story</h4>
                  </Link>
                  <hr />
                </div>
                {posts
                  .filter(post => post.node.frontmatter.featured === true)                       /*This looks at only the md file with featured: true*/
                  .map(({ node: post }) => {
                    return (
                      <div className="container" key={post.id}>
                        <h1 pb>
                          <Link to={post.frontmatter.path}>{post.frontmatter.title}</Link>
                        </h1>
                        <h2>By <Link to={post.frontmatter.author.idpath}> {post.frontmatter.author.id}</Link> in <Link to={post.frontmatter.issue.idpath}> {post.frontmatter.issue.id}</Link></h2>          /*THIS IS THE ONLY PLACE ON THIS PAGE WHERE THE ISSUE YAML ARRAY SEEMS TO WORK. IT ONLY WORKS WHEN featured === true WHICH IS CRAZY*/
                        <p>{post.excerpt}</p>
                      </div>
                      )
                    })}
              </div>      
              <div className="thin">
              {posts
                  .filter(post => post.node.frontmatter.featured === true)                     /*This looks at only the md file with featured: true*/
                  .map(({ node: post }) => {
                    return (

                      <Link to="/latest">
                        <Image className="topimage"
                          fixed={post.frontmatter.currentcover.childImageSharp.fixed}      /*This pulls the image from the md file with featured: true (current cover)*/
                        />
                      </Link>

                      )
                    })}
              </div>
            </div>
            <hr />

            <div className="col-12">
              {posts
              .filter(post => !post.node.frontmatter.featured)
              .filter(post => post.node.frontmatter.issue === "Issue One Summer 2020")          /*THIS SHOULD FILTER ONLY MD FILES WITH issue: Issue One Summer 2020"*/
              .slice(0, 6)
              .map(({ node: post }) => {
                return (
                  <div className="postbody" key={post.id}>

                      <h2 pb>
                        <Link to={post.frontmatter.path}>{post.frontmatter.title}</Link> by <Link to={post.frontmatter.author.idpath}> {post.frontmatter.author.id}</Link> ({post.frontmatter.category})
                      </h2>
                  </div>
                )
              })}
            </div>

            <hr />

          </div>
        </div>
      </div>


    <div className="postbody">
      <div className="container pt-8 pt-md-4">
        <div className="row2 justify-content-start pt-2">
          <div className="col-12">
            <Link to="/fiction">
                <h4>Latest Fiction</h4>
            </Link>
            <hr />
          </div>
                                                                                      {/*FICTION*/}
          <div className="container">

            {posts
              .filter(post => !post.node.frontmatter.featured)
              .filter(post => post.node.frontmatter.category === "fiction")          /*This should only pull from md files with category "fiction", excluding posts marked featured*/
              .slice(0, 6)
              .map(({ node: post }) => {
                return (
                  <div className="container" key={post.id}>
                      <Image className="inlineimage"
                        fluid={post.frontmatter.cover.childImageSharp.fluid}          /*This should pull image from md files with category "fiction"*/
                      />
                      <h1 pb>
                        <Link to={post.frontmatter.path}>{post.frontmatter.title}</Link>
                      </h1>
                      <h2>By <Link to={post.frontmatter.author.idpath}> {post.frontmatter.author.id}</Link> in  <Link to={post.frontmatter.issue}> {post.frontmatter.issue}</Link></h2>
                      <p>{post.excerpt}</p>
                      <hr />
                  </div>
                )
              })}
            <div className="col-12 text-center pb-3">
              <Link className="button button-primary" to="/fiction">
                View All Stories
              </Link>
            </div>
          </div>
        </div>
      </div>
    </div>          
    </Layout>
  );
};

export const query = graphql`
  query {
    allAuthorYaml {
      nodes {
        bio
        id
        idpath
        picture {
          childImageSharp {
            fixed(width: 200) {
              ...GatsbyImageSharpFixed 
            }
            fluid(maxWidth: 150, maxHeight: 150) {
              ...GatsbyImageSharpFluid
            }
          }
        }
        stories {
          item
        }
        twitter
      }
    }
    allIssueYaml {
      edges {
        node {
          artist
          artistbio
          id
          idpath
          text
          artistimage {
            childImageSharp {
              fixed(width: 200) {
                ...GatsbyImageSharpFixed 
              }
              fluid(maxWidth: 150, maxHeight: 150) {
                ...GatsbyImageSharpFluid
              }
            }
          }
          currentcover {
            childImageSharp {
              fixed(width: 403) {
                ...GatsbyImageSharpFixed 
              }
              fluid(maxWidth: 300, maxHeight: 300) {
                ...GatsbyImageSharpFluid
              }
            }
          }
        }
      }
    }
    allMarkdownRemark(
      filter: { fileAbsolutePath: { regex: "/.*.md$/" }}
      sort: { fields: [frontmatter___date], order: DESC }
      ) {
      totalCount
      edges {
        node {
          id
          frontmatter {
            featured
            path
            title
            author {
              id
              idpath
              bio
              twitter
              picture {
                childImageSharp {
                  fixed(width: 200) {
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 150, maxHeight: 150) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
            }
            issue {
              id
              idpath
              currentcover {
                childImageSharp {
                  fixed(width: 403) {
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 300) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
              text
              artist
              artistimage {
                childImageSharp {
                  fixed(width: 200) {
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 150, maxHeight: 150) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
              artistbio 
            }
            date(formatString: "DD MMMM YYYY")
            category
            currentcover {
              childImageSharp {
                fixed(width: 403) {
                  ...GatsbyImageSharpFixed 
                }
                fluid(maxWidth: 300) {
                  ...GatsbyImageSharpFluid
                }
              }
            }
            cover {
              childImageSharp {
                fixed(width: 403) {
                  ...GatsbyImageSharpFixed 
                }
                fluid(maxWidth: 300) {
                  ...GatsbyImageSharpFluid
                }
              }
            }            
          }
          excerpt(pruneLength: 650)
        }
      }
    }
    allFeaturesJson {
      edges {
        node {
          id
          title
          description
          image
        }
      }
    }
  }
`;

export default Home;

A typical template with broken array:

import { graphql, Link, withPrefix } from 'gatsby';
import SEO from '../components/SEO';
import Layout from '../layouts/index';
import Helmet from 'react-helmet';
import Image from 'gatsby-image';

export default class Fictionarchive extends React.Component {
  render() {
    const posts = this.props.data.allMarkdownRemark.edges
    const json = this.props.data.allFeaturesJson.edges;

    const { FICcurrentPage, FICnumPages } = this.props.pageContext
    const isFirst = FICcurrentPage === 1
    const isLast = FICcurrentPage === FICnumPages
    const prevPage = FICcurrentPage - 1 === 1 ? "/" : `/fiction/${FICcurrentPage - 1}`
    const nextPage = `/fiction/${FICcurrentPage + 1}`

    return (
      <Layout bodyClass="page-home">
      <SEO title="Fiction" />
      <Helmet>
        <meta
          name="description"
          content="all fiction of Haven Quarterly"
        />
      </Helmet>

    <div className="postbody">
      <div className="container pt-md-5">
        <div className="row2 justify-content-start">
          <div className="col-12">
                <h3>Latest Fiction</h3>
            <hr />
          </div>

          <div className="container">
          {posts
              .filter(post => post.node.frontmatter.category === "fiction")
              .map(({ node: post }) => {
                return (
                  <div className="container" key={post.id}>
                      <Image className="inlineimage"
                        fluid={post.frontmatter.cover.childImageSharp.fluid}
                      />
                      <h1 pb>
                        <Link to={post.frontmatter.path}>{post.frontmatter.title}</Link>
                      </h1>
                      <h2>By <Link to={post.frontmatter.author.idpath}> {post.frontmatter.author.id}</Link> in  <Link to={post.frontmatter.issue.idpath}> {post.frontmatter.issue.id}</Link></h2>
                      <p>{post.excerpt}</p>
                      <hr />
                  </div>
                )
              })}
              <div className="container">
                <div className="row">
                  <div className="col-sm">
                    <p className="text-left">
                      {!isFirst && (
                        <Link to={prevPage} rel="prev">
                          ← Previous Page
                        </Link>
                      )}
                    </p>
                  </div>            
                </div>         
              </div>
          </div>
        </div>
      </div>
    </div>


    </Layout>
    )
  }
}

export const fictionarchiveQuery = graphql`
  query fictionarchiveQuery($skip: Int!, $limit: Int!) {
    allMarkdownRemark(
      filter: { frontmatter: {category:{eq:"fiction"} } }
      sort: { fields: [frontmatter___date], order: DESC }
      limit: $limit
      skip: $skip
    ) {
      edges {
        node {
          excerpt(pruneLength: 750)    
          frontmatter {
            category
            featured
            path
            title
            author {
              id
              idpath
              bio
              twitter
              picture {
                childImageSharp {
                  fixed(width: 400) {                                           
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 400, maxHeight: 400) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
            }
            issue {
              id
              idpath
              currentcover {
                childImageSharp {
                  fixed(width: 403) {
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 300) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
              text
              artist
              artistimage {
                childImageSharp {
                  fixed(width: 200) {
                    ...GatsbyImageSharpFixed 
                  }
                  fluid(maxWidth: 150, maxHeight: 150) {
                    ...GatsbyImageSharpFluid
                  }
                }
              }
              artistbio 
            }
            date(formatString: "DD MMMM YYYY")
            cover {
              childImageSharp {
                fixed(width: 322) {
                  ...GatsbyImageSharpFixed 
                }
                fluid(maxWidth: 450) {
                  ...GatsbyImageSharpFluid
                }
              }
            }
          }
          html
        }
      }
    }
    allFeaturesJson {
      edges {
        node {
          id
          title
          description
          image
        }
      }
    }
  }
`

Answer 1

Solution

Be smarter than me and make sure you've actually defined all of the values you have in your markdown files in your yaml file.

Longer Explanation

I asked on the GitHub repository for Gatsby, and the solution turned out to be really easy:

Everything was set up more or less correctly. The problem turned out to be that in my second yaml file, I wasn't defining all of the values that I was listing in my markdown files. So, between the 30+ .md files, I had four values total for the issue field, but in my issue.yaml file, I was only defining two of them. It returned a null error because when it cycled through the markdown files, it said, "I have this third value, but nothing that corresponds to it in the yaml file."

The person who answered my question said there were two solutions:

1. Define all issue IDs in issue.yaml so that you get at least the id back
2. Define an alternative issueTitle in your frontmatter of the posts and check whether issue returns null. If yes, hide your components and use issueTitle instead

I thought this didn't matter because I thought I was only calling data from markdown files with the fields I had defined (in other words, I thought I was already excluding those .md files that I hadn't defined in my yaml), but that was erroneous.

Other Things to Note

Two other simple issues that tripped me up: 1. Make sure that the spelling is exactly the same between what's defined in the yaml file and what is provided in the corresponding field of the markdown file. 2. Make sure you use an id field, as per this suggestion, when mapping your yaml file to a frontmatter field.

Final Thoughts

Turns out I don't know what I'm doing. But I hope that this answer helps anyone else noodling around with Gatsby who gets stuck on the same problem.

Link to the fuller answer kindly provided by LekoArts: https://github.com/gatsbyjs/gatsby/issues/25373