How to read text files with Elm and Webpack

Question

I have written a simple quizz application with Elm. As it was explained in the tutorial, the only way in Elm to access external files is to use ports with Javascript. So I have included ports in my Elm file, and I now have to add them in the index.js file that I use as an entry point. I use webpack to build the complete app.
However, I don't get the webpack logic. This is my file tree:

resources
    |---- images
    |---- questions
              |---- question_1.txt
              |---- question_2.txt
              |---- ...
    |---- scores
              |---- scores_table.json
src
    |---- MyElm.elm
    |---- Questions.elm
    |---- index.js
    |---- index.html

webpack.config.js

My JS component needs to read all possible questions in the questions folder to both determine the total number of questions and provide them to Elm through ports. In the same way, the JS component needs to parse the scores_table.json file to send results to the Elm app.

What can I use in my index.js app to read these files? I tried with require, but I think I didn't use it correctly.

It's my first question on Stack Overflow, so if there's anything missing, please tell me.

Minimal example

This is a simplified version of what I have:

webpack.config.js

var path = require("path");

module.exports = {
  entry: {
    app: [
      './src/index.js'
    ]
  },

  output: {
    path: path.resolve(__dirname + '/dist'),
    filename: '[name].js',
  },

  module: {
    rules: [
      {
        test: /\.txt$/,
        use: 'raw-loader'
      },
      {
        test: /\.(css|scss)$/,
        loaders: [
          'style-loader',
          'css-loader',
        ]
      },
      {
        test:    /\.html$/,
        exclude: /node_modules/,
        loader:  'file-loader?name=[name].[ext]',
      },
      {
        test:    /\.elm$/,
        exclude: [/elm-stuff/, /node_modules/],
        loader:  'elm-webpack-loader?verbose=true&warn=true',
      },
      {
        test: /\.woff(2)?(\?v=[0-9]\.[0-9]\.[0-9])?$/,
        loader: 'url-loader?limit=10000&mimetype=application/font-woff',
      },
      {
        test: /\.(ttf|eot|svg)(\?v=[0-9]\.[0-9]\.[0-9])?$/,
        loader: 'file-loader',
      },
    ],

    noParse: /\.elm$/,
  },

  devServer: {
    inline: true,
    stats: { colors: true },
  },

};

index.js

const RESOURCES_DIR = "../resources/";
const IMAGES_DIR = RESOURCES_DIR + "images/"
const QUESTIONS_DIR = RESOURCES_DIR + "questions/"
const SCORES_DIR = RESOURCES_DIR + "scores/"

require("./index.html");
const scores_table =
    require(SCORES_DIR + "scores_table.json");

var total_question_nb = 0;
var questions_table = [];
var end = false;

while (!end) {
    try {
        data = require(
            "raw!" +
            QUESTIONS_DIR +
            "question_${total_question_nb}.txt");
        questions_table.push(data);
        total_question_nb += 1;
    } catch (e) {
        end = true;
    }
}

console.log(questions_table[0]);
console.log(total_question_nb);

var Elm = require("./MyElm.elm");
var mountNode = document.getElementById("elm-app");

var app = Elm.MyElm.embed(mountNode);

// Need to add port gestion there

MyElm.elm

...
import Questions
...

Questions.elm

...
-- (current_question_no, ans_id)
port ask_question_score : (Int, Int) -> Cmd msg

-- next_question_no
port ask_next_question : Int -> Cmd msg

-- question_score
port receive_question_score : (List Int -> msg) -> Sub msg

-- (next_question_no, total_question_nb, next_question_text)
port receive_next_question : ((Int, Int, String) -> msg) -> Sub msg

-- ()
port receive_end_question : (() -> msg) -> Sub msg
...

And this is what I get when I load the page using Webpack:

Uncaught Error: Cannot find module "../resources/scores/scores_table.json".
    at r (app.js:1)
    at Object.<anonymous> (app.js:1)
    at r (app.js:1)
    at Object.<anonymous> (app.js:1)
    at r (app.js:1)
    at app.js:1
    at app.js:1

Answer 1

This I use in React project. It will allow you to change files without rebuilding with webpack:

in config/index.js

const CONFIG_FILE = '/config.json';

let loadedConfig = {};

export const loadConfig = () => {
  const headers = new Headers();

  headers.append('Content-type', 'application/json');
  try {
    return fetch(CONFIG_FILE, { headers })
      .then(response => response.json())
      .then((json) => { loadedConfig = json; });
  } catch (error) {
    console.log(error); // eslint-disable-line no-console
  }
};

export default () => ({ ...loadedConfig });

In index.jsx I load it:

import React from 'react';
import ReactDOM from 'react-dom';

import { loadConfig } from 'config';

loadConfig().then(() => {
  require.ensure([], (require) => {
    const App = require('App').default;
    ReactDOM.render(
      <App />,
      document.getElementById('root'),
    );
  });
});

And then I import it and use

import config from 'config';

console.log(config().SOME_VAR_FROM_JSON);

Answer 2

TLDR You'll need to setup Webpack's code splitting with dynamic imports to enable dynamic require's

Webpack is designed to compress all of your source files into one 'build' file. This, of course, relies on identifying which files you're importing. When you pass an expression rather than a plain string to require, webpack may not correctly identify the files you want.

In order to explicitly tell webpack what to include, you can use a "dynamic" import, ie code splitting. I'd say code splitting is rather advanced, if you want to avoid it, just hardcode the files you want to import. That should be fine if the files don't change often.

---

Examples

If you know the filenames:

const scores = require('../resources/scores/scores_table.json')
// or
import scores from '../resources/scores/scores_table.json'

// The `import()` function could be used here, but it returns a Promise
// I believe `require()` is blocking, which is easier to deal with here
// (though not something I'd want in a production application!)
const questions = [
  require('../resources/questions/question_1.txt'),
  require('../resources/questions/question_2.txt'),
]

If you want to dynamically import files (like you would probably do with questions):

// Utility function
// Example:
//     arrayOfLength(4) -> [0, 1, 2, 3]
const arrayOfLength = length =>
  (new Array(length)).fill(0).map((val, i) => i)

const questionsCount = 100 // <- You need to know this beforehand

// This would import questions "question_0.txt" through "question_99.txt"
const questionPromises = arrayOfLength(questionsCount)
  .map(i =>
    import(`../resources/questions/question_${i}.txt`)
      .catch(error => {
        // This allows other questions to load if one fails
        console.error('Failed to load question ' + i, error)
        return null
      })
  )

Promise.all(questionPromises)
  .then(questions => { ... })

With the dynamic imports, you need to handle promises. You could also use async / await to make it look a bit nicer (that isn't support in all browsers -- needs transpiling set up)

---

If the files do change often, that means you are frequently modifying questions and/or the score table, and you should probably be using a database instead of dynamic imports.