CODEWITHSUNDEEP
hclnomad
Michael Amos
Michael Amos

Reputation: 33

How to break nomad job spec files into smaller manageable units

I have a job spec file that I am working to improve. It has about 3000 lines of HCL code and is difficult to maintain. I looked on HashiCorp's documentation and not seeing how to break a job spec file into multiple files, i.e, break out the tasks, groups into separate files, etc. I am not seeing anything. I also asked ChatGPT and it gave me a solution that it admittingly said didnt exist after I asked ChatGPT to point me to the documentation on HashiCorp's site referencing the include parameter.

ChatGPT and the vendor documentation

Upvotes: 1

Views: 1286

Answers (3)

KamilCuk
KamilCuk

Reputation: 141698

It looks like HCL2 language wants to be the solution, but it is currently very limited, with an only bunch of basic functions https://developer.hashicorp.com/nomad/docs/job-specification/hcl2 , that do not allow to do like eval(file("./file")) evaluation of a string and also is not dynamic.

The most common solution, like nomad-pack or levant, is to add your own preprocessor to HCL files to do what you want. There is no standardization, so use what you want to do the generation. Other common preprocessors are jinja2, m4 or php.

Personally, I write python wrapper scripts around nomad run and plan commands for cicd and use jinja2 for templating.

Upvotes: 1

sbaildon
sbaildon

Reputation: 248

You've asked for HCL specifically, but on the off-chance you're unaware, jobs can be specified in JSON — then Jsonnet, CUE, pkl, and others can be used to template and generate complete configs

Upvotes: 0

Martijn Vermeulen
Martijn Vermeulen

Reputation: 11

One solution would be to use Nomad Pack.

This is a (open-source) HashiCorp tool to, amongst other things, "Re-use common patterns across internal applications".

This will allow you to split up different parts of the HCL file into template files and include them later on in your nomad pack. Some documentation can be found here: Helper templates. For example, create a template something.tpl file with some common code like:

[[- define "something_template" -]]
  variable "MY_VAR" {
    type = string
    default = "StackOverflow"
  }
[[- end -]]

Then in your active working Nomad Pack, in the metadata.hcl file, add a dependency to that new something.tpl template (could be a folder mention). And then include it in your active working Nomad Pack like this:

[[ template "something_template" . ]]

This should allow you to split the bigger job specs into more manageable pieces to maintain, but still having the same output for Nomad.

Hope this helps!

Upvotes: 1

Related Questions