CODEWITHSUNDEEP
puppet
UncleBob
UncleBob

Reputation: 1391

Puppet download a file without triggering a change notification

I'm having a bit of a puppet conundrum here. Most resources that download files from the web, like wget or remote_file, rely on one of two things to validate if they need to update the file: A Last-Modified header from the server, or a checksum. Checksums are not practical if what you want to ensure is that always the latest version of the file is installed, as you'd need to update the checksum in the puppet master every time which is exactly what I set out to avoid. Last-Modified headers, on the other hand, are unfeasible if the server you're downloading the files from just doesn't hand them out. The good old file-resource, on the other hand, would support dowload-and-compare, but unfortunately does not support authentication, so we cannot use it in our case either.

What the server hands out instead are small files containing only the md5 checksums of the actual files. So the way to go is to download that checksum, store it locally, and on each puppet run download these few bytes, compare them to what's stored, and download the big file if it's different.

That's not too complicated, so I wrote a module to generalise the operation. That module works very well, except it's got one big problem: It always issues a change notification. If I call the module and subscribe other steps to it, what I want is the notification to be issued if the big file needs to download. However, for that I need to download the small file first, and that will be enough for puppet to consider the state changed and notify the subscribers, even if it turns out that downloading the big file is unneccessary. I also tried to download the file with an Exec, but it there too I cannot control whether it notifies or not, only if it executes or not, and execute it must...

What I need is to suppress that notification, or a way to download a file without triggering that notification, but right now I'm starting to think it's impossible. Does anybody have an idea how to solve this dilemma?

Upvotes: 0

Views: 963

Answers (2)

John Bollinger
John Bollinger

Reputation: 180113

I think you would be best served by writing a custom type (i.e. in Ruby) instead of trying to implement it via built-in Puppet resource types and the Puppet DSL. You can consider your initial attempt to be a prototype.

If you must do it with the DSL, however, then you should be able to structure it around an Exec that runs the digest download + check via its unless or onlyif command, and the main file download and installation via its main command.

Upvotes: 1

balder
balder

Reputation: 799

From what i can tell what you have implemented is what the puppet file resource does by default e.g.

file{ '/var/big.txt.gz:  
  ensure => file,
  source => 'https://www.example..org/big.txt.gz',
}

Puppet determines if file content is synchronized by computing a checksum for the local file and comparing it against the checksum_value parameter. If the checksum_value parameter is not specified for puppet and file sources, Puppet computes a checksum based on its Puppet[:digest_algorithm]. For http(s) sources, Puppet uses the first HTTP header it recognizes out of the following list: X-Checksum-Sha256, X-Checksum-Sha1, X-Checksum-Md5 or Content-MD5. If the server response does not include one of these headers, Puppet defaults to using the Last-Modified header. Puppet updates the local file if the header is newer than the modified time (mtime) of the local file.

https://puppet.com/docs/puppet/7/types/file.html#file-attribute-source

Upvotes: 1

Related Questions