CODEWITHSUNDEEP
dependency-management
ceremcem
ceremcem

Reputation: 4360

How to handle dependency source version issues

If a project depends on some libraries, there are basically two options available:

  1. Locate the libraries in a well known global path (like /usr/src/linux-headers-4.13.0-1-amd64/
  2. Copy the sources into the project folder.

Using the global location approach takes minimum space and time (download time). If the dependencies are well versioned (tagged, like v3.1, v3.1.1, etc...), this option just works well.

If our project needs to use the latest commit of the library, versioning is not an option. If we just pull the latest commit into the well known location and let our project use the library, chances are we will be unable to compile our project 6 months later, which is unacceptable.

If we add the dependency as a subproject into our project, we will always be able to compile our project. This is the safest approach. Problem is that if the library is around 100MB, copying sources into the project folder is just a waste of disk usage, download time etc...

How do the people handle this problem?

Upvotes: 1

Views: 394

Answers (2)

ceremcem
ceremcem

Reputation: 4360

Here is how you can both have and eat your cake at the same time (see quantum physics):

  1. Maintain your dependencies (libraries) in global locations with their own distributed version control system (git, hg, bazaar, etc.). (See note 1)
  2. Maintain a dependency version database (a text file) inside the project folder and update it on each build. (Format: MyLibrary commit-hash-or-version)

Project will always use the latest version of the dependencies. If compilation fails at some point (when you try to compile the project 3 years later, for example),

  1. Go and get which version of a dependency was used at the last successful build point
  2. Create a checkout of the dependency with that hash in a temporary location.
  3. Point the temporary location as your dependency path in your build process.

Obviously this "return-to-successful-point" process can be easily automated.

Note 1: Dependencies are not meant to be only source codes, they might be simply some tools in binary format. In this case the dependency manager hook should invoke the tool with --version parameter (or what is appropriate for that tool), get the version, append to the dependencies.txt file. Obviously you should be prepared to be able to get the older version of the tool in case you'll need it at some point in the future.

Upvotes: 0

Doc Brown
Doc Brown

Reputation: 20054

You can't have your cake and eat it:

  • either you are going to rely on a stable, versioned library, then reference that (whereever it is located)

  • or you are going to rely always on the "latest and the greatest" - then reference a copy of that

Disk space is not really an issue - obviously you need to have at least one local copy of the lib you are using once on your developer machine, it does not make a difference if it is a "proper versioned" code or "head of trunk". And when it is always "head of trunk", you will probably use some version control system like git or svn to get the library, so whenever you update your local copy, you will pull only the changes from the repo, not the "full 100 MB".

However, to make your builds reproducible, with each version of your source code, you should version all dependencies as well together with your source code. If you are using third party libs, you can either

  • rely on the vendor providing you with exactly the versions you used during the project, and expect them to provide you with these versions for the next 10 years or so

  • or, keep your own copy of each version of the third party libs you used (probably put them in your own source code repo as well). That is the approach I always prefer for professional development, who knows if the library vendor is maintaining his stuff still next week, next month, next year?

Upvotes: 1

Related Questions