Managing different software versions with git tags across multiple projects

Question

I recently ran into a problem I can't quite solve. Our company among many other projects develops a group of them centered around one - let's call it the engine. The engine contains most of the code base using the MVC framework yii. There are multiple other projects that use the engine as their base. While they can too contain unique code, they mostly use or extend methods and classes from the engine, specially tailored for the client's specifications (to a reasonable extent). These projects, although similar, can contain unique parts, making them different from each other. I'd say each project has 30-40% of it's codebase unique to them.

In the past months we ran into some difficulties when it come to releases and updates. Whenever a client request a new function, we have to develop it in a way that doesn't interfere with the other projects. Most of the time this is not a problem, but sometimes can lead to unexpected bugs or extended development time.

Still, the biggest headache comes before releases. While we can push them out in a timely manner, testing the new piece of code's effect on all the projects can take up more time than we'd like to (we are in the process of writing unit and selenium tests to make the process faster, and also because it's the rational thing to do if we want to manage a codebase this big and still keep our sanity... Unfortunately we are seriously behind as the management thought we can always do it later...).

We use git as our version tracking system which should be able to handle something like this, at least I hope. Our lead programmer manages quite a few projects with similar structure: multiple sites running off one central engine (the two engines are made for two completely different purposes). He ran into this very same issue years ago, and started using git tags to manage the more important releases. Some client's site uses the engine's 1.3 version, others the 1.4 and so on. While this system makes pushing updates and bug fixes easier and faster, I don't think it's practical in the long run. Or that it's the best practice.

Seems like "my" engine should use the same system, so we can enjoy the faster development cycle, but I'd like to know if there's a better way.

My main question is: how long could we keep up with this method? While I understand the advantage of only pushing new code to the project that needs it, creating a new release for every major function seems like something that could get out of hand quickly.

Also, how should we handle fixes and updates on existing code? My understanding is that fixing something on 1.2 doesn't fixes the bug on 1.3 and 1.4 - or am I mistaken? Or should I fix it on 1.4? Is there a way which I can chose that make it easy to apply specific changes to older/newer versions, so I don't have to make the same changes multiple times? What about enabling a function written for 1.5 on older releases?

I started to look at using patches to deliver the fixes to every version that needs them, but I'm not sure it's such a great idea.

Or should I just say "f*ck it" and repair bugs only on the latest version, upgrading projects one by one so they receive the fixes when we can make sure they won't break anything on that specific system?

I'm lost. While I have found resources about both git tags and patches, I feel I haven't found the true solution to my problem. One that is useful, doesn't break anything (I am 99% sure we have to keep the current setup of one central engine supporting multiple projects), and is sustainable. Currently we are only talking about a handful of sites built around the engine, but there are already plans for many more for this year, and as development times shrink, this number will only grow.

I hope a few of you who met and solved this problem can help me - I would be extremely grateful (and relieved knowing there is a solution...).

Thank you for your help in advance - I'm not joking when I say you'd help me keep my sanity.

Answer 1

Also, how should we handle fixes and updates on existing code? My understanding is that fixing something on 1.2 doesn't fixes the bug on 1.3 and 1.4 - or am I mistaken? Or should I fix it on 1.4? Is there a way which I can chose that make it easy to apply specific changes to older/newer versions, so I don't have to make the same changes multiple times?

No, fixes on one version are not automatically applied to all other versions. I assume that you create a bug-fix branch based off of the commit tagged as version 1.2. After finding and fixing the bug, you should merge the fix into version 1.2 and eventually tag a release for 1.2.1. At some point in the process these commits should be merged into other versions (e.g. 1.3, 1.4, etc.), resolving conflicts as necessary. Then make releases for 1.3.1, 1.4.1, etc with any additional bug fixes that are specific to those versions.

To address the more general issue, I believe that treating your "engine" as a separate project with its own release cycle is a reasonable strategy. Each of the projects that use this engine can specify the version it uses in the projects dependency management system (requirements.txt for python, Maven or Ant for Java, yarn or npm for JavaScript).