CODEWITHSUNDEEP
google-cloud-platformairflowgoogle-cloud-composer
AYR
AYR

Reputation: 1179

GCP Composer Airflow 2 - Is the Preview version stable?

On the 13th of May 2021 GCP added composer-1.17.0-preview.0-airflow-2.0.1 to their composer version list. It has already been upgraded a few times since then with "Full support end date" for each version, however it is still flagged as a Preview version.

I have already created a composer instance of composer-1.17.0-preview.7-airflow-2.0.2 and it seems to work smooth.

My question is what is the meaning of this "preview version"? Is it production worthy? If not, what is the purpose of it?

Upvotes: 1

Views: 1199

Answers (2)

rafalbiegacz
rafalbiegacz

Reputation: 96

Composer's support for Airflow 2 is now on the GA level. Please, take a look at the release notes: https://cloud.google.com/composer/docs/release-notes#September_15_2021

Upvotes: 3

Jarek Potiuk
Jarek Potiuk

Reputation: 20107

I think it should run smoothly and is ready to run production traffic, the main thing you don't get with preview, I believe, the main difference is that this version has no full guarantees that "official" release has.

I think this is mainly about https://cloud.google.com/composer/docs/concepts/versioning/composer-versioning-overview#version-deprecation-and-support - so you might expect that you might need to migrate to newer officially supported version of Airflow sooner than those, and that some, particular things will not be fully supported during the migration.

So while you do not have all the benefits of running a managed service (which typically frees you from worrying about maintenance pretty much completely), in this case you might expect some small maintenance and migration overhead when the officially supported version is released.

However my opinion is that it should be production-ready in general and if you are considering starting your Airflow installatiom, Airlfow 2.0.2 is a good choice. At the recent Composer Airflow Summit talk the Composer team mentioned that they are going to move out of preview for Airflow 2 pretty soon. Also Airflow 2 as a product released by the community have moved a long way in Airflow 2 than it was in 1.10.

Unlike 1.10.* versions - Airflow 2 fully follows SemVer approach. This means that migration to 2.* versions should be easy and backwards compatible. Airflow community treats the "SemVer" approach and promises very seriously https://github.com/apache/airflow#semantic-versioning

So I'd say you should expect very little disruptions even if you have to migrate sooner rather than later to newer version of Airlfow in Composer.

Upvotes: 4

Related Questions