Undesired "<SomeOperator>.execute cannot be called outside TaskInstance!" warning about airflow HOT 6 OPEN

My 2 cents, this code use internals methods, as result it is expected that this approach could be broken on changes in airflow codebase

def deco(cls):
    orig_init = cls.__init__

    def new_init(self, *args, default_args=None, **kwargs):
        orig_init(self, *args, **kwargs)
        self.default_args = default_args

    cls.__init__ = cls._apply_defaults(new_init)
    return cls

from airflow.

Thanks for the investigation and feedback, @Taragolis and @dabla!

Yes, everything still works despite the warning message. I created this issue just as a "preventive" measure because @potiuk wrote in the closing comment of #37937:

Having a warning is "good enough" for now. If we decide to do something more (for example allow such use to behave more "as expected") we can always add it later.

What would you recommend if we need to inject some common logic into the constructor and the execute method of around 15 different (standard) operators? Or shall I patch the TaskInsntance and BaseOperator classes to insert the __sentinel property?

from airflow.

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

from airflow.

cc @dabla WDYT?

from airflow.

WDYT

Seems like the deco method is considered as calling he execute method of the PythonOperator outside of the TaskInstance, this is the protection mechanism which would warn you when mixing task decorated tasks with native operators. But normally it should still work atm, as this only triggers a warning.

Ok think I understand, it's because the class AlloyPythonOperator is instantiated outside the class TaskInstance, due to the deco method, so the sentinel which acts as a guard isn't set and thus Airflow considers that the execute method is called outside the TaskInstance

from airflow.

Thanks for the investigation and feedback, @Taragolis and @dabla!

Yes, everything still works despite the warning message. I created this issue just as a "preventive" measure because @potiuk wrote in the closing comment of #37937:

Having a warning is "good enough" for now. If we decide to do something more (for example allow such use to behave more "as expected") we can always add it later.

What would you recommend if we need to inject some common logic into the constructor and the execute method of around 15 different (standard) operators? Or shall I patch the TaskInsntance and BaseOperator classes to insert the __sentinel property?

In your example above, if you only have this "issue" of common logic with the PythonOpertor, then I would suggest you use the @task decorated method, there you could delegate to another non decorated method having all the common logic. Personally, I would not try to "mess" with native operators and start extending them.

from airflow.