Unit/integration testing durable functions about azure-functions-durable-python HOT 7 OPEN

HI @paulschroeder-tomtom - thanks for the feedback and apologies for the delayed response. My original intention was to get back to you and Brandon after we wrap up planning, because that way I can tell you whether this is something we'll work on next. (The survey was intended to gather data to help with work prioritization for the upcoming semester.) That said, I do acknowledge that the turnaround time wasn't great, so I understand your frustration here.

The good news is that helping customers easily write Python unit tests is something we'll prioritize in the next semester, which starts in April. I can't give you a definitive date now on when the improved experience will go out, since there's existing current semester work that needs wrap up, but we do see the feedback here and will address this ASAP when in progress work is done. I'll be sure to keep this thread updated.

I'll add @davidmrdavid to chime in with anything he'd like to add.

from azure-functions-durable-python.

Hi @paulschroeder-tomtom, thanks for reaching out.

Yeah we've received a bit of feedback about the lack of clarity for testing, so we realize this is an area that needs improvement. I think we can use your project as a driving case to try to improve the experience here.

So to answer your question directly: I am personally not immediately sure that there's a "best practice"/ approved way to mock the DF Client and DF Context today (but there should be!), but I'm pretty certain that it is possible at the expense of it being hacky Are these the main objects you're struggling to mock? If so, I can try to put together the utilities that would allow you to do this and release them soon-ish.

In general, if you can put together a minimal app (emphasis on minimal, so that it's something I can run on my end) that you can share with me and comment in there the specific functionality you're and struggling trying to mock, that would help me identify the missing utilities (if any) and help provide guide the testing strategy.

from azure-functions-durable-python.

@davidmrdavid We are also struggling to really unittest our durable functions in v2. The decorators while really nice to work with, are a pain to unravel when testing. In activity functions we are testing them like <function_name>._function._func(<inputs) which isnt a huge deal but the orchestrators are almost impossible to test. Ill give an example. Say you wanted to write a unittest for the following orchestrator:

@bp.orchestration_trigger(context_name="context")
def orchestrator(context: df.DurableOrchestrationContext):
    x: int = context.get_input()
    y = yield context.call_activity(plusone, x)
    return y

@bp.activity_trigger(input_name="number")
def plusone(number: int) -> int:
    return number + 1

It isnt immediately apparent to me how you would do this. We copied over a contextbuilder class similar to how you guys did here but it ends up bringing with it almost the entire DF project. Its brittle and just overly complex. Im not sure what the right answer is here but I would love to have a simple way to test orchestrators and suborchestrators.

from azure-functions-durable-python.

Hey @davidmrdavid

totally agree with @brandonwatts . I quickly coded up a minimal example:

import logging

import azure.durable_functions as df
import azure.functions as func

ips_orchestrators = df.Blueprint()
app = df.DFApp(http_auth_level=func.AuthLevel.ANONYMOUS)


# /ips
@app.route(route="ips/{version}")
@app.function_name(name="ips")
@app.durable_client_input(client_name="client")
async def ips(req: func.HttpRequest, context: func.Context, client) -> func.HttpResponse:
    """
    Could be more complex, so it needs to be tested. Kept short just for the sake of this example.
    NOTE:
        As @brandonwatts said, the decorators "eat" up the function (I guess durable_client_input is returning
        None instead of the original, other decorators also magle it from a regular Callable to DF) and so
        the ips() does not end up in the scope if this file after importing it.
    """
    # ... do something
    # ... do something with the context
    client_input = dict(
        des_version=req.route_params.get('version')
    )

    instance_id = await client.start_new('list_ips_orchestrator', client_input=client_input)
    # ... do something more
    return client.create_check_status_response(instance_id=instance_id, request=req)


# Construct and send primitive API calls to create project
@ips_orchestrators.orchestration_trigger(context_name="context")
def list_ips_orchestrator(context: df.DurableOrchestrationContext):
    """
    I know about context.task_all() but I wanted to keep it like that, because it reflects more our use case.
    Same here with the decorators
    """
    des_version = context.get_input()['des_version']

    ip_v4 = yield context.call_http(
        method='GET',
        uri='https://ipv4.icanhazip.com',
    )
    ip_v6 = yield context.call_http(
        method='GET',
        uri='https://ipv6.icanhazip.com',
    )

    return dict(
        des_verison=des_version,
        ip_v4=ip_v4['content'],
        ip_v6=ip_v6['content'],
    )


app.register_blueprint(ips_orchestrators)

from azure-functions-durable-python.

It already 3 months ago that we opened up that issue. Is there any development on that?

from azure-functions-durable-python.

Hi @paulschroeder-tomtom , @brandonwatts - thank you for using Durable Functions! I'm a PM working on DF and would love to learn about your experience using the product. You can share your feedback in this quick survey to help influence what the team works on next. If you're building intelligent apps, there's also an opportunity to participate in a compensated UX study. Thanks!

from azure-functions-durable-python.

Hey @lilyjma to share my feedback also here: I am still waiting for member of the DF team to appropriately answer on that request right here as well as several other bugs/issues/question and feature requests.

I am quite annoyed by the behavior of MS. You are always asking for more information, a minimal example or (like here) a survey, without anything in return. No solution, no help. It really feels one sided! Improve on that.

from azure-functions-durable-python.