Giter VIP home page Giter VIP logo

functions-framework-python's Introduction

Functions Framework for Python

PyPI version

Python unit CI Python lint CI Python conformace CI Security Scorecard

An open source FaaS (Function as a service) framework for writing portable Python functions -- brought to you by the Google Cloud Functions team.

The Functions Framework lets you write lightweight functions that run in many different environments, including:

The framework allows you to go from:

def hello(request):
    return "Hello world!"

To:

curl http://my-url
# Output: Hello world!

All without needing to worry about writing an HTTP server or complicated request handling logic.

Features

  • Spin up a local development server for quick testing
  • Invoke a function in response to a request
  • Automatically unmarshal events conforming to the CloudEvents spec
  • Portable between serverless platforms

Installation

Install the Functions Framework via pip:

pip install functions-framework

Or, for deployment, add the Functions Framework to your requirements.txt file:

functions-framework==3.*

Quickstarts

Quickstart: HTTP Function (Hello World)

Create an main.py file with the following contents:

import flask
import functions_framework

@functions_framework.http
def hello(request: flask.Request) -> flask.typing.ResponseReturnValue:
    return "Hello world!"

Your function is passed a single parameter, (request), which is a Flask Request object.

Run the following command:

functions-framework --target hello --debug
 * Serving Flask app "hello" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

(You can also use functions-framework-python if you have multiple language frameworks installed).

Open http://localhost:8080/ in your browser and see Hello world!.

Or send requests to this function using curl from another terminal window:

curl localhost:8080
# Output: Hello world!

Quickstart: CloudEvent Function

Create an main.py file with the following contents:

import functions_framework
from cloudevents.http.event import CloudEvent

@functions_framework.cloud_event
def hello_cloud_event(cloud_event: CloudEvent) -> None:
   print(f"Received event with ID: {cloud_event['id']} and data {cloud_event.data}")

Your function is passed a single CloudEvent parameter.

Run the following command to run hello_cloud_event target locally:

functions-framework --target=hello_cloud_event

In a different terminal, curl the Functions Framework server:

curl -X POST localhost:8080 \
   -H "Content-Type: application/cloudevents+json" \
   -d '{
	"specversion" : "1.0",
	"type" : "example.com.cloud.event",
	"source" : "https://example.com/cloudevents/pull",
	"subject" : "123",
	"id" : "A234-1234-1234",
	"time" : "2018-04-05T17:31:00Z",
	"data" : "hello world"
}'

Output from the terminal running functions-framework:

Received event with ID: A234-1234-1234 and data hello world

More info on sending CloudEvents payloads, see examples/cloud_run_cloud_events instruction.

Quickstart: Error handling

The framework includes an error handler that is similar to the flask.Flask.errorhandler function, which allows you to handle specific error types with a decorator:

import functions_framework


@functions_framework.errorhandler(ZeroDivisionError)
def handle_zero_division(e):
    return "I'm a teapot", 418


def function(request):
    1 / 0
    return "Success", 200

This function will catch the ZeroDivisionError and return a different response instead.

Quickstart: Pub/Sub emulator

  1. Create a main.py file with the following contents:

    def hello(event, context):
         print("Received", context.event_id)
  2. Start the Functions Framework on port 8080:

    functions-framework --target=hello --signature-type=event --debug --port=8080
  3. In a second terminal, start the Pub/Sub emulator on port 8085.

    export PUBSUB_PROJECT_ID=my-project
    gcloud beta emulators pubsub start \
        --project=$PUBSUB_PROJECT_ID \
        --host-port=localhost:8085

    You should see the following after the Pub/Sub emulator has started successfully:

    [pubsub] INFO: Server started, listening on 8085
    
  4. In a third terminal, create a Pub/Sub topic and attach a push subscription to the topic, using http://localhost:8080 as its push endpoint. Publish some messages to the topic. Observe your function getting triggered by the Pub/Sub messages.

    export PUBSUB_PROJECT_ID=my-project
    export TOPIC_ID=my-topic
    export PUSH_SUBSCRIPTION_ID=my-subscription
    $(gcloud beta emulators pubsub env-init)
    
    git clone https://github.com/googleapis/python-pubsub.git
    cd python-pubsub/samples/snippets/
    pip install -r requirements.txt
    
    python publisher.py $PUBSUB_PROJECT_ID create $TOPIC_ID
    python subscriber.py $PUBSUB_PROJECT_ID create-push $TOPIC_ID $PUSH_SUBSCRIPTION_ID http://localhost:8080
    python publisher.py $PUBSUB_PROJECT_ID publish $TOPIC_ID

    You should see the following after the commands have run successfully:

    Created topic: projects/my-project/topics/my-topic
    
    topic: "projects/my-project/topics/my-topic"
    push_config {
      push_endpoint: "http://localhost:8080"
    }
    ack_deadline_seconds: 10
    message_retention_duration {
      seconds: 604800
    }
    .
    Endpoint for subscription is: http://localhost:8080
    
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Published messages to projects/my-project/topics/my-topic.
    

    And in the terminal where the Functions Framework is running:

     * Serving Flask app "hello" (lazy loading)
     * Environment: production
       WARNING: This is a development server. Do not use it in a production deployment.
       Use a production WSGI server instead.
     * Debug mode: on
     * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
     * Restarting with fsevents reloader
     * Debugger is active!
     * Debugger PIN: 911-794-046
    Received 1
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 2
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 5
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 6
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 7
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 8
    127.0.0.1 - - [11/Aug/2021 14:42:22] "POST / HTTP/1.1" 200 -
    Received 9
    127.0.0.1 - - [11/Aug/2021 14:42:39] "POST / HTTP/1.1" 200 -
    Received 3
    127.0.0.1 - - [11/Aug/2021 14:42:39] "POST / HTTP/1.1" 200 -
    Received 4
    127.0.0.1 - - [11/Aug/2021 14:42:39] "POST / HTTP/1.1" 200 -
    

For more details on extracting data from a Pub/Sub event, see https://cloud.google.com/functions/docs/tutorials/pubsub#functions_helloworld_pubsub_tutorial-python

Quickstart: Build a Deployable Container

  1. Install Docker and the pack tool.

  2. Build a container from your function using the Functions buildpacks:

     pack build \
         --builder gcr.io/buildpacks/builder:v1 \
         --env GOOGLE_FUNCTION_SIGNATURE_TYPE=http \
         --env GOOGLE_FUNCTION_TARGET=hello \
         my-first-function
    
  3. Start the built container:

     docker run --rm -p 8080:8080 my-first-function
     # Output: Serving function...
    
  4. Send requests to this function using curl from another terminal window:

     curl localhost:8080
     # Output: Hello World!
    

Run your function on serverless platforms

Google Cloud Functions

This Functions Framework is based on the Python Runtime on Google Cloud Functions.

On Cloud Functions, using the Functions Framework is not necessary: you don't need to add it to your requirements.txt file.

After you've written your function, you can simply deploy it from your local machine using the gcloud command-line tool. Check out the Cloud Functions quickstart.

Cloud Run/Cloud Run on GKE

Once you've written your function and added the Functions Framework to your requirements.txt file, all that's left is to create a container image. Check out the Cloud Run quickstart for Python to create a container image and deploy it to Cloud Run. You'll write a Dockerfile when you build your container. This Dockerfile allows you to specify exactly what goes into your container (including custom binaries, a specific operating system, and more). Here is an example Dockerfile that calls Functions Framework.

If you want even more control over the environment, you can deploy your container image to Cloud Run on GKE. With Cloud Run on GKE, you can run your function on a GKE cluster, which gives you additional control over the environment (including use of GPU-based instances, longer timeouts and more).

Container environments based on Knative

Cloud Run and Cloud Run on GKE both implement the Knative Serving API. The Functions Framework is designed to be compatible with Knative environments. Just build and deploy your container to a Knative environment.

Configure the Functions Framework

You can configure the Functions Framework using command-line flags or environment variables. If you specify both, the environment variable will be ignored.

Command-line flag Environment variable Description
--host HOST The host on which the Functions Framework listens for requests. Default: 0.0.0.0
--port PORT The port on which the Functions Framework listens for requests. Default: 8080
--target FUNCTION_TARGET The name of the exported function to be invoked in response to requests. Default: function
--signature-type FUNCTION_SIGNATURE_TYPE The signature used when writing your function. Controls unmarshalling rules and determines which arguments are used to invoke your function. Default: http; accepted values: http, event or cloudevent
--source FUNCTION_SOURCE The path to the file containing your function. Default: main.py (in the current working directory)
--debug DEBUG A flag that allows to run functions-framework to run in debug mode, including live reloading. Default: False

Enable Google Cloud Function Events

The Functions Framework can unmarshall incoming Google Cloud Functions event payloads to event and context objects. These will be passed as arguments to your function when it receives a request. Note that your function must use the event-style function signature:

def hello(event, context):
    print(event)
    print(context)

To enable automatic unmarshalling, set the function signature type to event using the --signature-type command-line flag or the FUNCTION_SIGNATURE_TYPE environment variable. By default, the HTTP signature will be used and automatic event unmarshalling will be disabled.

For more details on this signature type, see the Google Cloud Functions documentation on background functions.

See the running example.

Advanced Examples

More advanced guides can be found in the examples/ directory. You can also find examples on using the CloudEvent Python SDK here.

Contributing

Contributions to this library are welcome and encouraged. See CONTRIBUTING for more information on how to get started.

functions-framework-python's People

Contributors

anguillanneuf avatar anniefu avatar asriniva avatar chizhg avatar di avatar garethgeorge avatar glasnt avatar grant avatar hkwinterhalter avatar ilpersi avatar joelgerard avatar jrmfg avatar kakaingoog avatar kanekv avatar kappratiksha avatar kaylanguyen avatar kenneth-rosario avatar matthewrobertson avatar michaelkarpe avatar mikemoore63 avatar mtraver avatar multani avatar nifflets avatar nurkiewicz avatar picardparis avatar release-please[bot] avatar renovate-bot avatar sesi avatar step-security-bot avatar svleeuwen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

functions-framework-python's Issues

Support Signature Type: CloudEvent

Support signature type cloudevent

This signature type should support HTTP requests with CloudEvent objects delivered via either structured and binary CloudEvent objects. Usually a CloudEvents SDK can handle transforming the HTTP request to a language-native CloudEvent object.

New signature cloudevent in Functions Framework contract

The end-user should see the parameter cloudevent, which is a native CloudEvent object using the CloudEvent SDK if possible.

Example User Experience (in Node)

function handler(cloudevent) {
  console.log(cloudevent.id);
  console.log(cloudevent.data.message);
  console.log(cloudevent.custom_values_should_be_accepted);
  return "anything"
}

Example HTTP requests

The CloudEvents specification requires support for both "binary" and "structured" encodings for CloudEvents delivered via HTTP. The Function Framework must support both encodings. A CloudEvents SDK might make this easier, but most SDKs are not complete, so logic may exist within the Framework.

Examples:

Binary content mode

POST /any/url/is/accepted HTTP/1.1
Host: example.com
ce-specversion: 1.0
ce-type: com.example.someevent
ce-time: 2018-04-05T03:56:24Z
ce-id: 1234-1234-1234
ce-source: /mycontext/subcontext
x-google-custom-header: 77
Content-Type: <contentType>
Content-Length: 33
{
    "message": "Hello World!"
}

Structured content mode

POST /someresource HTTP/1.1
Host: example.com
Content-Type: <contentType>
Content-Length: 266
{
    "specversion": "1.0",
    "type": "com.example.someevent",
    "time": "2018-04-05T03:56:24Z",
    "id": "1234-1234-1234",
    "source": "/mycontext/subcontext",
    "datacontenttype": "application/json",
    "data": {
        "message": "Hello World!"
    },
    "custom_values_should_be_accepted": 42
}

A user should be able to access all of the HTTP headers.
A user should be able to access the body of the CloudEvent in the data field of an object.


Run function in debug mode

Is it possible to run the flask server in debug mode?

Let's say I need to debug the function code using the framework from VSCode for example, is there a way to do that currently?

How can I register sqlalchemy extension to the current flask application globally?

The default way of using sqlalchemy models is calling globally:

db = SQLAlchemy()
db.init_app(app)

I don't see a way to get the "app" globally with functions-framework. sqlalchemy doesn't allow us registering it after a request is made. The desired use case is a simple:

result = SomeTable.query.all()

More information about using Flask-SqlAlchemy is here.

Using decorator @functions_framework.errorhandler makes the python script impossible to import

Using decorator @functions_framework.errorhandler makes the python script impossible to import without creating an application.
Import produces the error:

AttributeError: module 'functions_framework' has no attribute 'errorhandler'

Example

Trying to start run 'importer.py' script produce the above error.

decorated.py

from typing import Tuple

from flask.typing import ResponseValue, StatusCode

import functions_framework


@functions_framework.errorhandler
def handle_exception(ex: Exception) -> Tuple[ResponseValue[str], StatusCode]:
    return "Unhandled exception", 500

importer.py

import decorated

if __name__ == '__main__':
    print("Imported")

Possible source of the issue

The error handler decorator is registered in functions_framework.create_app() and it is not available without calling this functons.

Document ability to use Functions Framework as a WSGI app

While #36 removed the example of fine-tuning gunicorn, it's still possible to use the Functions Framework as a WSGI app in this way.

We should document how to do this if you need finer control over the HTTP server used, and also document some other WSGI servers that users could use instead of gunicorn if they need to.

[Feature Request] Continue to run even if introduce syntax errors

If I used native flask run to launch a functions as a server, it will keep running even though I raise a syntax error. But functions-framework can't. Is there a way to implement this mechanism in this nice tool? It's annoying to always re-command manually.

I survey how flask works with it and found an article about this mechanism.

Accessing Flask app context results in `RuntimeError: Working outside of application context.`

Hi,

I was trying to deploy some Google Cloud Functions today and found out about this wonderful project to test the functions locally. I can't thank you enough for this project, since it saves a lot of time (deploying is very time consuming, and being able to test it locally is a big relief).

I've created a minimum viable example that can help to reproduce the error. The same example works wonderfully when deployed as a Google Cloud Function, but it fails with an error when run with functions-framework. The error is as follows (I'm using python 3.7 to create a virtual env and testing within that):

(venv) .../python/min-eg  $ functions-framework --port 5000 --target hello
Traceback (most recent call last):
  File "/home/msharma/git/python/min-eg/venv/bin/functions-framework", line 8, in <module>
    sys.exit(_cli())
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/functions_framework/_cli.py", line 37, in _cli
    app = create_app(target, source, signature_type)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/functions_framework/__init__.py", line 229, in create_app
    spec.loader.exec_module(source_module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/msharma/git/python/min-eg/main.py", line 1, in <module>
    from app import db
  File "/home/msharma/git/python/min-eg/app.py", line 29, in <module>
    with app.app_context():
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/werkzeug/local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/werkzeug/local.py", line 306, in _get_current_object
    return self.__local()
  File "/home/msharma/git/python/min-eg/venv/lib/python3.7/site-packages/flask/globals.py", line 52, in _find_app
    raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.

This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context().  See the
documentation for more information.

The Example is as follows (2 files):

requirements.txt file

alembic==1.4.2
cffi==1.14.1
click==7.1.2
cloudevents==0.3.0
Flask==1.1.2
Flask-JWT-Extended==3.24.1
Flask-Migrate==2.5.3
Flask-SQLAlchemy==2.4.4
functions-framework==2.0.0
gunicorn==20.0.4
itsdangerous==1.1.0
Jinja2==2.11.2
Mako==1.1.3
MarkupSafe==1.1.1
pathtools==0.1.2
psycopg2==2.8.5
py-bcrypt==0.4
pycparser==2.20
PyJWT==1.7.1
python-dateutil==2.8.1
python-editor==1.0.4
six==1.15.0
SQLAlchemy==1.3.18
watchdog==0.10.3
Werkzeug==1.0.1

main.py file

import os

from flask import current_app as app
from flask_sqlalchemy import SQLAlchemy, sqlalchemy

def setup_context():
    # Set these in the environment variables for the function
    db_user = os.environ["DB_USER"]
    db_password = os.environ["DB_PASSWORD"]
    db_name = os.environ["DB_NAME"]

    db_connection_instance_name = os.environ["DB_CONNECTION_INSTANCE_NAME"]
    db_host = os.environ["DB_HOST"]

    app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False 

    driver_name = 'postgresql+psycopg2'
    app.config['SQLALCHEMY_DATABASE_URI'] = sqlalchemy.engine.url.URL(
        drivername=driver_name,
        username=db_user,
        password=db_password,
        database=db_name,
        host=db_host
    )

    db = SQLAlchemy(app)
    return db

with app.app_context():
    db = setup_context()

def hello(request):
    stmt = 'select * from users;'
    try:
        result = db.session.execute(stmt)
        for rowproxy in result:
            print(rowproxy.items())
    except Exception as e:
        return 'Error: {}'.format(str(e))
    return 'ok'

The issue seems to be in the __init__.py file where we create an instance of the Flask app (https://github.com/GoogleCloudPlatform/functions-framework-python/blob/master/src/functions_framework/__init__.py#L154-L175). I'm not great at python, but it looks to me as if the module is imported before the Flask app context is even created. So, if I move the app = flask.Flask(target, template_folder=template_folder) before I import the source module, I can successfully run functions-framework without an errors.

In terms of code, something like this (i'm not sure if I'm doing it correctly, so please bear with me):

    app = flask.Flask(target, template_folder=template_folder)
    app.config["MAX_CONTENT_LENGTH"] = MAX_CONTENT_LENGTH

    with app.app_context():
        # Load the source file:
        # 1. Extract the module name from the source path
        realpath = os.path.realpath(source)
        directory, filename = os.path.split(realpath)
        name, extension = os.path.splitext(filename)

        # 2. Create a new module
        spec = importlib.util.spec_from_file_location(name, realpath)
        source_module = importlib.util.module_from_spec(spec)

        # 3. Add the directory of the source to sys.path to allow the function to
        # load modules relative to its location
        sys.path.append(directory)

        # 4. Add the module to sys.modules
        sys.modules[name] = source_module

        # 5. Execute the module
        spec.loader.exec_module(source_module)

Is it a bug, or am I doing something wrong here?

Support `click>=7.0<9.0`

We currently depend on click but restrict it's major version:

"click>=7.0,<8.0",

The click==8.0.0 release was made yesterday https://pypi.org/project/click/8.0.0/ and a full changelog is here: https://click.palletsprojects.com/en/8.0.x/changes/

This can introduce conflicts with the functions framework if the user has a dependency or subdependency on click, which results in an error at deploy-time like:

$ gcloud functions deploy visitme --runtime python38 --trigger-http --allow-unauthenticated
Deploying function (may take a while - up to 2 minutes)...โ น
For Cloud Build Stackdriver Logs, visit: https://console.cloud.google.com/logs/viewer?project=wescpy-ccsc&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3D84accc73-466c-4dbb-a681-9eb777000e24%0AlogName%3Dprojects%2Fwescpy-ccsc%2Flogs%2Fcloudbuild
Deploying function (may take a while - up to 2 minutes)...failed.
ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed: found incompatible dependencies: "functions-framework 2.1.2 has requirement click<8.0,>=7.0, but you have click 8.0.0."; Error ID: 945b0f01

We should make any changes necessary to support click>=7.0<9.0 and make a new release with this updated version range.

But don't take away exception handling!

Howdy!

I love the idea of this repository. We have tons of GCP Cloud Functions and Cloud Runs (what an awful noun). It can be hard for a dev to switch between the two, particularly to go to Cloud Run land when they've never seen a Dockerfile before.

But we're moving more and more of our stuff into Cloud Run because it offers better options for handling exceptions. Many of these are super-duper simple functions and Cloud Run feels like overkill, just to allow me to specify how we should handle those exceptions.

I realize this is more like a feature request for Cloud Functions, but consider how easy you could make this for everyone:

def entrypoint_do_thing(request):
  if request.args['foo'] == 'bar:
    return 'some_data', 200
  else:
    raise BadnessError()

@functions_framework.error_handler(BadnessError)
def handle_badness(e):
  return 'badness happened', 400

Something like the above would be simple and familiar for lots of folks since it looks like Flask error handling, but you still wouldn't have to expose the underlying Flask app to the world.

Thoughts?

Different source than the root directory doesn't seem to work

Hi!

Consider following project structure:

.
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ src
โ”‚  โ”œโ”€โ”€ main.py
โ”‚  โ”œโ”€โ”€ requirements.txt

Not sure whether such structure makes sense but currently it works for me with Google Cloud Functions + terraform. src directory gets zipped, uploaded to gcs and deployed. Now, I tried to also create an option for easier local development, but:

Problem

Using different than default GOOGLE_FUNCTION_SOURCE doesn't seem to work. Using the GOOGLE_FUNCTION_SOURCE with directory path (e.g. GOOGLE_FUNCTION_SOURCE=src) fails with:

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/bin/functions-framework", line 8, in <module>
    sys.exit(_cli())
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/functions_framework/_cli.py", line 37, in _cli
    app = create_app(target, source, signature_type)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/functions_framework/__init__.py", line 216, in create_app
    source_module = importlib.util.module_from_spec(spec)
  File "<frozen importlib._bootstrap>", line 553, in module_from_spec
AttributeError: 'NoneType' object has no attribute 'loader'

With full path to the file with the function specified (i.e. GOOGLE_FUNCTION_SOURCE=src/main.py) functions framework fails to install packages for the function and the function fails on the first import of external package, in my case:

ModuleNotFoundError: No module named 'weasyprint'

Solution

Allow users to specify the source directory instead of source file, in fact, I think that's how the Google Cloud Functions handle it: https://cloud.google.com/functions/docs/writing#structuring_source_code

How to serve multiple routes on the same Url : Port

Hi,

I'm trying to serve multiple routes from the same Host / Port combination for local testing.

Each of these routes would point to a specific target.
All cloud functions (on GCloud) are sharing the same hostname.
So I would like to replicate this environment as much as possible

Is this possible ? I have not seen anything in the documentation to help me perform this setup.
I would like to serve something like this:

http://localhost:8080/doFunctionA
http://localhost:8080/doFunctionB
http://localhost:8080/doFunctionC

Thanks

Reconsider adding gunicorn as required dependency

In the PR #36, the Gunicorn server became the required dependency which means that each app will have to download the Gunicorn and moreover if you bundle the app as a package with functions-framework, the Gunicorn will be in that package as well.

While it looks like smth that ease development and deployment and allows bootstrapping production-ready instances faster, in many cases I've seen Docker configurations where the application itself is bundled as an installable module and the Gunicorn is set up only inside the actual runner container. Moreover, I'd say such a way allows developers to fine-tune their setup better. The other case is people just using different WSGI servers.

My suggestion is to roll-back the latest change and do not require gunicorn as a dependency.

Installing on macOS fails due to failed watchdog installation

Due to gorakhargosh/watchdog#689 and watchdog not providing built distributions, installation has been failing on macOS:

$ python3 -m pip install functions-framework
Collecting functions-framework
  Downloading https://files.pythonhosted.org/packages/63/bc/9ec82d36af2f58ad39522f66684561f316a4fd8133fdf645d6e739025fb3/functions_framework-2.0.0-py3-none-any.whl
Collecting click<8.0,>=7.0 (from functions-framework)
  Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
Collecting watchdog>=0.10.0 (from functions-framework)
  Downloading https://files.pythonhosted.org/packages/e6/76/39d123d37908a772b6a281d85fbb4384d9db7e13d19d10ad409006bd2962/watchdog-1.0.1.tar.gz (97kB)
     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 102kB 4.1MB/s 
Collecting gunicorn<21.0,>=19.2.0; platform_system != "Windows" (from functions-framework)
  Downloading https://files.pythonhosted.org/packages/69/ca/926f7cd3a2014b16870086b2d0fdc84a9e49473c68a8dff8b57f7c156f43/gunicorn-20.0.4-py2.py3-none-any.whl (77kB)
     |โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 81kB 14.2MB/s 
Collecting cloudevents<1.0 (from functions-framework)
  Downloading https://files.pythonhosted.org/packages/50/f9/ff5db473ee17d2a83a697ea238e6cf41970057fe082b3b5a5c18abc4511b/cloudevents-0.3.0-py3-none-any.whl
Collecting flask<2.0,>=1.0 (from functions-framework)
  Using cached https://files.pythonhosted.org/packages/f2/28/2a03252dfb9ebf377f40fba6a7841b47083260bf8bd8e737b0c6952df83f/Flask-1.1.2-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=3.0 in /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages (from gunicorn<21.0,>=19.2.0; platform_system != "Windows"->functions-framework) (41.2.0)
Collecting Werkzeug>=0.15 (from flask<2.0,>=1.0->functions-framework)
  Using cached https://files.pythonhosted.org/packages/cc/94/5f7079a0e00bd6863ef8f1da638721e9da21e5bacee597595b318f71d62e/Werkzeug-1.0.1-py2.py3-none-any.whl
Collecting itsdangerous>=0.24 (from flask<2.0,>=1.0->functions-framework)
  Using cached https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting Jinja2>=2.10.1 (from flask<2.0,>=1.0->functions-framework)
  Using cached https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.10.1->flask<2.0,>=1.0->functions-framework)
  Downloading https://files.pythonhosted.org/packages/0c/12/37f68957526d1ec0883b521934b4e1b8ff3dd8e4fab858a5bf3e487bcee9/MarkupSafe-1.1.1-cp38-cp38-macosx_10_9_x86_64.whl
Building wheels for collected packages: watchdog
  Building wheel for watchdog (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Library/Developer/CommandLineTools/usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/f3/n1z3ncc57rzcsdmjrdqpk33000rnw4/T/pip-install-2va8a80x/watchdog/setup.py'"'"'; __file__='"'"'/private/var/folders/f3/n1z3ncc57rzcsdmjrdqpk33000rnw4/T/pip-install-2va8a80x/watchdog/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/f3/n1z3ncc57rzcsdmjrdqpk33000rnw4/T/pip-wheel-7b2gmfbc --python-tag cp38
       cwd: /private/var/folders/f3/n1z3ncc57rzcsdmjrdqpk33000rnw4/T/pip-install-2va8a80x/watchdog/
  Complete output (146 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.14.6-x86_64-3.8
  creating build/lib.macosx-10.14.6-x86_64-3.8/watchdog
  copying src/watchdog/watchmedo.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog
  copying src/watchdog/version.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog
  copying src/watchdog/events.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog
  copying src/watchdog/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog
  creating build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/patterns.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/dirsnapshot.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/delayed_queue.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/platform.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/bricks.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  copying src/watchdog/utils/echo.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/utils
  creating build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/fsevents.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/inotify.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/api.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/inotify_buffer.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/winapi.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/read_directory_changes.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/kqueue.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/inotify_c.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/polling.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  copying src/watchdog/observers/fsevents2.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/observers
  creating build/lib.macosx-10.14.6-x86_64-3.8/watchdog/tricks
  copying src/watchdog/tricks/__init__.py -> build/lib.macosx-10.14.6-x86_64-3.8/watchdog/tricks
  running egg_info
  writing src/watchdog.egg-info/PKG-INFO
  writing dependency_links to src/watchdog.egg-info/dependency_links.txt
  writing entry points to src/watchdog.egg-info/entry_points.txt
  writing requirements to src/watchdog.egg-info/requires.txt
  writing top-level names to src/watchdog.egg-info/top_level.txt
  reading manifest file 'src/watchdog.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.h' under directory 'src'
  writing manifest file 'src/watchdog.egg-info/SOURCES.txt'
  running build_ext
  building '_watchdog_fsevents' extension
  creating build/temp.macosx-10.14.6-x86_64-3.8
  creating build/temp.macosx-10.14.6-x86_64-3.8/src
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/Headers -arch arm64 -arch x86_64 -DWATCHDOG_VERSION_STRING="1.0.1" -DWATCHDOG_VERSION_MAJOR=1 -DWATCHDOG_VERSION_MINOR=0 -DWATCHDOG_VERSION_BUILD=1 -I/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8 -c src/watchdog_fsevents.c -o build/temp.macosx-10.14.6-x86_64-3.8/src/watchdog_fsevents.o -std=c99 -pedantic -Wall -Wextra -fPIC -Wno-nullability-completeness -Wno-nullability-extension -Wno-newline-eof -Wno-error=unused-command-line-argument
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
  In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/limits.h:63:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
  #error Unsupported architecture
   ^
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:11:
  In file included from /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/include/limits.h:21:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/limits.h:64:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported
  #error architecture not supported
   ^
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:33:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
  #error architecture not supported
   ^
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:27:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'
  typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
          ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
  typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
          ^
  note: '__int128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:57:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
  typedef __int32_t       __darwin_dev_t;         /* dev_t */
          ^
  note: '__int128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:60:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_gid_t;         /* [???] process and group IDs */
          ^
  note: '__uint128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:61:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_id_t;          /* [XSI] pid_t, uid_t, or gid_t*/
          ^
  note: '__uint128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:62:9: error: unknown type name '__uint64_t'
  typedef __uint64_t      __darwin_ino64_t;       /* [???] Used for 64 bit inodes */
          ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:68:9: error: unknown type name '__darwin_natural_t'
  typedef __darwin_natural_t __darwin_mach_port_name_t; /* Used by mach */
          ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:70:9: error: unknown type name '__uint16_t'; did you mean '__uint128_t'?
  typedef __uint16_t      __darwin_mode_t;        /* [???] Some file attributes */
          ^
  note: '__uint128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:71:9: error: unknown type name '__int64_t'
  typedef __int64_t       __darwin_off_t;         /* [???] Used for file sizes */
          ^
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:72:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
  typedef __int32_t       __darwin_pid_t;         /* [???] process and group IDs */
          ^
  note: '__int128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:73:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_sigset_t;      /* [???] signal set */
          ^
  note: '__uint128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:74:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
  typedef __int32_t       __darwin_suseconds_t;   /* [???] microseconds */
          ^
  note: '__int128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:75:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_uid_t;         /* [???] user IDs */
          ^
  note: '__uint128_t' declared here
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types.h:76:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_useconds_t;    /* [???] microseconds */
          ^
  note: '__uint128_t' declared here
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:71:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_types.h:43:9: error: unknown type name '__uint32_t'; did you mean '__uint128_t'?
  typedef __uint32_t      __darwin_wctype_t;
          ^
  note: '__uint128_t' declared here
  In file included from src/watchdog_fsevents.c:22:
  In file included from /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/include/python3.8/Python.h:25:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h:64:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/_stdio.h:75:
  In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/_types/_va_list.h:31:
  /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/machine/types.h:37:2: error: architecture not supported
  #error architecture not supported
   ^
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  20 errors generated.
  error: command 'clang' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for watchdog
  Running setup.py clean for watchdog
Failed to build watchdog
Installing collected packages: click, watchdog, gunicorn, cloudevents, Werkzeug, itsdangerous, MarkupSafe, Jinja2, flask, functions-framework
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Python/3.8'
Consider using the `--user` option or check the permissions.

WARNING: You are using pip version 19.2.3, however version 20.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Expose trace IDs in background and cloud event functions

The X-Cloud-Trace-Context header is included in requests, but as we don't expose headers to the event-style functions it is inaccessible. This value is the only means of grouping StackDriver logs by request, and so we should expose it to the user function code.

Background events: The trace ID should be available in the context parameter.

CloudEvents: The trace ID should be encoded as an extension on the event, leveraging the traceparent extension field as described in https://github.com/cloudevents/spec/blob/master/extensions/distributed-tracing.md.

Not able to deploy function with .zip archive

I do understand that the issue is not related to the functions-framework directly, but maybe someone from the team can reach the GCF team and ask them to check it out.

So, the issue is the following. I'm not able to deploy a GCF source as .zip archive via Cloud Console UI or using terraform. But if I unpack the function and go with gcloud functions deploy it works just fine. Here is the error message I got both from Cloud Console UI and from terraform:

Error: Error waiting for Creating CloudFunctions Function: Error code 3, message: Build failed: {"cacheStats": [{"status": "MISS", "hash": "8e51312351ee02e20ebdcdcefb100908ee770a1c81810601ddfcd599221c617b", "type": "docker_layer_cac
he", "level": "global"}, {"status": "MISS", "hash": "8e51312351ee02e20ebdcdcefb100908ee770a1c81810601ddfcd599221c617b", "type": "docker_layer_cache", "level": "project"}]}

Does anyone have some ideas about what could be wrong? Unfortunately, I cannot share to source code as it is under the NDA. I'll try to create a reproducible example and share it.
Meanwhile, I have also created issue in the public tracker.

Deployment failure: Function failed on loading user code. Error message: File main.py is expected to contain a function named cleanEdges

Getting the error in title despite being able to see my code in the google cloud platform cloud functions console with the named function.

import cloudstorage as gcs
# from google.cloud import storage
import open3d 
import networkx as nx
import numpy as np

client = gcs.Client()
RETURN_BUCKET = 'outgoing_meshtools'

def cleanEdges(data, context):
    ***content of function

**other functions

Trigger: Cloud Storage
Event Type: Finalize/Create

Module Not Found error from dependency in requirements.txt

I am trying to test a function using the functions-framework that has dependencies inside a requirements.txt file but I am getting an error that the module is not found after running the functions-framework --target=main command.

image

** Edit the functions-framework only seems to work when I run pip install functions-framework in my virtual env. Why would it not work with a functions-framework installation in my main environment?

Better document testing

Hi,
neither the docs or code examples mention how to set up tests - which I think is important for any non trivial function.

I currently hook up my tests like this:

from flask import testing
import pytest
from functions_framework import create_app
import pathlib


@pytest.fixture
def client() -> testing.Client:
    target = "function_name"
    source = pathlib.Path(__file__).parent.parent / "main.py"

    with create_app(target, str(source)).test_client() as client:
        yield client

def test_something(client: testing.Client):
    rv = client.post('/', data="no json")
    assert rv.status_code == 400

this assumes that the test file is inside a test folder.

Missing cloud event adapter tests

We need a complete set of Event adapter tests to ensure that the various combinations of event types (Cloud Event, Binary Cloud Event, Legacy Event) against the different signature types (FUNCTION_SIGNATURE_TYPE=[cloudevent,event]) work.

Don't ship CEs without this in order to ensure backwards compatibility.

Functions Framework does not work with the Pub/Sub emulator

Hello,

I am trying to emulate the behavior of my application.
To do so I use the Pub/Sub Emulator and the Google cloud function emulator.
The cloud function I created looks like the image following and is supposed to be triggered by an event :
image

Both emulators are communicating without issue.
I followed the example of this repository: https://github.com/GoogleCloudPlatform/functions-framework-nodejs to link them.

When, following the example, I use this command line:

curl -d "@mockPubsub.json" -X POST
-H "Ce-Type: true"
-H "Ce-Specversion: true"
-H "Ce-Source: true"
-H "Ce-Id: true"
-H "Content-Type: application/json"
http://localhost:8080

On my Cloud function emulator I do receive this :
image

So this is exactly what I want.
Yet, I do not want to use the command line to send messages but I want the cloud function to be triggered by a Pub/Sub event.

Here you can see the methods creating:
I. a pull subscription
II. a push subsciption
III. publishing messages

image

However When I want to publish messages, the Pub/Sub emulator detect the connection:

[pubsub] INFO: Detected HTTP/2 connection.

But then the Cloud Function emulator does not receive the data, although it detects an event occurred:
image

I think it is probably because POSTING to a Gcloud Function needs some headers with the 'Ce-' prefix.
But searching a bit on how to add headers to my push subscriber I found out this wasn't possible.
It's possible I totally missed something...

I hope this is understandable.
Thank you

Cut 1.1 Release on PyPi

Hey @di, I think we need to cut a new release on pypi to update the signature-type typo.

The change seems to be on master but not released on pypi.

Related:

Testing Functions in Google Colab

Use Case:

  • Test functions in a Google Colab (within Google Drive).
  • Include flask-ngrok in the requirement.txt already.

This will allow the Flask app (which is running on localhost) to be available temporarily over the Internet via the excellent ngrok tool, when one runs main.py thereafter:

from flask import Flask
from flask import request
from flask_ngrok import run_with_ngrok
app = Flask(__name__)
run_with_ngrok(app)
@app.route("/")
def root():
    url = request.method
    return "Hello world!"
app.run()

Access function(s) by navigating to the following urls:

 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
 * Running on http://<dynamic-value>.ngrok.io
 * Traffic stats available on http://127.0.0.1:4040

functions framework with google pub/sub emulator

I am attempting to replicate our cloud functions locally but they all leverage pub/sub. I can easily use the pub/sub emulator but I don't see how to subscribe to a topic on the local emulator using the local cloud functions? Am I missing something, how would you be able to run this locally and emulate your workflow without this?

Would anyone have any recommendations on how to run the full stack locally for local development environments?

I was assuming I would be able to set the pub/sub emulator env var and the cloud function would automatically subscribe and pull like it does in our testing and production envs.

Thanks for the help!

Add actionable guidance on payload validation error

When working with background events locally, it can be difficult to construct the payload correctly. If you are missing the data or context properties, the framework will raise an HTTP 400 response without actionable details. (#83 addresses documentation)

Current Behavior

The current framework response is:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not understand.</p>

The current log output is:

127.0.0.1 - - [16/Apr/2021 12:38:05] "POST / HTTP/1.1" 400 -

Expected behavior

Log output should specify payload validation errors down to the property & type level.

Thread safe feature in macOS > Sierra breaks default behavior. Make worker sync

I'm trying to run my integration tests as stated by the GCP guide to perform integration tests in cloud functions

However I find an error in macOS where I have to disable the "safe thread" option so that gunicorn can fork the current process

[2020-11-06 13:36:37 +0100] [6124] [INFO] Starting gunicorn 20.0.4
[2020-11-06 13:36:37 +0100] [6124] [INFO] Listening at: http://0.0.0.0:8080 (6124)
[2020-11-06 13:36:37 +0100] [6124] [INFO] Using worker: threads
[2020-11-06 13:36:37 +0100] [6127] [INFO] Booting worker with pid: 6127
objc[6127]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[6127]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

To enable it I had to set this flag (thanks to StackOverflow)

OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

I see that, when the functions-framework starts it says Using worker: threads. So can we use the "synchronous" gunicorn worker to avoid this? Or at least make it configurable as a click cli option?

Code I'm using

import signal
import subprocess
import sys

process = subprocess.Popen(
    [
        'functions-framework',
        '--target', 'api',
        '--port', '8080',
    ],
    stdout=subprocess.PIPE,
)


def signal_handler(sig, frame):
    process.kill()
    process.wait()
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)
signal.pause()

Binaries should have language-specific alias

The Python FF can be invoked via a system-wide binary.

This is fine for environments where only one FF is installed, but environments with multiple FFs (i.e. FFs for multiple languages on the same machine, or multiple FF versions for a single language) make it harder to select specific FFs.

See this issue for more context.

Impossible to use get_type_hints in main file

get_type_hints is used by Pydantic, dacite etc. to create their model classes, so it'S kinda important.

Currentliy, this is broken in main.py:

Traceback (most recent call last):
  File "***\main.py", line 40, in ar_v4
    data = dacite.from_dict(RequestData2, message)
  File "***\venv\lib\site-packages\dacite\core.py", line 43, in from_dict
    data_class_hints = get_type_hints(data_class, globalns=config.forward_references)
  File "C:\dev\python3.8\lib\typing.py", line 1220, in get_type_hints
    base_globals = sys.modules[base.__module__].__dict__
KeyError: 'main'

reproduction:

install dacite
in main.py

import dacite
import dataclasses

@dataclasses.dataclass
class RequestData2:
    product_number: int

def hello_world(request):
    message = request.get_json()
    data = dacite.from_dict(RequestData2, message)
    return ""

post some data to it and observe the error.

Workaround:

create another file and place your data classes in them

make host configurable

On mac every time it reloads there is a firewall alert because it's fixed listen on 0.0.0.0. Can we add --host option to override the interface?

Inconsistency in the path between emulator and the deployed cloud function

I have been using the functions framework to emulate my cloud functions locally for development and
had no issue until I tried to do some simple routing based on the request path.

When running in the emulator the request.path contains the function name like so: "/function_name" or "/function_name/some_route".
Meanwhile in deployment cloud functions only receive in the request.pathrespectively: "/" and "/some_route". I assume the decision was made based on if I'm already inside a given function I have no use for its "full path".

I do not know if the decision to handle the path in the emulator differently has to do with other use cases for the framework other than just simple emulation.
But would it would be great if I could set a flag to make the path handle the same way in the emulator so that we don't need different parsing for emulated and deployed code.

Docs vs code - --signature-type or --signature_type?

Hi, the documentation says the command line flag is --signature-type (which is matches the common style of - instead of _ in command line flags), but it's actually implemented as --signature_type - which is intended?

Document incoming request type and/or add request parsing examples

Please document the incoming "request" object to the function stub. It is important in the weakly typed languages.

For python, it happens to be a Flask request, which means I have to call request.get_json() to get the request body of POSTs.

But it could also have feasibly been some alternative like Django HttpRequest, which means I use request.body to get the request body of POSTs.

It would also be nice to have an example that parses a request body and/or request URL parameters and provides an example cURL request instead of just returning a string.

Unable to use functions-framework-python with pubsub beta emulator

Following the README.md, I am able to trigger functions but I am expecting 'data' as a part of the event but unfortunately the message being passed from google beta emulatar is in the following format:

{
      "subscription": "projects/test/subscriptions/test1",
      "message": {
        "data": "TWVzc2FnZSBudW1iZXIgNA==",
        "messageId": "4",
        "attributes": {}
      },
      "id": 3
    }

Is there something I am missing? Please guide!

Make it easier to debug functions in IDEs

Currently, it is super hard to configure the local debug of the functions in PyCharm, moreover, I found it impossible without hacking the cli.py.

Here is a way to do it in PyCharm now:

  1. Add main function to the cli.py:
    if __name__ == '__main__':
        cli(sys.argv[1:])
  2. Create PyCharm Python config and choose functions_framework\cli.py as a script.
  3. Pass required params as Parameters in the configuration, e.g. --port 8088 --signature-type http, etc.
  4. Make sure to set the working directory to the project root (so that the functions framework is able to load the entry point function).
  5. Run the created configurations and use PyCharm debugger.

What we can to make life easier:

  1. Create __main__.py in the functions_framework module and allow running the module as the CLI. Or, add the main function to the cli.py itself, maybe worth having both.
  2. Create well-structured documentation about debugging functions locally with popular IDEs (PyCharm, VSCode, others?)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.