felixnext / python-functown Goto Github PK

Add additional serialization options, including:

• Thrift (Compact)?
• Protobuf
• FlatBuf
• Parquet
• CSV

Provide also a MimeResponse and MimeRequest decorator that allows conditional logic.

This is an extension of #28 (introduced in v1.1.0)

Add a bit of stuff to build ReadTheDocs code.

Follow basic tutorial here: https://docs.readthedocs.io/en/stable/tutorial/index.html

Decorators could be refactored to derive from a base class.

This could take care of certain abstractions (such as creating loggers) as well as automatically decomposing stacked operators (to avoid the @clean decorator at the outer layer.

References:

• https://machinelearningmastery.com/a-gentle-introduction-to-decorators-in-python
• https://python-patterns.guide/gang-of-four/decorator-pattern/
• https://realpython.com/primer-on-python-decorators/

This still requires a bunch of design and depends on #10 and #9

Right now KWs are always passed in the constructor here:

However, this should be dependent on the init-parameters (namely return_logs and enable_logger).

Allows to re-use the token.

Idea is to add a new field to the access class

We do not test the verify option for JWT right now, as that would required requests against a token provider backend.

The idea is to mock the requests with realistic responses coming from a token provider and increase test coverage.

#10 brings the lib to the alpha stage, but we should update the documentation to be up to date.

Specifically this includes:

• Bring code in readme up to date and simplify
• Add "Technical Debt" section to the readme
• Create docs folder and create a page for each sub-system
• Create a section for extension
• Create a section for contributors

The Application Insights telemetry client provides various exception handling and metric tracking abilities.

It would be great to directly integrate them into FuncTown (thereby allowing to track return types, exceptions and various metrics through App Insights)

Design for this has yet to be done

At least there should be an option to add additional error parameters to provide try catch support around this code:

Basically this fails with an error that is caught by ErrorHandler if token is invalid (this should not be the case)

It seems that OpenCensus and OpenTrace have merged into https://opentelemetry.io/. Make sure that we update the SDK used to ensure long-term support.

See getting started guide: https://opentelemetry.io/docs/instrumentation/python/getting-started/

Azure SDK Tutorial: https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/telemetry-application-insights?pivots=programming-language-python

Depends on #10

Add an example Azure Function that uses the library to display the functionality and test it more easily

The functions cannot be debugged due to internal links to functown instead of functown_local when the lib is not yet deployed (since the changes are too big). Need to deploy and debug then (for now).

See Blog Post here: https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-v2-python-programming-model/ba-p/3665168

The MetricHandler has different modes that can be applied to the create_metrics function.

These modes are currently not covered by unit-tests.

Goal would be to cover:

• All 4 different modes
• Expected behavior in case of namespace conflicts
• Proper values in metrics in case of Reuse

Many parameters in different functions are optional in a sense that None can be passed to them.

The goal of this task is to properly add the Optional Type to the typing hints in all signatures

By adding more decorators and functions to the base library, the list of dependent imports is growing.

The goal of this issue would be to split of parts of the library into seperate sub-modules that can be installed as needed. Obvious candidates for this would be:

• Serialization
• Insights

The install routine should then look like this:

pip install functown
pip install functown[insights]
pip install functown[serialization]

Right now the validate.ps1 script (introduced in #20 ) in the example folder is just a loose port. Make sure it is up to par with the validate.sh script.

It seems that json parsing is not as straight forward as it is for Protobuf, as it requires to wrap the Cpp code.

Is therefore a larger issue that is handled separately from main PR #38

This goes into the same direction as #5

It would be great to remove response boilerplate code from functions.

# current
def main(req):
  # ...
  data = {...}
  return HttpResponse(json.dumps(data), mimetype="application/json", status_code=200)

# updated
@json
def main(req):
  # ...
  return data

The json decorator might take optional parameters such as status_code. Should also be possible to extend to different structures later on (e.g. protobuf(pb=Obj) or multi which takes a Return object that contains the mimetype).

In particular mutli would allow to handle requests from #5 .

Depends on changes in #8

In that PR a logger is introduced that writes all log messages to a list.
Explore if that can be leveraged as a tag to ensure that we can also return all logs in the response!

When many decorators are used in parallel (in unittests for example) the is_first_decorator system can be flaky (maybe due to collisions in the id(func) space?).

This forced to enable the removal of variable keyword parameters to all function signatures in BaseDecorator. This is not ideal and should be fixed! (In that case shield the variable keyword removal again for only the function in is_first_decorator.

Note that this adjustment has to happen after the __increase_count call in line 199:

python-functown/functown/utils/base_decorator.py

Line 199 in 75617ec

self.__increase_count(id(self.func), id(execute))

In many cases we might want to use a stacked variant of decorators. In that case we want to have a function like:

def stacked(*args):
  dec1 = ArgsHandler()
  dec2 = AuthHandler(...)
  # ...
  return StackDecorator([dec1, dec2, ...])


@stacked(...)
def main(req, ...):
  # ....

This should make it easier to use decorators across multiple functions (and make function code itself a bit clearer).

Most of the structure can probably be derived from Insights Decorator

The MetricHandler in current state is a Singleton object. Meaning it will use one MeasurementMap for all metrics.
In most cases (esp in Azure Functions) this should not be a problem, since we only operate with one set of metrics at a time.

There might be edge cases though, where we want to have support for multiple MeasurementMaps? (Confirm that). If that is the case it would make sense to change the self._map module from a direct instance to a Dict[str, MeasurementMap] and create a default namespace.

If create_metrics is called with no namespace parameter this will use the default namespace, otherwise check if the namespace exists and if not create a new MeasurementMap.

This namespace parameter would then also need to be exposed to the Decorator. It might also require to create a thin wrapper object around MetricHandler (e.g. MetricNamespaceHandler or MetricNSHandler) to auto-pass the namespace for interactions with metrics inside the annotated function.

Bonus: We might also want to add a __namespace column automatically to all metrics and pass this information along in the TagMap.

Help Wanted

Please specify if this is a functionality that is really required!

Performance of azure functions can be improved with async. Build a decorator that can handle the async parts.

See documentation here: https://learn.microsoft.com/en-us/azure/azure-functions/python-scale-performance-reference#async

Also look into handling metric and other calls introduced in #10 via async (to avoid distraction from main runtime)

Exact design for this is still open.

Implement Getter to headers (such as ContentType) to make parsing easier.

Mock could be something like this:

from functown import RequestArgHandler, RequestHeaders as rh

args = RequestArgHandler(req)
value, meta = args.get_header(rh.ContentType)

The request currently supports json serialization.

Might add additional serializers that can be passed to RequestArgsHandler to retrieve body data.

Design for this still needs to be done but should support things like ProtoBuf (also look into overlap with RPC tools?)

Right now authorization has to be done manually.

However, we could also create an auth decorator that handles this kind of authorization and passes the JWT object along.

Likely depends on the overhaul of the Decorator System #10

Since we added serialization in release 1.1.0, we should have an Azure Function Example that demonstrates this

Due to dependency conflicts the test-coverage for the insights decorator is extremely low.

The goal would be to fix that. This requires to mock a bunch of functionality in opencensus.

It also requires to define a clearer structure for multiple view defined in parallel.

This will likely require changes in the metrics as well.

Note: Some of the unit tests already exists, but are flaky. Hence they are marked as pytest.mark.skip

It seems that the application order of decorators from StackDecorator is reversed.

We should change that (like a bug fix release)