Events dispatched to logx.* sub-domain can take a response time of 0.407 sec.

Are these endpoints service critical?
Is there any reason not to dispatch events to them if the JS client is already logging I think?

@bradtaylorsf @dsiroker @koomen @travisbeck @loganlinn

would it be possible to send an exception to the error handler when schema validation is enabled but the datafile provided is not valid? current behavior is log error but continue as is.

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 7717: character maps to <undefined>

Currently when calling event_dispatcher it's a blocking call. This network request slows down calls to activate and track, which can add up if we're making a lot of calls. I wonder if it's possible to make it in a task to make the dispatch_event call smoother.

https://github.com/optimizely/python-sdk/blob/master/optimizely/event_dispatcher.py#L34-L40

Ref : python/cpython@e250061

tests/benchmarking/benchmarking_tests.py
28:        start_time = time.clock()
30:        end_time = time.clock()
34:        start_time = time.clock()
36:        end_time = time.clock()
40:        start_time = time.clock()
42:        end_time = time.clock()
47:        start_time = time.clock()
51:        end_time = time.clock()
56:        start_time = time.clock()
58:        end_time = time.clock()
63:        start_time = time.clock()
65:        end_time = time.clock()
70:        start_time = time.clock()
72:        end_time = time.clock()
77:        start_time = time.clock()
79:        end_time = time.clock()
84:        start_time = time.clock()
88:        end_time = time.clock()
93:        start_time = time.clock()
95:        end_time = time.clock()
100:        start_time = time.clock()
102:        end_time = time.clock()
107:        start_time = time.clock()
109:        end_time = time.clock()
114:        start_time = time.clock()
116:        end_time = time.clock()
120:        start_time = time.clock()
122:        end_time = time.clock()
126:        start_time = time.clock()
130:        end_time = time.clock()
134:        start_time = time.clock()
136:        end_time = time.clock()
140:        start_time = time.clock()
142:        end_time = time.clock()
146:        start_time = time.clock()
150:        end_time = time.clock()
154:        start_time = time.clock()
156:        end_time = time.clock()
160:        start_time = time.clock()
164:        end_time = time.clock()

The Python SDK uses the package six but does not appear to require it in any of the requirements

If you overflow the event_queue in BatchEventProcessor, the queue.Full event is caught and produces a debug level message in the logger.

        try:
            self.event_queue.put_nowait(user_event)
        except queue.Full:
            self.logger.debug(
                'Payload not accepted by the queue. Current size: {}'.format(str(self.event_queue.qsize()))
            )

This solution is problematic for at least two reasons:

1. Data loss is not a debug level issue. It should be at least warn or maybe error.
2. If the logger is not set, this code silently fails enqueuing the event. Unless the event source owns the queue, there is no way for it to know that operation failed.

We have many features that are set to 'off' in production, which have historically been included in the datafile. Today, these features went missing from the datafile, which seems to have started logging tons of "Feature "my_feature" is not in datafile." error. This was very surprising, and the increased rate of error logs caused alerts to be sent to engineers.

Our temporary solution: find all flags where production is set to 'off' and change it to 'on' and '0%'. This seems to be a distinction without a difference, but it makes the torrent of error messages go away.

We're on the latest Optimizely SDK version, and haven't upgraded recently, which leads me to believe that this was a back-end change. Was this intended to cause errors, or was this a bug in the Python SDK?

Hi,
According to this implementation, adding a variation to a running experiment will change how existing users are allocated to their variations.
How can that be fixed?

Thanks

there is no link to the Python SDK docs, and the Full-Stack Docs don't currently have a working link to the Python SDK

Sending an event with the revenue tag set generates a warning log message. It seems to be looking for a numeric value tag, even though that tag is not set in the call.

The revenue does appear to be tracked despite the warning, and the value tag works as expected with no output.

Optimizely SDK version 3.2.0

Example code:

optimizely_client = Optimizely(datafile)
optimizely_client.track(event_key='some_event', user_id='abc123', event_tags={'revenue': 456})

# WARNING:optimizely.logger.NoOpLogger:The provided numeric metric value None is in an invalid format and will not be sent to results.

I just read that user profiles for the backend-sdk is not currently supported and that sdk consumers are required to roll their own variation stickiness.

https://developers.optimizely.com/x/solutions/sdks/reference/index.html?language=python#profiles

I am wondering if/when this support is planned to be added to the SDK as we are currently evaluating the implementation of optimizely within our platform? I think this question is also related to: #49

Thank you and looking forward to your response.

Thanks for providing this package. It looks like this package pins (ranged) versions for cryptography and pyopenssl, yet doesn't actually use those packages (directly or transitively). This creates unnecessary dependency locks for consumers of this package as they're forced to install those packages even if nothing else is using them.

In turn, this means that consumers of this package may be exposed to security vulnerabilities in those packages due to their inclusion in this package.

If these packages are truly unused (I wasn't able to find a reference to them in the source of this project, nor are they mentioned by any of the other packages this one depends upon), please could they be removed from the package dependencies and an updated package be published?

Hi, we're considering pulling in this SDK in several Python projects. Because we build those projects in pretty bare containers, the compiled dependencies here (mmh3, and cryptography from requests[security]) mean that we need to install some additional OS packages at build time. That's not a showstopper, but some questions came up:

• Is requests[security] needed? From https://github.com/psf/requests/pull/4825/files, it seems like it's probably not needed anymore, since fixes have been introduced in the standard library in Python 2.7.9 and 3.4.3.

• There is already fallback code in case mmh3 isn't installed:

  python-sdk/optimizely/bucketer.py

  Lines 16 to 19 in d3ffe33

  	try:
  	import mmh3
  	except ImportError:
  	from .lib import pymmh3 as mmh3

  , but the mmh3 requirement doesn't seem to be optional. Could it be made optional?

• Do you have any details on the performance penalty of using the pure-Python Murmur3 implementation, as compared to the compiled version? I suspect that, in our use case (AWS Lambda, process per request), it may not be an issue.

I am trying to use the Optimizely SDK, but my application also requires jsonschema >= 4.0.1, because it includes support for the newer JSONSchema specification drafts. Optimizely SDK requires exactly jsonschema = 3.2.0, so I am not able to use them together.

Is it possible to un-pin the version of jsonschema and pyrsistent?

Pinning dependencies in libraries (as opposed to "applications") is very unusual, and puts a lot of restrictions on users.

Normally one would expect that a minimum version is set, of course, but not an exact pinned version!

And in this case, it also seems like the version pin isn't even necessary, since the last version bump didn't seem to require any code changes.

For example, it is effectively impossible to use Optimizely SDK and support OpenAPI Specification 3.1 in the same application. This is because OAS 3.1 specifies a JSONSchema draft that is not available in jsonschema 3.2.0. Practically, most schemas can be parsed with Draft 7, which is available in 3.2.0, so you can mostly work around the problem. But it was surprising to find this dependency conflict at all.

Is there a compelling reason to keep dependencies pinned to exact patch versions in an SDK/library? It would make this much mroe user-friendly if these constraints were relaxed.

I see that dependency versions were discussed in the past, but I didn't see any discussion of un-pinning.

The default EventDispatcher only handles network related errors and ignores HTTP errors(500, 503, etc) without any logging messages

try:
  if event.http_verb == enums.HTTPVerbs.GET:
    requests.get(event.url, params=event.params, timeout=10)
  elif event.http_verb == enums.HTTPVerbs.POST:
    requests.post(event.url, json=event.params, headers=event.headers, timeout=10)
except request_exception.RequestException as error:
  logging.error('Dispatch event failed. Error: %s' % str(error))

HTTP errors in requests do not raise Exceptions

In [2]: requests.get('https://httpbin.org/status/500')
Out[2]: <Response [500]>   # This is an object, not an raise error therefore not caught by try/except

You would need to raise from the response

response = requests.get(event.url, params=event.params, timeout=10)
In [4]: response.raise_for_status()
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-4-46d4bf99ee16> in <module>()
----> 1 requests.get('https://httpbin.org/status/500').raise_for_status()

python3.6/site-packages/requests/models.py in raise_for_status(self)
    937 
    938         if http_error_msg:
--> 939             raise HTTPError(http_error_msg, response=self)
    940 
    941     def close(self):

HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://httpbin.org/status/500

do you support python 3.5 and python 3.6 ?

The following code may ask for an item from the event_queue with a negative timeout interval.

class BatchEventProcessor(BaseEventProcessor):
    def _run(self):
        try:
            while True:
                if self._get_time() >= self.flushing_interval_deadline:
                    self._flush_batch()
                    self.flushing_interval_deadline = self._get_time() + \
                        self._get_time(self.flush_interval.total_seconds())
                    self.logger.debug('Flush interval deadline. Flushed batch.')
                try:
                    interval = self.flushing_interval_deadline - self._get_time()
                    item = self.event_queue.get(True, interval) ## interval can be negative

If the flushing_interval_deadline is between the first call of _get_time() and the second call of _get_time() then interval will be negative.

Proposed fix:

class BatchEventProcessor(BaseEventProcessor):
    def _run(self):
        try:
            while True:
                loop_time = self._get_time()  ## only call get_time once per loop iteration
                if loop_time >= self.flushing_interval_deadline:
                    self._flush_batch()
                    self.flushing_interval_deadline = self._get_time() + \
                        self._get_time(self.flush_interval.total_seconds())
                    self.logger.debug('Flush interval deadline. Flushed batch.')
                try:
                    interval = self.flushing_interval_deadline - loop_time
                    item = self.event_queue.get(True, interval) 

I'm currently trying to integrate with optimizely from a backend code. I get this error which causes the application to fail

Fetching datafile from https://cdn.optimizely.com/datafiles/XXXXXXXXXXX.json failed. Error: 

HTTPSConnectionPool(host='cdn.optimizely.com', port=443): Max retries exceeded with url:
 /datafiles/VhiH4avQ1x9uYuEGumBvf.json (Caused by SSLError(SSLError(1, '
[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1123)')))

Using version optimizely-sdk = "^3.8.0"

The default settings for

• Blocking request timeout
• Polling interval
• Request timeout
  are all default to 10 seconds

Whilst the library offers the ability to override the Blocking and Polling settings, the request timeout is not exposed.
Can you confirm that this is a design decision, or would you be happy to accept our PR to expose the timeout setting also?

The reason this is important to us is because of the need to limit and have predictable message processing duration in our high-throughput system, and to de-risk ourselves from any network problem.

We would be happy to accept the risk that if we will fall back to the default variant on timeout, it can cause a discrepancy in the recorded metrics of the fallback variant being used instead of the variant that may be returned eventually; we expect this the occurrence should be rare.