quanted / cts_app Goto Github PK

All calculators but Measured seems to be working with /cts/testing/ page.

After the calculator code is refactored and ready to deploy, it needs to be tested with the current setup of celery and nodejs. Once that's working fine on the server, it'll be time to integrate those changes into the django channels migration.

Celery has been used prior to django channels for distributing requests onto various workers in parallel. If a user's job was being processed and they canceled it or left the page, a new request would be sent to cancel the jobs.

Now that django channels is used instead of celery, does this still need to be performed explicitly? Is there a way to cancel jobs in django channels?

hello world

As a developer, it'd be great to not only deploy CTS with django channels locally, but also in production mode as well with Docker/reverse proxy through nginx.

"Error" message always shows up at the bottom right of the CTS API docs, but the page loads fine.

Downloading a PDF is sometimes taking two clicks to activate, and when it is activated, it builds the PDF, but chokes when trying to return the response. NOTE: This isn't happening with firefox, just chrome.

`2017-02-07 07:13:46,513 DEBUG Col widths: ['16.6666666667%', '16.6666666667%', '16.6666666667%', '16.6666666667%', '16.6666666667%', '16.6666666667%']
2017-02-07 07:13:46,640 DEBUG Col widths: []
2017-02-07 07:13:46,641 WARNING is empty
u''
[07/Feb/2017 07:13:47] "POST /cts/pchemprop/pdf HTTP/1.1" 200 59485
Traceback (most recent call last):
File "C:\Python27\Lib\wsgiref\handlers.py", line 86, in run
self.finish_response()
File "C:\Python27\Lib\wsgiref\handlers.py", line 128, in finish_response
self.write(data)
File "C:\Python27\Lib\wsgiref\handlers.py", line 212, in write
self.send_headers()
File "C:\Python27\Lib\wsgiref\handlers.py", line 270, in send_headers
self.send_preamble()
File "C:\Python27\Lib\wsgiref\handlers.py", line 194, in send_preamble
'Date: %s\r\n' % format_date_time(time.time())
File "C:\Python27\Lib\socket.py", line 328, in write
self.flush()
File "C:\Python27\Lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine
[07/Feb/2017 07:13:47] "POST /cts/pchemprop/pdf HTTP/1.1" 500 59
Traceback (most recent call last):
File "C:\Python27\Lib\SocketServer.py", line 596, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python27\Lib\SocketServer.py", line 331, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python27\Lib\SocketServer.py", line 654, in init
self.finish()
File "C:\Python27\Lib\SocketServer.py", line 713, in finish
self.wfile.close()
File "C:\Python27\Lib\socket.py", line 283, in close
self.flush()
File "C:\Python27\Lib\socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 10053] An established connection was aborted by the software in your host machine

Exception happened during processing of request from ('127.0.0.1', 8047)
----------------------------------------`

Test CTS Development Setup document on local machine to verify the document's effectiveness at guiding one through the setup process.

Document located on google drive, shared with ubertool_dev group.

After the past few weeks of dockerizing, etc., CTS API isn't loading, but that's probably just a templates thing. The main issue now is not using a hardcoded hostname for the swagger.json file. Comments aren't allowed in there, so it might be necessary to have two different files? When on the server, or even just within the EPA VPN, the 134 address for server 1 should be used to access CTS API. If running outside the EPA VPN, then it'll be whatever address the CTS app is hosted (probably localhost).

"timed out" values are populating the output CSV at /cts/testing/ despite them return values when testing them in the p-chem workflow of the main CTS pages.

Issue or moreso a question relating to EPI refactor issue #31

Often I find that EPI Suite returns a value of 10.00 for KOC. Should this be validating at the CTS backend and return a message to the user in the table cell instead of 10.00?

In a seemingly random pattern, CTS validation page at /cts/testing/ sometimes has nothing for water solubility values for ChemAxon in the CSV output file.

As a developer, I want to consolidate cts_calcs to recycle as much code as possible and to clearly (and in a single location) define all static content and POST/GET types per calculator. This refactor is an essential step before migrating to django channels and will also enable cts_calcs to reside in ONE place instead of in cts_celery and cts_app, making code maintainability much simpler.

Tried to change too many things at once and blew it...

Currently, master branch runs like gold on server1, but with the settings/wsgi attempted changes to cts-node-redis branch and the great reduction in unused code has caused an elusive "tasks" import error when trying to restart the apache server.

Should be easily resolved with step-wise changes to server instead one big 'ol dump.

Implement unit tests similar to ubertool.

When testing a request for 4 generations of transformation products for nicotine, a 504 timeout error is returned after about 30 seconds; this doesn't happen locally, but only in production with nginx/uwsgi.

uwsgi/django level error:

nginx level error:

On the SPARC website, the pKas in question show up as "out of range", and show up in CTS as "-1000". These values should not show up in CTS. Example chemical: 1,3,5,7-tetraazatricyclo[3.3.1.1³,⁷]decane (C1N2CN3CN1CN(C2)C3) and 3-(4-chlorophenyl)-1-(3,4-dichlorophenyl)urea (O=C(NC1=CC=C(Cl)C=C1)NC2=CC=C(Cl)C(Cl)=C2).

Currently, transformation products calls are performed with the chemaxon worker. Moving those requests to a separate metabolizer worker would free up chemaxon to only handle speciation and p-chem requests, which should increase performance for workflows like gentrans.

https://envipath.org/

Logic for breaking up products by generation is breaking at products with IDs > 9. For example, left-clicking product 1.10+ for nicotine displays the reaction pathway incorrectly and triggers the following JS error:
Uncaught TypeError: Cannot read property 'trim' of undefined

Here's the line: var img_obj = $.parseHTML(current_node.name.trim())[0];

TODO: Instead of substringing a range, use a split instead using "." as the place to split the string.

So on the generate transformation pathways output page, the user can right-click a metabolite and request p-chem data. This (sort of) works for one, but would be nice if it could be extended to selected multiple metabolites, and/or have a "select all" feature.

Containerize CTS with Docker, encapsulating the django, redis, nodejs, etc. that's used in CTS.

As a user, I want to be able to download the metadata associated with a workflow run in CTS. The metadata file would include all the SMILES processing, data validating, rounding, unit conversions, etc. CTS performs on the backend.

Currently working on retries and data validation for the SPARC calculator.

For gentrans p-chem requests of metabolites, don't send the JIT node/product objects to the backend. With large requests, a "request too large" is triggered, which will most likely be resolved with this change.

For example, requesting all calcs, all props for all 4 generations of nicotine triggers the "request too large" error.

For a while now, the p-chem data table for the pchemprop workflow is totally collapsed, with the words overflowing across the table. Attempts have been made to modify the PDF CSS in ctsGenerateReport.py (located in cts_app/views/), but alas, the table cells are still like 1px wide...

Now that CTS is using django channels, the frontend websocket code needs to be using the generic WebSocket object and not the socket.io wrapper library.

Reference File: https://github.com/quanted/templates_qed/blob/dev/cts/cts_pchemprop_requests.html

The p-chem properties calculations needs an "average" column that computes, well, computes an average based on the calculators' values.

Since the cts_calcs refactor, transformation products requests will now be handled with its own calculator. Currently, it's handled by the chemaxon calculator/worker, but moving transformation products requests to a separate worker will help CTS run more quickly for the gentrans workflow.

Reference file: https://github.com/popenc/cts_calcs/blob/channels/calculator_metabolizer.py

Since refactoring CTS calc code, the CTS API code isn't handling HTTP request types, which is resulting in errors on the API docs page.

Spending way too much time trying to resolve this issue... Chemaxon celery worker container is successfully making calls to nginx container and returning data, but now the CSV won't download, triggering a "502 Bad Gateway" error on the server. Per usual, works locally, so a test setup needs to be devised to get better details about this issue. Testing CTS with Docker/SSL would be a good start.

As a developer, I'd like to refactor the Measured code within CTS to get it ready for switching to django channels instead of having to use celery and nodejs.

TEST calculations are currently pulling the FDA results. Todd Martin recommended we use the Hierarchical method. It is faster and potentially more accurate than FDA.

The last workflow's output has a canvas which displays a spacetree of metabolites. There needs to be an option to zoom out so the user can see all the metabolites without having to pan.

this is Michaela creating a test issue

It'll be refreshing to have some legitimate stress testing results for CTS, testing common user patterns with multiple users at a time. A report of some results for Wednesday is the aim.

After pchemprop batch mode run of even just 3 molecules (getting all calcs and props), the CSV downloads up until 50+%, then a 413 error is returned.

Added "client_max_body_size 4M;" to server 1's nginx.conf, but the error was not resolved.

The values that CTS provides indicate that when a melting point is being specified to SPARC, the option to "assume not solid at 25 degrees C" is being used, which overrides any actual specified melting point. This option should NOT be used for specifying melting points - the standard procedure for entering a melting point into a calculator should be used (such as Measured if available, otherwise use the average of TEST FDA, Hierarchical, and Group methods).

As a developer, I'd like to refactor the TEST code within CTS to get it ready for switching to django channels instead of having to use celery and nodejs.

As a developer, I'd like to refactor the EPI Suite code within CTS to get it ready for switching to django channels instead of having to use celery and nodejs.