Giter VIP home page Giter VIP logo

mastcasjobs's Introduction

This provides a simple interface to the MAST CasJobs server (home of GALEX, Kepler, the Hubble Source Catalog, PanSTARRS, etc.) using Dan Foreman-Mackey's casjobs interface.

Installation

Install current versions of both the mastcasjobs and casjobs modules:

pip install mastcasjobs

If you want to do development on the mastcasjobs module, clone it and then install it using:

pip install .

Usage

Here is an example query that does a cone search for PS1 objects within 50 arc-sec of coordinates RA=187.706, Dec=12.391 (in degrees):

import mastcasjobs

query = """select o.objID, o.raMean, o.decMean,
o.nDetections, o.ng, o.nr, o.ni, o.nz, o.ny,
m.gMeanPSFMag, m.rMeanPSFMag, m.iMeanPSFMag, m.zMeanPSFMag, m.yMeanPSFMag
from fGetNearbyObjEq(187.706,12.391,50.0/60.0) nb
inner join ObjectThin o on o.objid=nb.objid and o.nDetections>1
inner join MeanObject m on o.objid=m.objid and o.uniquePspsOBid=m.uniquePspsOBid
"""

# user is your MAST Casjobs username
# pwd is your Casjobs password
# These can also come from the CASJOBS_USERID and CASJOBS_PW environment variables,
# in which case you do not need the username or password parameters.
# Create a Casjobs account at <https://mastweb.stsci.edu/ps1casjobs/CreateAccount.aspx>
#   if you do not already have one.

user = "myusername"
pwd = "My super secret password"

jobs = mastcasjobs.MastCasJobs(username=user, password=pwd, context="PanSTARRS_DR2")
results = jobs.quick(query, task_name="python cone search")
print(results)

Note that the results of the quick query are by default returned as an astropy table. You can add the optional parameter astropy=False to get a string instead.

The jobs object has other useful methods that allow you to do almost all the queries that you can run using the web interface. Use help(jobs.function) to get details. Some of the commonly used functions include:

quick:Run short queries that execute in less than 1 minute.
submit:Submit a long-running query.
status, monitor, cancel:Monitor a submitted query.
fast_table:Fast retrieval of data from a MyDB table (works only on MAST Casjobs).
get_table:Retrieve a small or large MyDB table (slower but works in other Casjobs installations too).
list_tables:List tables in MyDB (or in another context).
drop_table_if_exists:Delete a table from your MyDB (if it exists).

Requirements

This relies on the casjobs (version 0.0.2 or newer) and requests modules.

Since Python 2.7 is no longer supported, the installation of this version of the software requires Python 3.5 or greater. The software actually still runs in Python 2.7, but you will have to install it manually.

License

MIT

mastcasjobs's People

Contributors

jannisne avatar rlwastro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

mastcasjobs's Issues

Using Windows results in ExpatError: not well-formed (invalid token): line 14, column 132

Hello Rick, I'm getting an ExpatError running the sample code in the repo.

Code

import mastcasjobs
import getpass
import os

if not os.environ.get('CASJOBS_USERID'):
    os.environ['CASJOBS_USERID'] = input('Enter Casjobs UserID:')
if not os.environ.get('CASJOBS_PW'):
    os.environ['CASJOBS_PW'] = getpass.getpass('Enter Casjobs password:')

query = """select o.objID, o.raMean, o.decMean,
o.nDetections, o.ng, o.nr, o.ni, o.nz, o.ny,
m.gMeanPSFMag, m.rMeanPSFMag, m.iMeanPSFMag, m.zMeanPSFMag, m.yMeanPSFMag
from fGetNearbyObjEq(187.706,12.391,50.0/60.0) nb
inner join ObjectThin o on o.objid=nb.objid and o.nDetections>1
inner join MeanObject m on o.objid=m.objid and o.uniquePspsOBid=m.uniquePspsOBid
"""

jobs = mastcasjobs.MastCasJobs(context="PanSTARRS_DR2")
results = jobs.quick(query, task_name="python cone search")
print(results)

Traceback

---------------------------------------------------------------------------
ExpatError                                Traceback (most recent call last)
Input In [15], in <cell line: 11>()
      1 query = """select o.objID, o.raMean, o.decMean,
      2 o.nDetections, o.ng, o.nr, o.ni, o.nz, o.ny,
      3 m.gMeanPSFMag, m.rMeanPSFMag, m.iMeanPSFMag, m.zMeanPSFMag, m.yMeanPSFMag
   (...)
      6 inner join MeanObject m on o.objid=m.objid and o.uniquePspsOBid=m.uniquePspsOBid
      7 """
      9 jobs = mastcasjobs.MastCasJobs(context="PanSTARRS_DR2")
---> 11 results = jobs.quick(query, task_name="python cone search")
     12 print(results)

File ~\anaconda3\envs\stsci\lib\site-packages\mastcasjobs\__init__.py:152, in MastCasJobs.quick(self, q, context, task_name, system, astropy)
    131 def quick(self, q, context=None, task_name="quickie", system=False, astropy=True):
    132     """
    133     Run a quick job. Like CasJobs method but adds astropy option.
    134 
   (...)
    150 
    151     """
--> 152     results = super(MastCasJobs,self).quick(q, context=context, task_name=task_name, system=system)
    153     if astropy:
    154         return MastCasJobs.convert_quick_table(results)

File ~\anaconda3\envs\stsci\lib\site-packages\casjobs.py:176, in CasJobs.quick(self, q, context, task_name, system)
    173 params = {"qry": q, "context": context, "taskname": task_name,
    174         "isSystem": system}
    175 r = self._send_request("ExecuteQuickJob", params=params)
--> 176 return self._parse_single(r.text, "string")

File ~\anaconda3\envs\stsci\lib\site-packages\casjobs.py:148, in CasJobs._parse_single(self, text, tagname)
    134 def _parse_single(self, text, tagname):
    135     """
    136     A hack to get the content of the XML responses from the CAS server.
    137 
   (...)
    146 
    147     """
--> 148     return minidom.parseString(text)\
    149             .getElementsByTagName(tagname)[0].firstChild.data

File ~\anaconda3\envs\stsci\lib\xml\dom\minidom.py:1998, in parseString(string, parser)
   1996 if parser is None:
   1997     from xml.dom import expatbuilder
-> 1998     return expatbuilder.parseString(string)
   1999 else:
   2000     from xml.dom import pulldom

File ~\anaconda3\envs\stsci\lib\xml\dom\expatbuilder.py:925, in parseString(string, namespaces)
    923 else:
    924     builder = ExpatBuilder()
--> 925 return builder.parseString(string)

File ~\anaconda3\envs\stsci\lib\xml\dom\expatbuilder.py:223, in ExpatBuilder.parseString(self, string)
    221 parser = self.getParser()
    222 try:
--> 223     parser.Parse(string, True)
    224     self._setup_subset(string)
    225 except ParseEscape:

ExpatError: not well-formed (invalid token): line 14, column 132

Additionally

I also get the same error running the first query in Hubble Source Catalog SWEEPS Proper Motion Notebook. In this case, I tried the slow, and the fast version.

Traceback 2

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [11], in <cell line: 4>()
      5     print("Retrieving table MyDB.{} (if it exists)".format(DBtable))
----> 6     tab = jobs.fast_table(DBtable, verbose=True)
      7 except ValueError:

File ~\anaconda3\envs\stsci\lib\site-packages\mastcasjobs\__init__.py:208, in MastCasJobs.fast_table(self, table, verbose)
    206 except Exception as e:
    207     # raise ValueError("table MyDB.{} not found".format(table)) from None
--> 208     raise_from(ValueError("table MyDB.{} not found".format(table)), None)
    209 # get table from the quick_casjobs.cgi service

File <string>:3, in raise_from(value, from_value)

ValueError: table MyDB.SWEEPS not found

During handling of the above exception, another exception occurred:

ExpatError                                Traceback (most recent call last)
Input In [11], in <cell line: 4>()
      8 print("Table MyDB.{} not found, running query to create it".format(DBtable))
     10 # drop table if it already exists
---> 11 jobs.drop_table_if_exists(DBtable)
     13 #get main information
     14 query = """
     15     select a.ObjID,  RA=a.raMean, Dec=a.decMean, RAerr=a.raMeanErr, Decerr=a.decMeanErr,
     16         c.NumFilters, c.NumVisits,
   (...)
     30      join AstromSumPropMagAper2Cat c on a.ObjID=c.ObjID
     31 """.format(DBtable)

File ~\anaconda3\envs\stsci\lib\site-packages\mastcasjobs\__init__.py:129, in MastCasJobs.drop_table_if_exists(self, table)
    120 def drop_table_if_exists(self, table):
    121     """
    122     Drop table from MyDB without an error if it does not exist
    123 
   (...)
    127 
    128     """   
--> 129     results = self.quick("DROP TABLE IF EXISTS {}".format(table),context="MYDB")

File ~\anaconda3\envs\stsci\lib\site-packages\mastcasjobs\__init__.py:152, in MastCasJobs.quick(self, q, context, task_name, system, astropy)
    131 def quick(self, q, context=None, task_name="quickie", system=False, astropy=True):
    132     """
    133     Run a quick job. Like CasJobs method but adds astropy option.
    134 
   (...)
    150 
    151     """
--> 152     results = super(MastCasJobs,self).quick(q, context=context, task_name=task_name, system=system)
    153     if astropy:
    154         return MastCasJobs.convert_quick_table(results)

File ~\anaconda3\envs\stsci\lib\site-packages\casjobs.py:176, in CasJobs.quick(self, q, context, task_name, system)
    173 params = {"qry": q, "context": context, "taskname": task_name,
    174         "isSystem": system}
    175 r = self._send_request("ExecuteQuickJob", params=params)
--> 176 return self._parse_single(r.text, "string")

File ~\anaconda3\envs\stsci\lib\site-packages\casjobs.py:148, in CasJobs._parse_single(self, text, tagname)
    134 def _parse_single(self, text, tagname):
    135     """
    136     A hack to get the content of the XML responses from the CAS server.
    137 
   (...)
    146 
    147     """
--> 148     return minidom.parseString(text)\
    149             .getElementsByTagName(tagname)[0].firstChild.data

File ~\anaconda3\envs\stsci\lib\xml\dom\minidom.py:1998, in parseString(string, parser)
   1996 if parser is None:
   1997     from xml.dom import expatbuilder
-> 1998     return expatbuilder.parseString(string)
   1999 else:
   2000     from xml.dom import pulldom

File ~\anaconda3\envs\stsci\lib\xml\dom\expatbuilder.py:925, in parseString(string, namespaces)
    923 else:
    924     builder = ExpatBuilder()
--> 925 return builder.parseString(string)

File ~\anaconda3\envs\stsci\lib\xml\dom\expatbuilder.py:223, in ExpatBuilder.parseString(self, string)
    221 parser = self.getParser()
    222 try:
--> 223     parser.Parse(string, True)
    224     self._setup_subset(string)
    225 except ParseEscape:

ExpatError: not well-formed (invalid token): line 14, column 132

Versions - yml file

  • conda environment running Jupyter Lab
name: stsci
channels:
  - defaults
  - conda-forge
dependencies:
  - appdirs=1.4.4=pyhd3eb1b0_0
  - asdf=2.11.1=pyhd8ed1ab_0
  - asdf-standard=1.0.2=pyhd8ed1ab_0
  - asdf-transform-schemas=0.2.2=pyhd8ed1ab_0
  - astropy=5.0.4=py310h9128911_0
  - astroquery=0.4.6=pyhd8ed1ab_0
  - asttokens=2.0.5=pyhd3eb1b0_0
  - atomicwrites=1.4.0=py_0
  - attrs=21.4.0=pyhd3eb1b0_0
  - backcall=0.2.0=pyhd3eb1b0_0
  - beautifulsoup4=4.11.1=py310haa95532_0
  - black=19.10b0=py_0
  - blas=1.0=mkl
  - blosc=1.21.0=h19a0ad4_0
  - bottleneck=1.3.4=py310h9128911_0
  - brotli=1.0.9=ha925a31_2
  - brotlipy=0.7.0=py310h2bbff1b_1002
  - bzip2=1.0.8=he774522_0
  - ca-certificates=2022.4.26=haa95532_0
  - certifi=2020.6.20=pyhd3eb1b0_3
  - cffi=1.15.0=py310h2bbff1b_1
  - cfitsio=3.470=he774522_6
  - charls=2.2.0=h6c2663c_0
  - charset-normalizer=2.0.4=pyhd3eb1b0_0
  - click=8.0.4=py310haa95532_0
  - cloudpickle=2.0.0=pyhd3eb1b0_0
  - colorama=0.4.4=pyhd3eb1b0_0
  - coverage=6.3.2=py310h2bbff1b_0
  - cryptography=37.0.1=py310h21b164f_0
  - cycler=0.11.0=pyhd3eb1b0_0
  - cytoolz=0.11.0=py310h2bbff1b_0
  - dask-core=2022.2.1=pyhd3eb1b0_0
  - debugpy=1.5.1=py310hd77b12b_0
  - decorator=5.1.1=pyhd3eb1b0_0
  - eigen=3.3.7=h59b6b97_1
  - entrypoints=0.4=py310haa95532_0
  - et_xmlfile=1.1.0=py310haa95532_0
  - executing=0.8.3=pyhd3eb1b0_0
  - fonttools=4.25.0=pyhd3eb1b0_0
  - freetype=2.10.4=hd328e21_0
  - fsspec=2022.2.0=pyhd3eb1b0_0
  - giflib=5.2.1=h62dcd97_0
  - glib=2.69.1=h5dc1a3c_1
  - gst-plugins-base=1.18.5=h9e645db_0
  - gstreamer=1.18.5=hd78058f_0
  - gwcs=0.17.1=pyhd8ed1ab_0
  - hdf5=1.10.6=h7ebc959_0
  - html5lib=1.1=pyhd3eb1b0_0
  - hypothesis=6.29.3=pyhd3eb1b0_0
  - icc_rt=2019.0.0=h0cc432a_1
  - icu=58.2=ha925a31_3
  - idna=3.3=pyhd3eb1b0_0
  - imagecodecs=2021.8.26=py310h6cc35ac_0
  - imageio=2.9.0=pyhd3eb1b0_0
  - importlib-metadata=4.11.3=py310haa95532_0
  - importlib-resources=5.7.1=pyhd8ed1ab_0
  - importlib_metadata=4.11.3=hd3eb1b0_0
  - importlib_resources=5.7.1=pyhd8ed1ab_0
  - iniconfig=1.1.1=pyhd3eb1b0_0
  - intel-openmp=2021.4.0=haa95532_3556
  - ipykernel=6.9.1=py310haa95532_0
  - ipython=8.3.0=py310haa95532_0
  - jedi=0.18.1=py310haa95532_1
  - jmespath=0.10.0=pyhd3eb1b0_0
  - joblib=1.1.0=pyhd3eb1b0_0
  - jpeg=9e=h2bbff1b_0
  - jsonschema=4.4.0=py310haa95532_0
  - jupyter_client=7.2.2=py310haa95532_0
  - jupyter_core=4.10.0=py310haa95532_0
  - keyring=23.4.0=py310haa95532_0
  - kiwisolver=1.3.1=py310hd77b12b_0
  - lcms2=2.12=h83e58a3_0
  - lerc=3.0=hd77b12b_0
  - libaec=1.0.4=h33f27b4_1
  - libdeflate=1.8=h2bbff1b_5
  - libffi=3.4.2=h604cdb4_1
  - libiconv=1.16=h2bbff1b_2
  - libogg=1.3.5=h2bbff1b_1
  - libpng=1.6.37=h2a8f88b_0
  - libprotobuf=3.5.1=he0781b1_0
  - libtiff=4.2.0=hd0e1b90_0
  - libvorbis=1.3.7=he774522_0
  - libwebp=1.2.2=h2bbff1b_0
  - libwebp-base=1.2.2=h2bbff1b_0
  - libxml2=2.9.12=h0ad7f3c_1
  - libxslt=1.1.34=he774522_0
  - libzopfli=1.0.3=ha925a31_0
  - locket=0.2.1=py310haa95532_2
  - lxml=4.8.0=py310h1985fb9_0
  - lz4-c=1.9.3=h2bbff1b_1
  - matplotlib=3.5.1=py310haa95532_1
  - matplotlib-base=3.5.1=py310hd77b12b_1
  - matplotlib-inline=0.1.2=pyhd3eb1b0_2
  - mkl=2021.4.0=haa95532_640
  - mkl-service=2.4.0=py310h2bbff1b_0
  - mkl_fft=1.3.1=py310ha0764ea_0
  - mkl_random=1.2.2=py310h4ed8f06_0
  - mplcursors=0.5.1=pyhd8ed1ab_0
  - mplfinance=0.12.9b0=pyhd8ed1ab_0
  - munkres=1.1.4=py_0
  - mypy_extensions=0.4.3=py310haa95532_1
  - natsort=7.1.1=pyhd3eb1b0_0
  - nest-asyncio=1.5.5=py310haa95532_0
  - networkx=2.7.1=pyhd3eb1b0_0
  - nltk=3.7=pyhd3eb1b0_0
  - numexpr=2.8.1=py310hb57aa6b_1
  - numpy=1.21.5=py310h6d2d95c_2
  - numpy-base=1.21.5=py310h206c741_2
  - opencv=4.5.4=py310hc26a207_3
  - openjpeg=2.4.0=h4fc8c34_0
  - openpyxl=3.0.9=pyhd3eb1b0_0
  - openssl=1.1.1o=h2bbff1b_0
  - packaging=21.3=pyhd3eb1b0_0
  - pandas=1.4.2=py310hd77b12b_0
  - pandas-datareader=0.10.0=pyhd3eb1b0_0
  - parso=0.8.3=pyhd3eb1b0_0
  - partd=1.2.0=pyhd3eb1b0_1
  - pathspec=0.7.0=py_0
  - patsy=0.5.2=py310haa95532_1
  - pcre=8.45=hd77b12b_0
  - photutils=1.4.0=py310h2873277_0
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pillow=9.0.1=py310hdc2b20a_0
  - pip=21.2.4=py310haa95532_0
  - pluggy=1.0.0=py310haa95532_1
  - prompt-toolkit=3.0.20=pyhd3eb1b0_0
  - psutil=5.8.0=py310h2bbff1b_1
  - pure_eval=0.2.2=pyhd3eb1b0_0
  - py=1.11.0=pyhd3eb1b0_0
  - pycparser=2.21=pyhd3eb1b0_0
  - pyerfa=2.0.0=py310h2bbff1b_0
  - pygments=2.11.2=pyhd3eb1b0_0
  - pyopenssl=22.0.0=pyhd3eb1b0_0
  - pyparsing=3.0.4=pyhd3eb1b0_0
  - pyqt=5.9.2=py310hd77b12b_6
  - pyrsistent=0.18.0=py310h2bbff1b_0
  - pysocks=1.7.1=py310haa95532_0
  - pytest=7.1.1=py310haa95532_0
  - pytest-arraydiff=0.3=py310h9909e9c_0
  - pytest-astropy=0.9.0=pyhd3eb1b0_0
  - pytest-astropy-header=0.1.2=pyhd3eb1b0_0
  - pytest-cov=3.0.0=pyhd3eb1b0_0
  - pytest-doctestplus=0.11.1=pyhd3eb1b0_0
  - pytest-filter-subpackage=0.1.1=pyhd3eb1b0_0
  - pytest-mock=3.6.1=pyhd3eb1b0_0
  - pytest-openfiles=0.5.0=pyhd3eb1b0_0
  - pytest-remotedata=0.3.2=py310haa95532_0
  - python=3.10.4=hbb2ffb3_0
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python_abi=3.10=2_cp310
  - pytz=2021.3=pyhd3eb1b0_0
  - pyvo=1.3=pyhd8ed1ab_0
  - pywavelets=1.3.0=py310h2bbff1b_0
  - pywin32=302=py310h2bbff1b_2
  - pywin32-ctypes=0.2.0=py310haa95532_1000
  - pyyaml=6.0=py310h2bbff1b_0
  - pyzmq=22.3.0=py310hd77b12b_2
  - qt=5.9.7=vc14h73c81de_0
  - regex=2022.3.15=py310h2bbff1b_0
  - requests=2.27.1=pyhd3eb1b0_0
  - scikit-image=0.19.2=py310h4ed8f06_0
  - scikit-learn=1.0.2=py310h4ed8f06_0
  - scipy=1.7.3=py310h6d2d95c_0
  - seaborn=0.11.2=pyhd3eb1b0_0
  - semantic_version=2.8.5=pyhd3eb1b0_0
  - setuptools=61.2.0=py310haa95532_0
  - sip=4.19.13=py310hd77b12b_0
  - six=1.16.0=pyhd3eb1b0_1
  - snappy=1.1.9=h6c2663c_0
  - sortedcontainers=2.4.0=pyhd3eb1b0_0
  - soupsieve=2.3.1=pyhd3eb1b0_0
  - sqlite=3.38.3=h2bbff1b_0
  - stack_data=0.2.0=pyhd3eb1b0_0
  - statsmodels=0.13.2=py310h2bbff1b_0
  - threadpoolctl=2.2.0=pyh0d69192_0
  - tifffile=2021.7.2=pyhd3eb1b0_2
  - tk=8.6.11=h2bbff1b_1
  - toml=0.10.2=pyhd3eb1b0_0
  - tomli=1.2.2=pyhd3eb1b0_0
  - toolz=0.11.2=pyhd3eb1b0_0
  - tornado=6.1=py310h2bbff1b_0
  - tqdm=4.64.0=py310haa95532_0
  - traitlets=5.1.1=pyhd3eb1b0_0
  - typed-ast=1.4.3=py310h2bbff1b_1
  - typing_extensions=4.1.1=pyh06a4308_0
  - tzdata=2022a=hda174b7_0
  - urllib3=1.26.9=py310haa95532_0
  - vc=14.2=h21ff451_1
  - vs2015_runtime=14.27.29016=h5e58377_2
  - wcwidth=0.2.5=pyhd3eb1b0_0
  - webencodings=0.5.1=py310haa95532_1
  - wheel=0.37.1=pyhd3eb1b0_0
  - win_inet_pton=1.1.0=py310haa95532_0
  - wincertstore=0.2=py310haa95532_2
  - xz=5.2.5=h8cc25b3_1
  - yaml=0.2.5=he774522_0
  - zfp=0.5.5=hd77b12b_6
  - zipp=3.8.0=py310haa95532_0
  - zlib=1.2.12=h8cc25b3_2
  - zstd=1.4.9=h19a0ad4_0
  - pip:
    - casjobs==0.0.1
    - cython==0.29.28
    - fastkde==1.0.19
    - mastcasjobs==0.0.4
    - multitasking==0.0.10
    - yfinance==0.1.67
prefix: C:\Users\trenton\anaconda3\envs\stsci

Best Regards,

Trenton

incompatibility with casjobs (python) master branch

I'm walking through your readme to test a simple query to CasJobs via the python CasJobs (by dfm).
When doing that, I get this error:
TypeError: __init__() got an unexpected keyword argument 'request_type'

It seems that the python CasJobs library defines a class but does not take "request_type" as an argument.

500 errors again

Rick,
I'm experiencing the same issue MikeeRead was having a month ago:

Exception: ExecuteQuickJob failed with status: 500
System.Data.SqlClient.SqlException: Could not find prepared statement with handle -1.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

Nothing is showing up in my history.

Cheers,
Michael Stroh

Quick jobs no longer working

Hi

I'd been using mastcastjobs to successfully run quick queries for several days eg

jobs = mastcasjobs.MastCasJobs(username=un, password=pwd, context="PanSTARRS_DR2")
checkResults = jobs.quick(checkQuery, task_name="check")
jobs.drop_table_if_exists(myDBTable)

since yesterday I've been getting errors. ExecuteQuickJob failed with status: 500

I'm also getting these errors going back to previous successful queries. I can run the same queries interactively via the quick web form.

I've contacted archive support but am wondering if anything has changed with the API's interface. Our mastcasjobs was installed 9 days ago and seems to point to the https URL

base_url="https://mastweb.stsci.edu/ps1casjobs/services/jobs.asmx",

the jobs submitted via mastcasjobs do not seem to reach my "history". Before they were (even the ones that failed I think).

Anyone else see the same issue?

Thanks
Mike

Error when using list_tables()

I have a script that uses the list_tables method.
When executing I get

File "[...]/python3.9/site-packages/mastcasjobs.py", line 118, in list_tables
    return [l[1:-1]for l in res.split('\n')[1:-1]]
AttributeError: 'Table' object has no attribute 'split'

list_tables() assumes the output of quickto be a string but it was changed to be a Table by default. That is what's causing the problem I presume.

getting 500 status errors again

Hi Rick

My mastcasjobs quick queue script has been running happily for about a week.

Yesterday evening it started giving errors, as did my long queue casjobs scripts,

More info since you added the error retrieval.

the mastjobs quick is giving

ExecuteQuickJob failed with status: 500
System.Data.SqlClient.SqlException: Could not find prepared statement with handle -1.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

and similar for the long queue casjobs

SubmitJob failed with status: 500
System.Data.SqlClient.SqlException: Could not find prepared statement with handle -1.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

As before I'm not seeing these queries reach my history via the web interface.

I've emailed archive support but wondering

a) if mastcasjobs queries work for you
b) any ideas

Thanks!

Mike

Code breaks at super() clause in __init__ function

When running a simple code using mastcasjobs.py, the code breaks with the following error. I have not yet been able to get a fix for this.

~/anaconda3/lib/python3.7/site-packages/mastcasjobs.py in init(self, username, password, userid, request_type, context, base_url, wsid_url, fast_url)
97
98 super(MastCasJobs,self).init(userid=userid, password=password,
---> 99 base_url=base_url, request_type=request_type, context=context)
100
101 def list_tables(self, context="MYDB"):

TypeError: init() got an unexpected keyword argument 'request_type'

Exception: GetJobStatus failed with status: 500

I am trying to script the running of a number of casjob queries of the ps1 database.

I am consistently running into the following Exception:

Exception: GetJobStatus failed with status: 500
System.InvalidOperationException: There is already an open DataReader associated with this Command which must be closed first.
at System.Data.SqlClient.SqlInternalConnectionTds.ValidateConnectionForExecute(SqlCommand command)

The line of code where this exception is thrown is:
status = mast.monitor( jobid )

This hasn't yet happened on the first job, but doesn't always happen on, say, the second one. It is always true that subsequent jobs have different task_names. It takes some significant time between when the job is submitted and the exception gets thrown; i don't know precisely, but more than 1 hour. It's not like the exception gets thrown when the job starts; the jobs are still sitting in the queue.

Here is the essential part of my script, if only to show i'm not doing anything particularly clever or fancy to get myself into trouble:
mast = MastCasJobs( username, password,
context='PanSTARRS_DR2' )
# construct query by filling in essential values to boilerplate _query
query = _query.format(
projectionID=projectionID, task_name=task_name,
dec_min=dec_min, dec_max=dec_max)

    mast.drop_table_if_exists( task_name )

    jobid = mast.submit( query, task_name=task_name, estimate=10 )

    status = mast.monitor( jobid )
    # failure here, but only after some (long) time

I had used these scripts and code quite successfully about a year ago. I don't think that i have changed anything in my script that might be producing this apparent clash of commands. It's possible something changed on the ps1 server side, maybe? But i don't see anyway to influence the situation within mastcasjobs.

I'm not sure whether there is a need for some extra settings or error handling within mastcasjobs, but ... well, very grateful for whatever advice or guidance you can offer!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.