bmrb-io / pynmrstar Goto Github PK

Hi John

I thought the attached project maybe of interest. It shows how easy (ie how little code) it is to read a NEF / NMRStar file with PyNMRStar which I cooked up for a prospective user. Also it has a few bits of code in utilities which maybe of interest to the wider community [iterate row as namespace, iterate row as dictionaries and convert Loop into a pandas data frame]

regards
simpleNEFReader.tgz

Gary

The latest release shown on the main github page is 2.6.x, shouldn't it be 3.0.4?

On a related not, is the software included in any package manager such as conda or pip?

Thanks!

Use the API to get the full list of macromolecule or metabolomics entries, and then provide a generator that yields all the entries in the DB in sequence.

Too late for 3.0.3 -- but subsequent releases

Adding something like
python_requires='>=3'

and for classifiers,
Programming Language :: Python :: 3 :: Only

I made a very simple script based on the README file:

#!/usr/bin/env python3

import pynmrstar

entry = pynmrstar.Entry.from_file("1l2y_mr.str")
entry.print_tree()

where 1l2y_mr.str was freshly downloaded from https://www.rcsb.org/structure/1l2y
This gives me the following output:

% ./nmrstar_test.py
Traceback (most recent call last):
File "./nmrstar_test.py", line 5, in
entry = pynmrstar.Entry.from_file("1l2y_mr.str")
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 1755, in from_file
return cls(file_name=the_file)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 1558, in init
parser.parse(star_buffer.read(), source=self.source)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 743, in parse
curframe.add_loop(curloop)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 2554, in add_loop
loop_to_add.category)
ValueError: You cannot have two loops with the same category in one saveframe. Category: '_Constraint_file'.

Is the input file broken or the script? The file 2lot_mr.str gives the same problem.

It would be helpful if pynmrstar used its own logger so when silencing warnings such as

WARNING:root:Loop with no data on line: xx

warnings could be silenced for pynmrstar and not the libraries

It would be great to have some simple examples, e.g. reading in the data, looping over conditions etc.

An NMR-STAR file with invalid pointers caused the deposition system to not function properly. Therefore it is a class of issue that should be checked for here as well.

it doesn't appear to be possible to get the tags for a loop without the category?

if I am right it would be nice to have

class Loop:
    ...
    def get_tag_names(no_category: bool =False) -> List[str]:
       ...

?

pynmrstar/parser.py:73

raise ParsingError("Invalid file. NMR-STAR files must start with 'data_' followed by the data name. "
f"Did you accidentally select the wrong file? Your file started with '{self.token}'.",
self.line_number)

would be better to be

raise ParsingError("Invalid file. STAR files must start with 'data_' followed by the data name. "
f"Did you accidentally select the wrong file? Your file started with '{self.token}'.",
self.line_number)

since PyNMRSTAR can be used as a general statement file parser?

Don't allow the exception in quote_value to bubble up all the way.

This will make the validation more strict.

consider

from pynmrstar import Entry, Saveframe, Loop

def test_frame_no_loop():
    entry = Entry.from_scratch('ni')
    frame = Saveframe.from_scratch('spam', 'swallow')
    entry.add_saveframe(frame)
    frame.add_tag('parrot', 'swallow')
    loop = Loop.from_scratch('bedlvere')
    frame.add_loop(loop)
    loop.add_tag(['galahad','excalibur'])
    loop.add_data(['guinivere', 'excalibur'])

    print(entry)

if __name__ == '__main__':
    test_frame_no_loop()

this prints

data_ni

save_spam
   _swallow.parrot  swallow

   loop_
      _bedlvere.galahad
      _bedlvere.excalibur

     guinivere   excalibur    

   stop_

save_

but

from pynmrstar import Entry, Saveframe, Loop

def test_frame_no_loop():
    entry = Entry.from_scratch('ni')
    frame = Saveframe.from_scratch('spam', 'swallow')
    entry.add_saveframe(frame)
    # frame.add_tag('parrot', 'swallow')
    loop = Loop.from_scratch('bedlvere')
    frame.add_loop(loop)
    loop.add_tag(['galahad','excalibur'])
    loop.add_data(['guinivere', 'excalibur'])

    print(entry)

if __name__ == '__main__':
    test_frame_no_loop()

prints the text below missing the loop bedlevre

data_ni


save_spam

save_

Currently the indenting in star files produced by pynmrstar uses 3 spaces per level, but many editors are setup to use 2 or more typically 4 spaces per ident level. Is it possible to control the indentation level in an output star file...

There is a problem when I read a text block containing ; in it. There was no problem while writing the text block using the parser, but while reading it breaks.
Example This loop can't be

parsed but can be written using PyNMRSTAR without any problem

`
loop_
_Software_specific_info.Software_saveframe_ID
_Software_specific_info.Software_saveframe
_Software_specific_info.Software_specific_info_list_ID

 1   

;
save_cyana_additional_data_1
_cyana_additional_data.sf_category cyana_additional_data
_cyana_additional_data.sf_framecode cyana_additional_data_1
cyana_additional_data.special_version 5
save
;
1
2
;
save_xplor_raw_data_T1_T2_values_1
_xplor_raw_data.sf_category xplor_raw_data
_xplor_raw_data.sf_framecode xplor_raw_data_T1_T2_values_1
_xplor_raw_data.file_name input_file_name.extension
xplor_raw_data.details
;
Optional item
For comments.
Any multiline text can be put here
;
xplor_raw_data.text
;
If the string starts with a new line (as in this case) it is skipped. This avoids issues with how many initial spaces to strip
;
save
;
1
stop
`

Filter method is not documented in README.md

If i don't add a category in the API of a frame the API complains loudly, but importing a star file without categories in the Saveframe doesn't raise warnings. Is this correct?

Is it possible to silence warning of loops with no data ? These are quite legal star constructs (? i believe?)

At first, thanks to create great project!

I notice that PyNMRSTAR prompts unclear error message against the following invalid STAR file:

data_nef_D_800107
 
save_nef_nmr_meta_data
 _Entry.Sf_category                    entry_information
 _Entry.Sf_framecode                   nef_nmr_meta_data
 _Entry.NMR_STAR_version               3.2.1.18
 _Entry.Source_data_format             nmr_exchange_format
 _Entry.Source_data_format_version     1.1
 _Entry.Generated_software_name        PDBStat
 _Entry.Generated_software_version     5.20-exp
 _Entry.Generated_date                 2020-01-28T16:40:10
 _Entry.UUID                           1580226010
 _Entry.Related_coordinate_file_name   .
 _Entry.ID                             2PNG
# 
 _pdbx_nmr_assigned_chem_shift_list.entry_id         ?
 _pdbx_nmr_assigned_chem_shift_list.id               ?
 _pdbx_nmr_assigned_chem_shift_list.data_file_name   2png.nef
#
save_

The error message for ValueError is:

"Invalid token found in saveframe 'internaluseyoushouldntseethis_frame': 'data_nef_D_800107'", 1

But, both sameframe name and line number do not point out the real problem. I hope you improve the message in such cases.

Best,

Hi Jon,

I noticed a typo in error message, saveframe.py, line 579:
raise ValueError('The Sf_framecode tag cannot be different from the saveframe name. Error '
f'occurred in tag {self.tag_prefix}.Sf_framecode with value {value} which '
f'conflicts with with the saveframe name {self._name}.')

Could you please fix this?

pynmrstar is broken on centos 7.x and possibly 8.x and anything that uses gcc as opposed to clang or the ms compilers to compile cnmrstar because the c extension has the following construct

For (int I=0;

at line 149?

This is not legal in less than c99 apparently (pip by default uses an older c and requires extra command line flags to make this work)

Also annoyingly the bug doesn’t fully appear in the pip output unless you install with pip —verbose, some how a failure to compile is not getting to the pip machinery…

Attached is a jpeg of the install process failing for reference

test_nmr-star_3.1.txt

Hi Jon,

I noticed that data loss occurred while outputting NMR-STAR file if proper key tag, such as 'ID', was missing in a loop. I am using v2.6.5.1 and v3. The problem relates category definitions for spectral peak lists, '_Peak_general_char', '_Peak_char', and '_Assigned_peak_chem_shift'. Could you run read/write test with the attached file including these categories, or provide solution to turn-off clean-up feature while output?

Best regards,

Masashi

I have this function which makes working with rows in loops much easier, would its be possible to add it as a method on loop, or am I missing the right idiom for dealing with loops?

def _loop_row_dict_iter(loop: Loop):
    for row in loop:
        yield {tag.lower(): value for tag, value in zip(loop.tags,row)}

to be clear I was doing things like

loop = frame.loops[0]
tag_index_x = loop. tag_index('x')
for row in loop:
   x = row[tag_index_x]

which seems clunky

@colinwilburn - Thanks for the idea.

running tests on newly downloaded git repo gives an error

python3 pynmrstar/unit_tests                                                                                                                   1 ✘ ╱ base  ╱ at 22:29:50  
Python version: 3.8.13 (default, Mar 28 2022, 06:16:26) 
[Clang 12.0.0 ]
System platform: darwin
.......................E.....
======================================================================
ERROR: test_odd_strings (pynmrstar.unit_tests.TestPyNMRSTAR)
Make sure the library can handle odd strings.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/xxx/PyNMRSTAR/pynmrstar/unit_tests/__init__.py", line 57, in test_odd_strings
    with open(os.path.join(our_path, 'naughty-strings/blns.json')) as odd_string_file:
FileNotFoundError: [Errno 2] No such file or directory: 
'/Uxxx/PyNMRSTAR/pynmrstar/unit_tests/naughty-strings/blns.json'

----------------------------------------------------------------------
Ran 29 tests in 3.974s

FAILED (errors=1)

warnings such as the one at 556 in loop.py provide no information as to where the user function that called the deprecated function is located

I found a bug, while I am trying add data to loop. Here is how you can reproduce the bug

`import bmrb
ll=bmrb.Loop.from_scratch()
ll.add_column('_Atom_chem_shift.Auth_entity_assembly_ID')
ll.add_column('_Atom_chem_shift.Auth_seq_ID')
ll.add_column('_Atom_chem_shift.Auth_Comp_ID')
ll.add_column('_Atom_chem_shift.Auth_atom_ID')
ll.add_column('_Atom_chem_shift.Val')
ll.add_column('_Atom_chem_shift.Val_err')
ll.add_column('Atom_chem_shift.Ambiguity_code')
dat1=['A','2','LYS','HB1','1.779323999994285','0.009758767141997112','2']
ll.add_data(dat1)
print ll
loop
_Atom_chem_shift.Auth_entity_assembly_ID
_Atom_chem_shift.Auth_seq_ID
_Atom_chem_shift.Auth_Comp_ID
_Atom_chem_shift.Auth_atom_ID
_Atom_chem_shift.Val
_Atom_chem_shift.Val_err
_Atom_chem_shift.Ambiguity_code

 A   2   LYS   HB1   1.779323999994285   0.009758767141997112   2    

stop_

dat1[3]="HB2"
ll.add_data(dat1)
print ll
loop_
_Atom_chem_shift.Auth_entity_assembly_ID
_Atom_chem_shift.Auth_seq_ID
_Atom_chem_shift.Auth_Comp_ID
_Atom_chem_shift.Auth_atom_ID
_Atom_chem_shift.Val
_Atom_chem_shift.Val_err
_Atom_chem_shift.Ambiguity_code

 A   2   LYS   HB2   1.779323999994285   0.009758767141997112   2    
 A   2   LYS   HB2   1.779323999994285   0.009758767141997112   2    

stop_
`

note HB1 in the first row become HB2, if I add HB3 then all 3 become HB3. Am I doing something wrong? or is it a bug?

title says it all

in def category(self, category): there appears to be an error

# Update the sf_category tag too
lc_tags = self._lc_tags
if 'sf_category' in lc_tags:
    tag_index = lc_tags['sf_category']
    self.tags[tag_index] = ['sf_category',category]

when run with

   save_nef_nmr_spectrum_k_ubi_hnco`1`

      _nef_nmr_spectrum.sf_category                   nef_nmr_spectrum
      _nef_nmr_spectrum.sf_framecode                  nef_nmr_spectrum_k_ubi_hnco`1`

when we print the frame I get

save_nef_test_k_ubi_hnco`1`
   _nef_nmr_spectrum.n                                     e
   _nef_nmr_spectrum.sf_framecode                 nef_test_k_ubi_hnco`1`

the correct code should be

# Update the sf_category tag too
lc_tags = self._lc_tags
if 'sf_category' in lc_tags:
    tag_index = lc_tags['sf_category']
    self.tags[tag_index] = ['sf_category',category]

It is considered bad form for a library to use logging.basicConfig. It establishes the logging format for the application if the import happens before logging is defined in the code with the main function.

Consider the following in the main application - with a typical code style - imports before functional code.

import logging
import pynmrstar

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s]-%(module)s.%(funcName)s: %(message)s")

As pynmrstar has done this already in the import - it overrides what is being specified.

In addition, a library should not set the root logging level. This should again be determined by the application.

when I create a frame with

frame = Saveframe.from_scratch(frame_code, category)

and use something like frame_code = nef_chemical_shift_list_nmrpipe when I do

     frame.add_tag("sf_category", category)

I get

save_nef_chemical_shift_list_nmrpipe
   _nef_chemical_shift_list.sf_category   nef_chemical_shift_list
   _nef_chemical_shift_list.sf_framecode  nef_chemical_shift_list_nmrpipe

   loop_
      _nef_chemical_shift_list.chain_code
    ...

whereas I expected

save_nef_chemical_shift_list_nmrpipe
   _nef_chemical_shift_list_nmrpipe.sf_category   nef_chemical_shift_list
   _nef_chemical_shift_list_nmrpipe.sf_framecode  nef_chemical_shift_list_nmrpipe

   loop_
      _nef_chemical_shift_list.chain_code
    ...

is this correct, is nef treating star file frames in an odd way

Add an internal loop method for checking if a tag exists in a loop to use to replace the 10 instances of [x.lower() for x in self.columns].