bmrb-io / pynmrstar Goto Github PK
View Code? Open in Web Editor NEWA Python module for reading, writing, and manipulating NMR-STAR files.
License: MIT License
A Python module for reading, writing, and manipulating NMR-STAR files.
License: MIT License
Hi John
I thought the attached project maybe of interest. It shows how easy (ie how little code) it is to read a NEF / NMRStar file with PyNMRStar which I cooked up for a prospective user. Also it has a few bits of code in utilities which maybe of interest to the wider community [iterate row as namespace, iterate row as dictionaries and convert Loop into a pandas data frame]
regards
simpleNEFReader.tgz
Gary
The latest release shown on the main github page is 2.6.x, shouldn't it be 3.0.4?
On a related not, is the software included in any package manager such as conda or pip?
Thanks!
Use the API to get the full list of macromolecule or metabolomics entries, and then provide a generator that yields all the entries in the DB in sequence.
Too late for 3.0.3 -- but subsequent releases
Adding something like
python_requires='>=3'
and for classifiers,
Programming Language :: Python :: 3 :: Only
I made a very simple script based on the README file:
#!/usr/bin/env python3
import pynmrstar
entry = pynmrstar.Entry.from_file("1l2y_mr.str")
entry.print_tree()
where 1l2y_mr.str was freshly downloaded from https://www.rcsb.org/structure/1l2y
This gives me the following output:
% ./nmrstar_test.py
Traceback (most recent call last):
File "./nmrstar_test.py", line 5, in
entry = pynmrstar.Entry.from_file("1l2y_mr.str")
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 1755, in from_file
return cls(file_name=the_file)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 1558, in init
parser.parse(star_buffer.read(), source=self.source)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 743, in parse
curframe.add_loop(curloop)
File "/home/spoel/wd/PyNMRSTAR/pynmrstar.py", line 2554, in add_loop
loop_to_add.category)
ValueError: You cannot have two loops with the same category in one saveframe. Category: '_Constraint_file'.
Is the input file broken or the script? The file 2lot_mr.str gives the same problem.
It would be helpful if pynmrstar used its own logger so when silencing warnings such as
WARNING:root:Loop with no data on line: xx
warnings could be silenced for pynmrstar and not the libraries
It would be great to have some simple examples, e.g. reading in the data, looping over conditions etc.
An NMR-STAR file with invalid pointers caused the deposition system to not function properly. Therefore it is a class of issue that should be checked for here as well.
it doesn't appear to be possible to get the tags for a loop without the category?
if I am right it would be nice to have
class Loop:
...
def get_tag_names(no_category: bool =False) -> List[str]:
...
?
pynmrstar/parser.py:73
raise ParsingError("Invalid file. NMR-STAR files must start with 'data_' followed by the data name. "
f"Did you accidentally select the wrong file? Your file started with '{self.token}'.",
self.line_number)
would be better to be
raise ParsingError("Invalid file. STAR files must start with 'data_' followed by the data name. "
f"Did you accidentally select the wrong file? Your file started with '{self.token}'.",
self.line_number)
since PyNMRSTAR can be used as a general statement file parser?
Don't allow the exception in quote_value
to bubble up all the way.
This will make the validation more strict.
consider
from pynmrstar import Entry, Saveframe, Loop
def test_frame_no_loop():
entry = Entry.from_scratch('ni')
frame = Saveframe.from_scratch('spam', 'swallow')
entry.add_saveframe(frame)
frame.add_tag('parrot', 'swallow')
loop = Loop.from_scratch('bedlvere')
frame.add_loop(loop)
loop.add_tag(['galahad','excalibur'])
loop.add_data(['guinivere', 'excalibur'])
print(entry)
if __name__ == '__main__':
test_frame_no_loop()
this prints
data_ni
save_spam
_swallow.parrot swallow
loop_
_bedlvere.galahad
_bedlvere.excalibur
guinivere excalibur
stop_
save_
but
from pynmrstar import Entry, Saveframe, Loop
def test_frame_no_loop():
entry = Entry.from_scratch('ni')
frame = Saveframe.from_scratch('spam', 'swallow')
entry.add_saveframe(frame)
# frame.add_tag('parrot', 'swallow')
loop = Loop.from_scratch('bedlvere')
frame.add_loop(loop)
loop.add_tag(['galahad','excalibur'])
loop.add_data(['guinivere', 'excalibur'])
print(entry)
if __name__ == '__main__':
test_frame_no_loop()
prints the text below missing the loop bedlevre
data_ni
save_spam
save_
Currently the indenting in star files produced by pynmrstar uses 3 spaces per level, but many editors are setup to use 2 or more typically 4 spaces per ident level. Is it possible to control the indentation level in an output star file...
There is a problem when I read a text block containing ; in it. There was no problem while writing the text block using the parser, but while reading it breaks.
Example This loop can't be
parsed but can be written using PyNMRSTAR without any problem
`
loop_
_Software_specific_info.Software_saveframe_ID
_Software_specific_info.Software_saveframe
_Software_specific_info.Software_specific_info_list_ID
1
;
save_cyana_additional_data_1
_cyana_additional_data.sf_category cyana_additional_data
_cyana_additional_data.sf_framecode cyana_additional_data_1
cyana_additional_data.special_version 5
save
;
1
2
;
save_xplor_raw_data_T1_T2_values_1
_xplor_raw_data.sf_category xplor_raw_data
_xplor_raw_data.sf_framecode xplor_raw_data_T1_T2_values_1
_xplor_raw_data.file_name input_file_name.extension
xplor_raw_data.details
;
Optional item
For comments.
Any multiline text can be put here
;
xplor_raw_data.text
;
If the string starts with a new line (as in this case) it is skipped. This avoids issues with how many initial spaces to strip
;
save
;
1
stop
`
Filter method is not documented in README.md
If i don't add a category in the API of a frame the API complains loudly, but importing a star file without categories in the Saveframe doesn't raise warnings. Is this correct?
Is it possible to silence warning of loops with no data ? These are quite legal star constructs (? i believe?)
At first, thanks to create great project!
I notice that PyNMRSTAR prompts unclear error message against the following invalid STAR file:
data_nef_D_800107
save_nef_nmr_meta_data
_Entry.Sf_category entry_information
_Entry.Sf_framecode nef_nmr_meta_data
_Entry.NMR_STAR_version 3.2.1.18
_Entry.Source_data_format nmr_exchange_format
_Entry.Source_data_format_version 1.1
_Entry.Generated_software_name PDBStat
_Entry.Generated_software_version 5.20-exp
_Entry.Generated_date 2020-01-28T16:40:10
_Entry.UUID 1580226010
_Entry.Related_coordinate_file_name .
_Entry.ID 2PNG
#
_pdbx_nmr_assigned_chem_shift_list.entry_id ?
_pdbx_nmr_assigned_chem_shift_list.id ?
_pdbx_nmr_assigned_chem_shift_list.data_file_name 2png.nef
#
save_
The error message for ValueError is:
"Invalid token found in saveframe 'internaluseyoushouldntseethis_frame': 'data_nef_D_800107'", 1
But, both sameframe name and line number do not point out the real problem. I hope you improve the message in such cases.
Best,
Hi Jon,
I noticed a typo in error message, saveframe.py, line 579:
raise ValueError('The Sf_framecode tag cannot be different from the saveframe name. Error '
f'occurred in tag {self.tag_prefix}.Sf_framecode with value {value} which '
f'conflicts with with the saveframe name {self._name}.')
Could you please fix this?
pynmrstar is broken on centos 7.x and possibly 8.x and anything that uses gcc as opposed to clang or the ms compilers to compile cnmrstar because the c extension has the following construct
For (int I=0;
at line 149?
This is not legal in less than c99 apparently (pip by default uses an older c and requires extra command line flags to make this work)
Also annoyingly the bug doesn’t fully appear in the pip output unless you install with pip —verbose, some how a failure to compile is not getting to the pip machinery…
Attached is a jpeg of the install process failing for reference
Hi Jon,
I noticed that data loss occurred while outputting NMR-STAR file if proper key tag, such as 'ID', was missing in a loop. I am using v2.6.5.1 and v3. The problem relates category definitions for spectral peak lists, '_Peak_general_char', '_Peak_char', and '_Assigned_peak_chem_shift'. Could you run read/write test with the attached file including these categories, or provide solution to turn-off clean-up feature while output?
Best regards,
Masashi
I have this function which makes working with rows in loops much easier, would its be possible to add it as a method on loop, or am I missing the right idiom for dealing with loops?
def _loop_row_dict_iter(loop: Loop):
for row in loop:
yield {tag.lower(): value for tag, value in zip(loop.tags,row)}
to be clear I was doing things like
loop = frame.loops[0]
tag_index_x = loop. tag_index('x')
for row in loop:
x = row[tag_index_x]
which seems clunky
@colinwilburn - Thanks for the idea.
running tests on newly downloaded git repo gives an error
python3 pynmrstar/unit_tests 1 ✘ ╱ base ╱ at 22:29:50
Python version: 3.8.13 (default, Mar 28 2022, 06:16:26)
[Clang 12.0.0 ]
System platform: darwin
.......................E.....
======================================================================
ERROR: test_odd_strings (pynmrstar.unit_tests.TestPyNMRSTAR)
Make sure the library can handle odd strings.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/xxx/PyNMRSTAR/pynmrstar/unit_tests/__init__.py", line 57, in test_odd_strings
with open(os.path.join(our_path, 'naughty-strings/blns.json')) as odd_string_file:
FileNotFoundError: [Errno 2] No such file or directory:
'/Uxxx/PyNMRSTAR/pynmrstar/unit_tests/naughty-strings/blns.json'
----------------------------------------------------------------------
Ran 29 tests in 3.974s
FAILED (errors=1)
warnings such as the one at 556 in loop.py provide no information as to where the user function that called the deprecated function is located
I found a bug, while I am trying add data to loop. Here is how you can reproduce the bug
`import bmrb
ll=bmrb.Loop.from_scratch()
ll.add_column('_Atom_chem_shift.Auth_entity_assembly_ID')
ll.add_column('_Atom_chem_shift.Auth_seq_ID')
ll.add_column('_Atom_chem_shift.Auth_Comp_ID')
ll.add_column('_Atom_chem_shift.Auth_atom_ID')
ll.add_column('_Atom_chem_shift.Val')
ll.add_column('_Atom_chem_shift.Val_err')
ll.add_column('Atom_chem_shift.Ambiguity_code')
dat1=['A','2','LYS','HB1','1.779323999994285','0.009758767141997112','2']
ll.add_data(dat1)
print ll
loop
_Atom_chem_shift.Auth_entity_assembly_ID
_Atom_chem_shift.Auth_seq_ID
_Atom_chem_shift.Auth_Comp_ID
_Atom_chem_shift.Auth_atom_ID
_Atom_chem_shift.Val
_Atom_chem_shift.Val_err
_Atom_chem_shift.Ambiguity_code
A 2 LYS HB1 1.779323999994285 0.009758767141997112 2
stop_
dat1[3]="HB2"
ll.add_data(dat1)
print ll
loop_
_Atom_chem_shift.Auth_entity_assembly_ID
_Atom_chem_shift.Auth_seq_ID
_Atom_chem_shift.Auth_Comp_ID
_Atom_chem_shift.Auth_atom_ID
_Atom_chem_shift.Val
_Atom_chem_shift.Val_err
_Atom_chem_shift.Ambiguity_code
A 2 LYS HB2 1.779323999994285 0.009758767141997112 2
A 2 LYS HB2 1.779323999994285 0.009758767141997112 2
stop_
`
note HB1 in the first row become HB2, if I add HB3 then all 3 become HB3. Am I doing something wrong? or is it a bug?
title says it all
in def category(self, category):
there appears to be an error
# Update the sf_category tag too
lc_tags = self._lc_tags
if 'sf_category' in lc_tags:
tag_index = lc_tags['sf_category']
self.tags[tag_index] = ['sf_category',category]
when run with
save_nef_nmr_spectrum_k_ubi_hnco`1`
_nef_nmr_spectrum.sf_category nef_nmr_spectrum
_nef_nmr_spectrum.sf_framecode nef_nmr_spectrum_k_ubi_hnco`1`
when we print the frame I get
save_nef_test_k_ubi_hnco`1`
_nef_nmr_spectrum.n e
_nef_nmr_spectrum.sf_framecode nef_test_k_ubi_hnco`1`
the correct code should be
# Update the sf_category tag too
lc_tags = self._lc_tags
if 'sf_category' in lc_tags:
tag_index = lc_tags['sf_category']
self.tags[tag_index] = ['sf_category',category]
It is considered bad form for a library to use logging.basicConfig. It establishes the logging format for the application if the import happens before logging is defined in the code with the main function.
Consider the following in the main application - with a typical code style - imports before functional code.
import logging
import pynmrstar
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s]-%(module)s.%(funcName)s: %(message)s")
As pynmrstar has done this already in the import - it overrides what is being specified.
In addition, a library should not set the root logging level. This should again be determined by the application.
when I create a frame with
frame = Saveframe.from_scratch(frame_code, category)
and use something like frame_code = nef_chemical_shift_list_nmrpipe
when I do
frame.add_tag("sf_category", category)
I get
save_nef_chemical_shift_list_nmrpipe
_nef_chemical_shift_list.sf_category nef_chemical_shift_list
_nef_chemical_shift_list.sf_framecode nef_chemical_shift_list_nmrpipe
loop_
_nef_chemical_shift_list.chain_code
...
whereas I expected
save_nef_chemical_shift_list_nmrpipe
_nef_chemical_shift_list_nmrpipe.sf_category nef_chemical_shift_list
_nef_chemical_shift_list_nmrpipe.sf_framecode nef_chemical_shift_list_nmrpipe
loop_
_nef_chemical_shift_list.chain_code
...
is this correct, is nef treating star file frames in an odd way
Make sure the library can handle odd strings.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/xxxx/PyNMRSTAR/pynmrstar/unit_tests/__init__.py", line 57, in test_odd_strings
with open(os.path.join(our_path, 'naughty-strings/blns.json')) as odd_string_file:
FileNotFoundError: [Errno 2] No such file or directory: '/xxx/PyNMRSTAR/pynmrstar/unit_tests/naughty-strings/blns.json'
Add an internal loop method for checking if a tag exists in a loop to use to replace the 10 instances of [x.lower() for x in self.columns].
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.