Comments (6)
Hi Peter,
yes, that was pretty much the issue I noticed shortly before leaving and could not fix quickly.
It started from the fact, that actually any SER file saves data in a series, so even single images are 1-member series. Therefore I got aware of this problem rather lately. I agree to reducing single images in EMD do avoid unnecessary dimensions. Its just an extra case I need to implement in io.ser.
After changing this back and forth in RingDiff, I came to the conclusion that we have to decide on the following design questions first, as it touches things beyond RingDiff: In which order to we want to save dimensions in EMD?
I think HDF5/h5py forces us to save series in (n, y, x) order, to have the fast changing indices last. Otherwise we run into performance issues. (n, x, y) seems to be possible without loosing to much, though.
Then the question is, whether we want single images as (y, x) or (x,y). (y,x) seems to be more consistent with the series and is putting the fast changing index last. (x,y) seems to be more intuitive, even though images usually are stored in matrices the (y,x) way and the functions somehow adopt for that.
Another possibility for RingDiff would be to not only implement the two specific cases of single images and image series, but rather let the user choose which dimension in their EMD files should be interpreted as what.
What's your opinion on the order question?
from openncem.
Then the question is, whether we want single images as (y, x) or (x,y). (y,x) seems to be more consistent with the series and is putting the fast changing index last. (x,y) seems to be more intuitive, even though images usually are stored in matrices the (y,x) way and the functions somehow adopt for that.
It seems to me that the performance penalty of [y,x] vs [x,y] is negligible in most cases. If you operate on a full image (which is usually the case) then it wont matter at all. Most modern array handling modules like Numpy are able to choose the fastest index if possible anyway. Check out this article about Numpy.
I would choose [y,x] to be consistent with the naming convention (i.e. [num,y,x]). I hate to say this, but Ive been following the [x,y,num] convention since I started Python programming as a habit from Matlab. Im planning to switch most of my code to [num,y,x].
Lastly, its important that the image look on the screen the same as it was taken on the microscope to avoid confusion.
Another possibility for RingDiff would be to not only implement the two specific cases of single images and image series, but rather let the user choose which dimension in their EMD files should be interpreted as what.
I think its worthwhile to implement the choice option. (Says he who will likely not do the programming. Haha.) Im happy to pitch in at some point though! I think that EMDviewer writes things out the wrong way [x,y,num and Im not sure how quickly we can get that changed. @cophus and I should discuss that.
from openncem.
I have modified io.ser to put series in (n,y,x) and single images in (y,x).
RingDiff has been modified to work with both and I have fixed a bug in center selecting.
It is all in branch 'iss3', if anyone wants to have a look at it. I think we could merge it into the development branch.
I have postponed the choice option for later, as that sounds like rather rare use cases. Maybe we can also get something like that into io.emd, however I do not want any Qt stuff in the package.
I think we should focus on implementing the DM3 reader and get a converter to get EMDs with dimensions in the above order. Right now it also works properly with the DM3s from emdviewer, only x and y are flipped for viewing.
from openncem.
I couldnt test this on the iss3 branch, because of the time tag issue (#4). However, it seems that iss4 branch includes this [n,y,x] array ordering. It worked on the time series data.
from openncem.
I have postponed the choice option for later, as that sounds like rather rare use cases. Maybe we can also get something like that into io.emd, however I do not want any Qt stuff in the package.
I think we should focus on implementing the DM3 reader and get a converter to get EMDs with dimensions in the above order. Right now it also works properly with the DM3s from emdviewer, only x and y are flipped for viewing.
I forgot to reply to this. I agree about postponing the choice option. I have started writing the dm3Reader.py. Ill try to make it match the ser.py so we can keep the style of the packages the same. Ill probably need your help @fniekiel to make that happen. Ill add it to the repo once I have it working for at least 1 file.
Ill also try to pull the new version of the RingDiffraction program to the Titan for Karen once we merge the fixed to ser.py ( #3 and #4) .
from openncem.
I have merged the fixes to #4 and will close this issue. We should open a new one for the dm3Reader.
from openncem.
Related Issues (20)
- SER array size backwards HOT 3
- ser tab complete infinite loop HOT 1
- Submodule name io clobbers built in python3 module io HOT 1
- 4D-STEM data reads incorrectly HOT 1
- Could not build ncempy. Cannot load backend 'Qt5Agg' which requires the 'qt5' HOT 34
- Change genfire import line HOT 1
- dm data not accessible after file closed using on_memory option HOT 2
- RingDiff tool and PyQT HOT 1
- module 'ncempy' has no attribute 'io' HOT 4
- EMD `memmap`s can lose their file handles prematurely HOT 1
- DM import can't read newest-version files HOT 1
- DM reader fails in `__del__` on current Python 3.9 HOT 1
- need to handle bad TagOffSetArray HOT 8
- `BufferError` caused by eager closing of DM file handles HOT 3
- emd import does not support default dim attributes HOT 3
- DM3 class HOT 8
- Unit tests HOT 4
- Example code HOT 2
- Matlab Code Location HOT 1
- ser.py and import ncempy.io.emd HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openncem.