Comments (4)
Hi Wenjie,
you should be able to do this with the already existing process_two_files_without_parallel_output()
method.
No need to add anything else I think.
- modify the process_function, to make it incorporate all the procedures. I think if so, it is possible to implement. But disadvantage is this will make the process_function so big and not very user-friendly?
I guess my suggestions also applies to that fear of your's but just split it up in a couple of functions and you should be more than fine.
Let me know if there is some roadblock in that approach and we can certainly add something else as well if required.
Cheers!
Lion
from pyasdf.
Hi Lion,
I ensembled an example for the preprocessing workflow based on your suggestion, using process_two_files_without_parallel_output()
method.
This example includes:
- observed signal processing
- synthetic singal processing
- window selection
- adjoint source constructor
I uploaded the file here(this is just an example and it is not yet 100% complete). But it delievers what we have been discussed.
A bit concern about this way is we have two many in-place function definition. The major reason for that is we need to define some variables outside the function and the argument list is pretty limited for process function passing into the process_two_files_without_parallel_output()
.
Sorry, I named it proc_combo.py.txt for uploading. Please rename it to *.py.
proc_combo.py.txt
from pyasdf.
Hi Wenjie,
yea that is actually an intended design choice. Otherwise the process()/process_two_files_without_parallel_output()
methods would have very awkward function signatures.
Python has reasonable support for functional programming, in this case closures and function currying. You are using closures: the function definitions you make bind the outside variables and thus the function can see them.
An alternative would be to use functools.partial
for function currying/partial application in Python. Your example could look like this:
from functools import partial
def combo_func(obsd_station_group, synt_station_group,
process_obsd_function, process_synt_function, window_function,
adjoint_source):
...
results = obsd_ds.process_two_files_without_parallel_output(
synt_ds, partial(combo_func,
process_obsd_function=process_obsd_function,
process_synt_function=process_synt_function,
window_function=window_function
adjoint_source=adjoint_source))
Using either allows to write a well structure program.
from pyasdf.
Thanks for the suggestion. I will use the functools.partial
👍
from pyasdf.
Related Issues (20)
- Why are tag paths validated against r"^[a-zA-Z0-9][a-zA-Z0-9_]*[a-zA-Z0-9]$"? HOT 7
- Slow performance when adding gappy data to ASDF HOT 3
- adding an existing StationXML throws TypeError HOT 1
- setup broken between 0.5.1 and 0.6.1 HOT 4
- How to make pyasdf to support multi-date-range station inventory + custom tags? HOT 6
- Calling parallel process() for a very large file gets stuck HOT 7
- Why the differences between input station XML file to ASDF and the output station XML file HOT 5
- Problem with hdf5 1.10 and _add_trace_write_collective_information during process() HOT 2
- SUGGESTION: adding public APIs for parallel writing functions HOT 1
- pkg_resources deprecated? HOT 1
- newer versions h5py > 2.1 will break pyasdf events HOT 2
- Error @opening after copying pyasdf-archive HOT 2
- Renaming a tag HOT 1
- Increasingly slow to add events to ASDF dataset HOT 3
- use dense attribute storage for large attributes
- Citation?
- Rounding start times to nearest sample rather than previous sample
- Issue in tutorial
- Whether each event can be associated with part of the waveform in ASDFDataset.h5
- Failed to flush and close HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyasdf.