Possible to keep things in the memory? about pyasdf HOT 4 CLOSED

Hi Wenjie,

you should be able to do this with the already existing process_two_files_without_parallel_output() method.

http://seismicdata.github.io/pyasdf/parallel_processing.html#process-two-files-without-parallel-output-method

No need to add anything else I think.

2. modify the process_function, to make it incorporate all the procedures. I think if so, it is possible to implement. But disadvantage is this will make the process_function so big and not very user-friendly?

I guess my suggestions also applies to that fear of your's but just split it up in a couple of functions and you should be more than fine.

Let me know if there is some roadblock in that approach and we can certainly add something else as well if required.

Cheers!

Lion

from pyasdf.

Hi Lion,

I ensembled an example for the preprocessing workflow based on your suggestion, using process_two_files_without_parallel_output() method.

This example includes:

1. observed signal processing
2. synthetic singal processing
3. window selection
4. adjoint source constructor

I uploaded the file here(this is just an example and it is not yet 100% complete). But it delievers what we have been discussed.

A bit concern about this way is we have two many in-place function definition. The major reason for that is we need to define some variables outside the function and the argument list is pretty limited for process function passing into the process_two_files_without_parallel_output().

Sorry, I named it proc_combo.py.txt for uploading. Please rename it to *.py.
proc_combo.py.txt

from pyasdf.

Hi Wenjie,

yea that is actually an intended design choice. Otherwise the process()/process_two_files_without_parallel_output() methods would have very awkward function signatures.

Python has reasonable support for functional programming, in this case closures and function currying. You are using closures: the function definitions you make bind the outside variables and thus the function can see them.

An alternative would be to use functools.partial for function currying/partial application in Python. Your example could look like this:

from functools import partial

def combo_func(obsd_station_group, synt_station_group,
              process_obsd_function, process_synt_function, window_function,
              adjoint_source):
    ...


results = obsd_ds.process_two_files_without_parallel_output(
    synt_ds, partial(combo_func,
                     process_obsd_function=process_obsd_function,
                     process_synt_function=process_synt_function,
                     window_function=window_function
                     adjoint_source=adjoint_source))

Using either allows to write a well structure program.

from pyasdf.

Thanks for the suggestion. I will use the functools.partial
👍

from pyasdf.