onlyzdd / clinical-fusion Goto Github PK

I already have the data extract and place them in the data folder includes:
adm_details.csv pivoted_lab.csv pivoted_vital.csv
and I suppose I need to run the following code:
$ python 00_define_cohort.py # define patient cohort and collect labels $ python 01_get_signals.py # extract temporal signals (vital signs and laboratory tests) $ python 02_extract_notes.py --firstday # extract first day clinical notes $ python 03_merge_ids.py # merge admission IDs $ python 04_statistics.py # run statistics $ python 05_preprocess.py # run preprocessing $ python 06_doc2vec.py --phase train # train doc2vec model $ python 06_doc2vec.py --phase infer # infer doc2vec vectors

however, when it comes to python 06_doc2vec.py --phase train
it always shows the error of RuntimeError: you must first build vocabulary before training the model

Are there any steps that I miss such that it will cause this error?

Thank you so much for your help!

When I run the line python 02_extract_notes.py

it shows the following warnings and I think this might affect the rest of the code:
`Reading data...
sys:1: DtypeWarning: Columns (5) have mixed types.Specify dtype option on import or set low_memory=False.
Extracting first 24 notes...
02_extract_notes.py:29: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_early['hr'] = (df_early['charttime'] - df_early['admittime']) / np.timedelta64(1, 'h')`

Hi,
I'm trying to run your code and cannot find the 'unstructure_dict.json' file . How is it generated?
thanks,

After setup the database with postgresql, I have tried to run the SQL in the query folder.

For example, this is what happen when I run the adm_details.sql: It shows SELECT 0

So I think it doesn't generate any views, not sure how to solve this. This also happens when I run other SQL
Thank you

Hi, I'm trying to run your code, and when I run 01_get_signals.py, I got an error keyerror:'hadm_id' in line 12.

I have successfully run the first five python file for preprocessing including:
$ python 00_define_cohort.py # define patient cohort and collect labels $ python 01_get_signals.py # extract temporal signals (vital signs and laboratory tests) $ python 02_extract_notes.py --firstday # extract first day clinical notes $ python 03_merge_ids.py # merge admission IDs $ python 04_statistics.py # run statistics $ python 05_preprocess.py # run preprocessing

However, when I tried to run the $ python 06_doc2vec.py --phase train # train doc2vec model
It shows:

The only line that I have modified is changed line 32 from:
train_ids = list(map(lambda x: int(x[-10:-4]), train_ids))
to
train_ids = list(map(lambda x: int(float(x[-10:-4])), train_ids))

Since if I don't add this float, it will result in the ValueError: invalid literal for int() with base 10:

When I first encounter the problem of RuntimeError: you must first build vocabulary before training the model I tried to change the min_count from 5 to 1 in line 39. However, it doesn't work.

Can you help me with this problem? Thank you so much for your help!❤️

Hi,

Thank you for the nice work.

Can you please share adm_details.csv, pivoted-lab.csv and pivoted-vital.csv files? The sql files are in the query folder but I need the csv files to run the code.

Thanks in advance.