I'm also getting the following error when I try to create a plot: <div class="snip

And here's the database as a more readable table: <tr

could not convert string to float: 'data' about chronqc HOT 14 OPEN

nilesh-tawari commented on June 3, 2024

could not convert string to float: 'data'

from chronqc.

Comments (14)

multimeric commented on June 3, 2024

It seems to relate to df_dup_all.rolling(win).mean() and how that expects all the columns to be floats perhaps? It's quite strange.

from chronqc.

nilesh-tawari commented on June 3, 2024

@TMiguelT can you print the headers of chronqc_stats_data as well?

from chronqc.

nilesh-tawari commented on June 3, 2024

Seems you are trying to plot column with all values as data in your example, hence it is showing error.

from chronqc.

multimeric commented on June 3, 2024

Do you mean the SQLite3 database headers?

from chronqc.

nilesh-tawari commented on June 3, 2024

@TMiguelT yes

from chronqc.

multimeric commented on June 3, 2024

Okay let me run through the steps I performed (after running multiQC):

Successfully created the database:

$ chronqc database --create --run-date-info run_date_info.csv -o . multiqc/multiqc_data/multiqc_general_stats.txt AshTrio                                               
Running ChronQC |##################################################| 100.0%                                                                                                                                 
Created ChronQC db: /mnt/chronqc_db/chronqc.stats.sqlite with 6 records                                                                                                                                     
Created ChronQC default JSON file: /mnt/chronqc_db/chronqc.default.json. Customize the JSON as needed before generating ChronQC plots.

Then immediately after, I tried to plot (I didn't modify the JSON or the database in any way):

$ chronqc plot -o . chronqc_db/chronqc.stats.sqlite AshTrio chronqc_db/chronqc.default.json -f                                                                          
Started ChronQC                                                                                                                                                                                             
Traceback (most recent call last):                                                                                                                                                                          
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 211, in _prep_values                                                                                                            
    values = ensure_float64(values)                                                                                                                                                                         
  File "pandas/_libs/algos_common_helper.pxi", line 311, in pandas._libs.algos.ensure_float64                                                                                                               
ValueError: could not convert string to float: 'Run-1'                                                                                                                                                      
                                                                                                                                                                                                            
During handling of the above exception, another exception occurred:                                                                                                                                         

Traceback (most recent call last):
  File "/usr/local/bin/chronqc", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc.py", line 168, in main
    args.func(args)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc.py", line 21, in run_plot
    chronqc_plot.main(args)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 676, in main
    df_chart = mean_and_stdev(df, column_name, win=win, per_sample=per_sample)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 218, in mean_and_stdev
    df_dup_all = rolling_mean(df_dup_all, Duplicates, win)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 186, in rolling_mean
    df_dup_all['mean'] = df_dup_all.rolling(win).mean().round(2)[Duplicates]
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 1728, in mean
    return super(Rolling, self).mean(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 1072, in mean
    return self._apply('roll_mean', 'mean', **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 841, in _apply
    values = self._prep_values(b.values)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 214, in _prep_values
    "".format(values.dtype))
TypeError: cannot handle this type -> object

Here's the SQLite table with headers:

$ sqlite3 chronqc_db/chronqc.stats.sqlite -header 'select * from chronqc_stats_data'
Sample|Run|Date|FastQC_mqc-generalstats-fastqc-avg_sequence_length|FastQC_mqc-generalstats-fastqc-percent_duplicates|FastQC_mqc-generalstats-fastqc-percent_fails|FastQC_mqc-generalstats-fastqc-percent_gc|FastQC_mqc-generalstats-fastqc-total_sequences|Panel
TD06-GV1008_R1|Run-1|2018-01-01 00:00:00|126.0|23.5838138955758|16.6666666666667|48.0|75193388.0|AshTrio
TD06-GV1008_R2|Run-1|2018-01-01 00:00:00|126.0|23.5528852817332|16.6666666666667|48.0|75193388.0|AshTrio
TD06-GV1009_R1|Run-1|2018-02-01 00:00:00|126.0|21.1391622609802|16.6666666666667|48.0|64304319.0|AshTrio
TD06-GV1009_R2|Run-1|2018-02-01 00:00:00|126.0|21.3620029114608|16.6666666666667|48.0|64304319.0|AshTrio
TD06-GV1010_R1|Run-1|2018-03-01 00:00:00|126.0|22.3713007497403|16.6666666666667|48.0|73074989.0|AshTrio
TD06-GV1010_R2|Run-1|2018-03-01 00:00:00|126.0|22.5016825110702|16.6666666666667|48.0|73074989.0|AshTrio

from chronqc.

multimeric commented on June 3, 2024

And here's the database as a more readable table:

Sample	Run	Date	FastQC_mqc-generalstats-fastqc-avg_sequence_length	FastQC_mqc-generalstats-fastqc-percent_duplicates	FastQC_mqc-generalstats-fastqc-percent_fails	FastQC_mqc-generalstats-fastqc-percent_gc	FastQC_mqc-generalstats-fastqc-total_sequences	Panel
TD06-GV1008_R1	Run-1	2018-01-01 00:00:00	126.0	23.5838138955758	16.6666666666667	48.0	75193388.0	AshTrio
TD06-GV1008_R2	Run-1	2018-01-01 00:00:00	126.0	23.5528852817332	16.6666666666667	48.0	75193388.0	AshTrio
TD06-GV1009_R1	Run-1	2018-02-01 00:00:00	126.0	21.1391622609802	16.6666666666667	48.0	64304319.0	AshTrio
TD06-GV1009_R2	Run-1	2018-02-01 00:00:00	126.0	21.3620029114608	16.6666666666667	48.0	64304319.0	AshTrio
TD06-GV1010_R1	Run-1	2018-03-01 00:00:00	126.0	22.3713007497403	16.6666666666667	48.0	73074989.0	AshTrio
TD06-GV1010_R2	Run-1	2018-03-01 00:00:00	126.0	22.5016825110702	16.6666666666667	48.0	73074989.0	AshTrio

from chronqc.

nilesh-tawari commented on June 3, 2024

You need to customize the JSON, in your first example above you were trying to plot column with 'data' string, and in your example above you are plotting "Run" column with all values as "Run-1". Both strings 'data' and 'Run-1' are causing the errors. To customize the JSON simply see what you are plotting "y-value" and make sure it is "numerical value". Hope this helps.

from chronqc.

multimeric commented on June 3, 2024

But my JSON is fine. All of the y_value fields are numerical. I don't mention the Run field anywhere:

$ cat chronqc_db/chronqc.default.json 
[
    {
        "table_name": "chronqc_stats_data",
        "chart_type": "time_series_with_mean_and_stdev",
        "chart_properties": {
            "y_value": "FastQC_mqc-generalstats-fastqc-avg_sequence_length"
        }
    },
    {
        "table_name": "chronqc_stats_data",
        "chart_type": "time_series_with_mean_and_stdev",
        "chart_properties": {
            "y_value": "FastQC_mqc-generalstats-fastqc-percent_duplicates"
        }
    },
    {
        "table_name": "chronqc_stats_data",
        "chart_type": "time_series_with_mean_and_stdev",
        "chart_properties": {
            "y_value": "FastQC_mqc-generalstats-fastqc-percent_fails"
        }
    },
    {
        "table_name": "chronqc_stats_data",
        "chart_type": "time_series_with_mean_and_stdev",
        "chart_properties": {
            "y_value": "FastQC_mqc-generalstats-fastqc-percent_gc"
        }
    },
    {
        "table_name": "chronqc_stats_data",
        "chart_type": "time_series_with_mean_and_stdev",
        "chart_properties": {
            "y_value": "FastQC_mqc-generalstats-fastqc-total_sequences"
        }
    }
]

from chronqc.

multimeric commented on June 3, 2024

I'm happy to give you the files if it would help debug the issue

from chronqc.

multimeric commented on June 3, 2024

I'm getting this same issue with the example files here: https://github.com/nilesh-tawari/ChronQC/tree/master/examples/multiqc_example_1.

From within examples/multiqc_example_1, I run:

 $ chronqc database --create --run-date-info run_date_info.csv -o . multiqc_data/multiqc_general_stats.txt SOMATIC
Running ChronQC |##################################################| 100.0% 
Created ChronQC db: /home/michael/Programming/ChronQC/examples/multiqc_example_1/chronqc_db/chronqc.stats.sqlite with 100 records
Created ChronQC default JSON file: /home/michael/Programming/ChronQC/examples/multiqc_example_1/chronqc_db/chronqc.default.json. Customize the JSON as needed before generating ChronQC plots.

$ chronqc plot -o . chronqc_db/chronqc.stats.sqlite SOMATIC chronqc_db/chronqc.default.json
Started ChronQC
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 222, in _prep_values
    values = _ensure_float64(values)
  File "pandas/_libs/algos_common_helper.pxi", line 3182, in pandas._libs.algos.ensure_float64
  File "pandas/_libs/algos_common_helper.pxi", line 3187, in pandas._libs.algos.ensure_float64
ValueError: could not convert string to float: 'CHH13847 (15.7), CHH13848 (16.18), CHH13846 (15.21), CHH13849 (16.8)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/chronqc", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc.py", line 168, in main
    args.func(args)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc.py", line 21, in run_plot
    chronqc_plot.main(args)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 676, in main
    df_chart = mean_and_stdev(df, column_name, win=win, per_sample=per_sample)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 218, in mean_and_stdev
    df_dup_all = rolling_mean(df_dup_all, Duplicates, win)
  File "/usr/local/lib/python3.6/dist-packages/chronqc/chronqc_plot.py", line 186, in rolling_mean
    df_dup_all['mean'] = df_dup_all.rolling(win).mean().round(2)[Duplicates]
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 1605, in mean
    return super(Rolling, self).mean(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 1058, in mean
    return self._apply('roll_mean', 'mean', **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 844, in _apply
    values = self._prep_values(b.values)
  File "/usr/local/lib/python3.6/dist-packages/pandas/core/window.py", line 225, in _prep_values
    "".format(values.dtype))
TypeError: cannot handle this type -> object

from chronqc.

multimeric commented on June 3, 2024

I've investigated this some more, and the last version of pandas this works with is 0.22.0. If you use 0.23.0 or higher, this error will occur.

from chronqc.

nilesh-tawari commented on June 3, 2024

Thank you @TMiguelT for investigating it I will look at it and fix to work with updated pandas.

from chronqc.

multimeric commented on June 3, 2024

Excellent, that would be a much better solution than my version pinning

from chronqc.

could not convert string to float: 'data' about chronqc HOT 14 OPEN

Comments (14)

Related Issues (5)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent