crash build release for linux (centos, alpine) about openjph HOT 14 CLOSED

Hi Dao,

Do you know the CPU on the Linux machine? does it support AVX?
The command

 int cpu_ext_level();

declared in ojph_arch.h, and defined in ojph_arch.cpp, handles this. It should return 7 or more if the CPU supports AVX.

I am not familiar with rust. See if you can find the bug. If you cannot find it, I will try to help.

Kind regards,
Aous.

from openjph.

@aous72 Thanks you for helping this is my cpu info from centos machine
cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
stepping : 4
microcode : 0xffffffff
cpu MHz : 2199.999
cache size : 25600 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
stepping : 4
microcode : 0xffffffff
cpu MHz : 2199.999
cache size : 25600 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 1
initial apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
stepping : 4
microcode : 0xffffffff
cpu MHz : 2199.999
cache size : 25600 KB
physical id : 0
siblings : 4
core id : 2
cpu cores : 4
apicid : 2
initial apicid : 2
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz
stepping : 4
microcode : 0xffffffff
cpu MHz : 2199.999
cache size : 25600 KB
physical id : 0
siblings : 4
core id : 3
cpu cores : 4
apicid : 3
initial apicid : 3
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx f16c rdrand hypervisor lahf_lm fsgsbase smep erms xsaveopt
bogomips : 4399.99
clflush size : 64
cache_alignment : 64
address sizes : 42 bits physical, 48 bits virtual
power management:

from openjph.

I tried to print cpu level int cpu_ext_level()
{
assert(cpu_level_initialized);
printf("CPU LEVEL %d\n",cpu_level);
return cpu_level;
}

Give me CPU LEVEL 7

from openjph.

The CPU has AVX. It shouldn't crash on that line.

Are you encoding or decoding? Could you please give me the image and command line arguments.
I am not familiar Rust. Can you guide me on how to generate the bug? Maybe an article somewhere on the web?

Regards,
Aous

from openjph.

i have wrote both encode and decode everything is working with mac (debug release mode) linux debug mode but not release mode its so bizarre and took we weeks to debug.
https://github.com/ngoclinh93qt/openjph-ffi

you can install rust https://www.rust-lang.org/tools/install
and run cargo build --release for release

I really appreciate your help i still doesnt know why it crash on me for just release build
I can explain how the code work

from openjph.

Thank you for the info.
I will try to see what is wrong, but I cannot promise I can fix it -- I am not familiar with Rust, and its tools.

Regards,
Aous

from openjph.

Could you please push the latest version. I am having a compilation error
openjph-ffi/target/debug/deps/openjph_ffi-09434cc51be4cf43.2v9mt9q1turc5q8h.rcgu.o: In function openjph_ffi::ffi::ojph_htj2kcompress::encodedao': /home/aous/openjph-ffi/src/ffi.rs:1092: undefined reference to __ZN4ojph13htj2kcompress9encodedaoEPKhmmb'

Thank you.

from openjph.

@aous72 thanks you so much i really appreciate your help

you can run this for the error "scl enable llvm-toolset-7 bash"
Run ./generated.sh it will generate ffi.rs

from openjph.

Update: I think the problem is in build.rs line 38

    cc.flag("-mavx2");

In my library, I set -mavx2 flag for *_avx2.cpp files only.
You set it for the whole library; therefore, your code has AVX2 instructions that cannot be executed on an AVX machine.
Solution:
Set -mavx2 flag for *_avx2.cpp files only <--- best solution , or
Remove -mavx2 flag for *_avx2.cpp files

------------------------------------------------------------ Previous answer.
I am not familiar with Rust, so I needed a bit of time to just compile it.
Both debug and release work for me; no crash. Debug takes 112ms and release 10ms.

I suspect, but not sure, you are compiling to 32 bit some part of the code.

To test:

1. Disable SIMD like before (ISO build) and see if the crash is fixed.
2. Check all are 64 bit code.

Maybe these tests help.

Kind regards,
Aous.

from openjph.

Hi Aous
Confirm it works on Ubuntu, Mac for release(and debug) but not for Centos (only work with debug mode)
Im trying to working on your suggestion for the flags
Let me know if you have another suggestions i can test that too
Best regards,

Dao

from openjph.

I have tested in real device centos real device doesnt support axv but supporting sse2 i debug and found out the cpu level return 6 and it seem chose sse2 for colour and transform but i still got crash because of

Program received signal SIGILL, Illegal instruction.
0x00005555556b34d2 in ojph::local::bibo_gains::get_bibo_gain_l (num_decomp=5,
reversible=true) at vendor/openjph/src/core/codestream/ojph_params.cpp:449
449 { return reversible ? gain_5x3_l[num_decomp] : gain_9x7_l[num_decomp]; }

Is any thing you suggest that help me to fix this @aous72

Best regards

Dao

from openjph.

Hi Dao,

My CMakesLists.txt has

	set_source_files_properties(src/core/transform/ojph_colour_avx.cpp PROPERTIES COMPILE_FLAGS -mavx)
	set_source_files_properties(src/core/transform/ojph_colour_avx2.cpp PROPERTIES COMPILE_FLAGS -mavx2)
	set_source_files_properties(src/core/transform/ojph_transform_avx.cpp PROPERTIES COMPILE_FLAGS -mavx)
	set_source_files_properties(src/core/transform/ojph_transform_avx2.cpp PROPERTIES COMPILE_FLAGS -mavx2)

Therefore, use flag "-mavx" only with
src/core/transform/ojph_colour_avx.cpp, & src/core/transform/ojph_transform_avx.cpp
Use flag "-mavx2" only with
src/core/transform/ojph_colour_avx2.cpp & ojph_transform_avx2.cpp
All other files cannot have "-mavx", "-mavx2".

You crash on the sse2 machine because AVX code is used.

Hope this helps.

Kind regards,
Aous

from openjph.

Thanks you so much for your suggestion.
I have a way to work on now

Best regards

Dao

from openjph.

Excellent. looking forward for your library.

from openjph.