Giter VIP home page Giter VIP logo

neondsr's Introduction

==== Big data analytics on dataset on hyperspectral images.

Lidar Matlab code UCSB

Linux alternatives to visualize and analyze LiDAR datasets Link

For compelete documentation visit https://sites.google.com/site/neondsrwiki/home

Comparison of ATCOR vs FLAASH here

Correction Features ATCOR FLAASH variable visibility yes yes aerosol type detection yes no adjacency effect yes yes water vapor retrieval yes 4) yes haze removal yes no cirrus cloud removal yes no cloud shadow removal yes no preview of spectra yes no inflight calibration yes no normalizing to nadir view (wide FOV imagery) yes no Physical Observer BRDF correction yes no spectral polishing yes 5) yes 5) spectral calibration yes yes spectral smile correction yes no empirical line correction no yes (ENVI)

neondsr's People

Contributors

mshahriarinia avatar

Watchers

avatar

Forkers

grseb9s hsiyjnd eburakaytekin sanwen211314

neondsr's Issues

SciDB Memory Error Halt

SciDB halts after a memory error and does not release memory.

Total memory: 2696164B Used: 2603516B, free: 92648B. Only 600 is used by non-SciDB softwares.

2013-10-19 18:45:39,429 [0x7fc3225ec700] [INFO ]: Executing query(1102219090553): redimension_store(hsi_img_flat, hsi_img); from program: 127.0.0.1:56429/opt/scidb/13.6/bin/iquery -a ;
2013-10-19 18:45:39,430 [0x7fc3225ec700] [DEBUG]: Request shared lock of array hsi_img_flat@1 for query 1102219090553
2013-10-19 18:45:39,430 [0x7fc3225ec700] [DEBUG]: Granted shared lock of array hsi_img_flat@1 for query 1102219090553
2013-10-19 18:45:39,454 [0x7fc3225ec700] [DEBUG]: Request exclusive lock of array hsi_img for query 1102219090553
2013-10-19 18:45:39,454 [0x7fc3225ec700] [DEBUG]: Granted exclusive lock of array hsi_img for query 1102219090553
2013-10-19 18:45:39,473 [0x7fc3225ec700] [DEBUG]: Request shared lock of array hsi_img@1 for query 1102219090553
2013-10-19 18:45:39,473 [0x7fc3225ec700] [DEBUG]: Granted shared lock of array hsi_img@1 for query 1102219090553
2013-10-19 18:45:39,487 [0x7fc3225ec700] [DEBUG]: [RedimStore] Begins.
2013-10-19 18:48:11,688 [0x7fc3225ec700] [DEBUG]: Create array NID_213@1:x(223) in query 1102219090553
2013-10-19 18:48:11,690 [0x7fc3225ec700] [DEBUG]: Request shared lock of array NID_213@1:x for query 1102219090553
2013-10-19 18:48:11,690 [0x7fc3225ec700] [DEBUG]: Granted shared lock of array NID_213@1:x for query 1102219090553
2013-10-19 18:49:58,725 [0x7fc3225ec700] [DEBUG]: Create array NID_213@1:y(224) in query 1102219090553
2013-10-19 18:49:58,726 [0x7fc3225ec700] [DEBUG]: Request shared lock of array NID_213@1:y for query 1102219090553
2013-10-19 18:49:58,726 [0x7fc3225ec700] [DEBUG]: Granted shared lock of array NID_213@1:y for query 1102219090553
2013-10-19 18:49:58,752 [0x7fc3225ec700] [DEBUG]: [RedimStore] build mapping index took 259264 ms, or 4 minutes 19 seconds 264 milliseconds
2013-10-19 19:10:03,048 [0x7fc3225ec700] [DEBUG]: [RedimStore] inputArray --> RowCollection took 1204062 ms, or 20 minutes 4 seconds 62 milliseconds
2013-10-19 19:10:40,538 [0x7fc3225ec700] [DEBUG]: Query::done: queryID=1102219090553, _commitState=0, erorCode=67
2013-10-19 19:10:40,564 [0x7fc3225ec700] [ERROR]: executeClientQuery failed to complete: SystemException in file: src/query/executor/SciDBExecutor.cpp function: executeQuery line: 233
Error id: scidb::SCIDB_SE_NO_MEMORY::SCIDB_LE_MEMORY_ALLOCATION_ERROR
Error description: Not enough memory. Error 'std::bad_alloc' during memory allocation.
Failed query id: 1102219090553
2013-10-19 19:10:40,610 [0x7fc3225ec700] [DEBUG]: Query (1102219090553) is being aborted
2013-10-19 19:10:40,610 [0x7fc3225ec700] [ERROR]: Query (1102219090553) error handlers (2) are being executed
2013-10-19 19:10:40,610 [0x7fc3225ec700] [DEBUG]: Update error handler is invoked for query (1102219090553)
2013-10-19 19:10:41,379 [0x7fc3224eb700] [DEBUG]: Query (1102219090553) is being aborted
2013-10-19 19:10:41,379 [0x7fc3224eb700] [DEBUG]: Deallocating query (1102219090553)
2013-10-19 19:10:41,493 [0x7fc3225ec700] [DEBUG]: UpdateErrorHandler::handleErrorOnCoordinator: the new version 1 of array hsi_img is being rolled back for query (1102219090553)
2013-10-19 19:10:41,502 [0x7fc3225ec700] [DEBUG]: Performing rollback
2013-10-19 19:10:41,565 [0x7fc3225ec700] [DEBUG]: End of log at position 224 rc=112
2013-10-19 19:10:41,565 [0x7fc3225ec700] [DEBUG]: End of log at position 0 rc=0
2013-10-19 19:10:41,691 [0x7fc3225ec700] [DEBUG]: Rollback complete
2013-10-19 19:10:42,320 [0x7fc3225ec700] [DEBUG]: Broadcast ABORT message to all instances for query 1102219090553
2013-10-19 19:10:42,320 [0x7fc3225ec700] [DEBUG]: Releasing locks for query 1102219090553
2013-10-19 19:10:42,320 [0x7fc3225ec700] [DEBUG]: SystemCatalog::deleteArrayLocks instanceId = 0 queryId = 1102219090553
2013-10-19 19:10:42,422 [0x7fc3225ec700] [DEBUG]: Release lock of array NID_213@1:x for query 1102219090553
2013-10-19 19:10:42,432 [0x7fc3225ec700] [DEBUG]: Release lock of array NID_213@1:y for query 1102219090553
2013-10-19 19:10:42,433 [0x7fc3225ec700] [DEBUG]: Release lock of array hsi_img for query 1102219090553
2013-10-19 19:10:42,433 [0x7fc3225ec700] [DEBUG]: Release lock of array hsi_img@1 for query 1102219090553
2013-10-19 19:10:42,433 [0x7fc3225ec700] [DEBUG]: Release lock of array hsi_img_flat@1 for query 1102219090553

SciDB Startall Stopall Fail

After restarting my machine. When trying to start SciDB I get:

scidb@dsr01:~$ scidb.py startall SingleInstance  
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.3
140%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
SciDB is still running. Try the stopall command before starting.

When trying to stop them:

scidb@dsr01:~$ scidb.py stopall SingleInstance                                                                                                                                                                           [23/168]
stop(server 0 (localhost) local instance 0)
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
871%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
897%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
790%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
790%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
735%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
690%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
627%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
770%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.3
015%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
957%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:52 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.2
966%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes
Failed to stop SciDB!

Check processes:

scidb@dsr01:~$ scidb.py check-pids SingleInstance                                                               
checking (server 0 (localhost) local instance 0) Welcome scidb to dsr01.             Mon, 14 Oct 2013 - 17:55 PM
 EDT  Operating system: CentOS release 6.4 (Final)  eth0 IP address: 128.227.170.232         Free memory:   43.0
446%  Disk usage: /dev/mapper/vg_dsr01-lv_root 46% /dev/sda1 21% /dev/mapper/vg_dsr01-lv_home 9%   ... 
Found 13 scidb processes

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.