juliageo / netcdf.jl Goto Github PK

View Code? Open in Web Editor NEW

115.0 14.0 28.0 7.48 MB

NetCDF support for the julia programming language

Home Page: http://juliageo.org/NetCDF.jl/

License: MIT License

Julia 100.00%

netcdf julia io geo geospatial raster

netcdf.jl's Introduction

NetCDF.jl

Documentation	Build Status

NetCDF support for the Julia programming language, there is a high-level and a medium-level interface for writing and reading netcdf files.

Installation

pkg> add NetCDF

Quickstart

First, load the library:

using NetCDF

The high-level interface is quite similar to the Matlab NetCDF interface, reading files is done by:

x = ncread("myfile.nc", "Radiation")

which will read the variable called "Radiation" from the file "myfile.nc". General information can be gained by using

ncinfo(filename)

which gives an overview of the dimensions, variables and attributes stored in the file.

filename = "myfile.nc"
varname  = "var1"
attribs  = Dict("units"   => "mm/d",
                "data_min" => 0.0,
                "data_max" => 87.0)

Creating variables and files is done by using the nccreate command:

nccreate(filename, varname, "x1", collect(11:20), "t", 20, Dict("units"=>"s"), atts=attribs)

This will create the variable called var1 in the file myfile.nc. The attributes defined in the Dict attribs are written to the file and are associated with the newly created variable. The dimensions "x1" and "t" of the variable are called "x1" and "t" in this example. If the dimensions do not exist yet in the file, they will be created. The dimension "x1" will be of length 10 and have the values 11..20, and the dimension "t" will have length 20 and the attribute "units" with the value "s".

Now we can write data to the file:

d = rand(10, 20)
ncwrite(d, filename, varname)

The full documentation can be found here

An alternative interface for reading NetCDF files can be found here: https://github.com/Alexander-Barth/NCDatasets.jl

DiskArray interface

As of version 0.9 we provide an interface to DiskArrays.jl, which lets you treat NetCDF variables as Julia AbstractArrays. Special methods for accessing, reading slices and reductions and broadcasting are implemented, so that working with the arrays should be efficient.

So the following returns a lazy AbstractArray container referencing the data in the variable.

v = NetCDF.open(filename, varname)

Credits

This package was originally started and is mostly maintained by Fabian Gans ([email protected]). The automatic C wrapper generator was contributed by Martijn Visser (https://github.com/visr). Many thanks to several people who contributed bug fixes and enhancements.

netcdf.jl's People

Contributors

Stargazers

Watchers

netcdf.jl's Issues

Deal with groups in nc data

Some netcdf data files have variables categorized into groups, is there a way to read such data.

For example, if there is a variable 'a' in group 'A' in 'data.nc',

in MATLAB ncread('data.nc','/A/a') reads out such variable.

in python NetCDF4.Dataset('data.nc').groups['A'].variables['a'] also works

Is there a way to read it in Julia?

Document chunking in nccreate

Please let me know if I'm doing this wrong but I was trying to find a nice balance between compression time and file size by benchmarking compression levels 0-9 but instead find that the compression level does nothing.

using NetCDF
N = 256^3
A = rand(N)

for cl in 0:9
    tic = time_ns()
    
    filename = "compress" * string(cl) * ".nc"
    varname  = "rands"
    attribs  = Dict("units"   => "m/s")

    nccreate(filename, varname, "x1", collect(1:N), Dict("units"=>"m"), atts=attribs, compress=cl)
    ncwrite(A, filename, varname)
    ncclose(filename)
    
    toc = time_ns()
    
    ts = prettytime(toc - tic)
    fs = datasize(filesize(filename); style=:bin, format="%.3f")
    println("Compression level $cl: $ts $fs")
end

Compression level 0: 4.784 s 233.989 MiB
Compression level 1: 4.618 s 233.989 MiB
Compression level 2: 4.358 s 233.989 MiB
Compression level 3: 5.900 s 233.989 MiB
Compression level 4: 4.456 s 233.989 MiB
Compression level 5: 4.643 s 233.989 MiB
Compression level 6: 4.353 s 233.989 MiB
Compression level 7: 5.022 s 233.989 MiB
Compression level 8: 6.425 s 233.989 MiB
Compression level 9: 4.271 s 233.989 MiB

NetCDF reads values from file wrongly.

Hello,

I am reading a large NetCDF file (around 900+ MB) using NetCDF. However, what happens is that when I use the NetCDF package, I retrieve a negative value. This is not possible, because the data is all positive.

I am able to read the NetCDF file using the package NCDatasets, and with MATLAB.

When using NetCDF:

julia> a = ncread("/Volumes/CliNat-ERA/era5/SEA/tcwv/tmp/era5-SEA-tcwv-sfc-2017.nc","tcwv",[3,4,5],[1,1,1])
1×1×1 Array{Int16,3}:
[:, :, 1] =
 -20740

When using NCDatasets

julia> ds = Dataset("/Volumes/CliNat-ERA/era5/SEA/tcwv/tmp/era5-SEA-tcwv-sfc-2017.nc")
julia> tcwv = ds["tcwv"][3,4,5]
18.572314914331237

And when using MATLAB

ncread("/Volumes/CliNat-ERA/era5/SEA/tcwv/tmp/era5-SEA-tcwv-sfc-2017.nc","tcwv",[3,4,5],[1,1,1])
ans = 
    18.5723

is there any sensing on what the issue might be?
NCDatasets returns the following

julia> ds = Dataset("/Volumes/CliNat-ERA/era5/SEA/tcwv/tmp/era5-SEA-tcwv-sfc-2017.nc")
Dataset: /Volumes/CliNat-ERA/era5/SEA/tcwv/tmp/era5-SEA-tcwv-sfc-2017.nc
Group: /

Dimensions
   longitude = 301
   latitude = 141
   time = 8760

Variables
  longitude   (301)
    Datatype:    Float32
    Dimensions:  longitude
    Attributes:
     units                = degrees_east
     long_name            = longitude

  latitude   (141)
    Datatype:    Float32
    Dimensions:  latitude
    Attributes:
     units                = degrees_north
     long_name            = latitude

  time   (8760)
    Datatype:    Int32
    Dimensions:  time
    Attributes:
     units                = hours since 1900-01-01 00:00:00.0
     long_name            = time
     calendar             = gregorian

  tcwv   (301 × 141 × 8760)
    Datatype:    Int16
    Dimensions:  longitude × latitude × time
    Attributes:
     scale_factor         = 0.0013398768625944388
     add_offset           = 46.3613610445399
     _FillValue           = -32767
     missing_value        = -32767
     units                = kg m**-2
     long_name            = Total column water vapour
     standard_name        = lwe_thickness_of_atmosphere_mass_content_of_water_vapor

Global attributes
  Conventions          = CF-1.6
  history              = 2019-04-15 07:16:09 GMT by grib_to_netcdf-2.10.0: /opt/ecmwf/eccodes/bin/grib_to_netcdf -o /cache/data0/adaptor.mars.internal-1555311761.772098-29751-1-e896a99a-0eaf-4622-b36a-eb9407eab000.nc /cache/tmp/e896a99a-0eaf-4622-b36a-eb9407eab000-adaptor.mars.internal-1555311761.7736168-29751-1-tmp.grib

Ncread throws: invalid array dimensions while reading a variable.

Hi! using v 0.4.2

I'm trying to read, from a NetCDF file, variable, in fact several of them with no success:
Example: This ZNU variable has the size 24*27 and i can't read it with.

julia> ncread(Arch,"ZNU",start=[1,1],count=[-1,1])
ERROR: invalid Array dimensions
 in call at essentials.jl:201
 in ncread at /home/argel/.julia/v0.4/NetCDF/src/NetCDF.jl:545

If i preallocate the destination array
Znu = zeros(27)
I can succesfully read it:

ncread!(Arch,"ZNU", Znu,start=[1,1],count=[-1,1])
27-element Array{Float64,1}

The description of this variable, according to ncinfo is:

ZNU                  units                                                          
ZNU                  stagger                                                        
ZNU                  FieldType            104                                       
ZNU                  MemoryOrder          Z                                         
ZNU                  description          eta values on half (mass) levels

Is this a bug or doing something wrong?

Wrong getindex behaviour when reading a 3D array

Here, f is a NetCDF file containing a 3D array.

f.vars[key][:, :, 1] should give a 2D array, but it gives a 3D one (with a 3rd dimension size of 1) instead.

Moving NetCDF to JuliaGEO (and to julia 0.4)

Hi all,

as I mentioned earlier, I have done a big rewrite of the NetCDF package. It was mainly cleaning up code so that it is hopefully easier to maintain, but there are also some new features, mainly that NcVar now inherits from AbstractArray, so that the following is possible:

v=NetCDF.open("myfile.nc","Tair") 
x=v[1:10,:,20:100]

instead of the old start and count syntax, which is of course still supported.
I think since julia 0.4 is going to be released soon I would like to ask interested people to test if everything is still working as expected and report any bugs if they appear.

Method error when running high.jl

When I running high.jl line by line, I got the following error.

julia> nccreate(fn1,"vchar","DimChar",20,"Dim2",t=NetCDF.NC_CHAR)
20×10 NcVar{NetCDF.ASCIIChar,2,2}:
Error showing value of type NcVar{NetCDF.ASCIIChar,2,2}:
ERROR: MethodError: no method matching nc_get_var1_x(::NcVar{NetCDF.ASCIIChar,2,2}, ::Array{UInt64,1}, ::Type{NetCDF.ASCIIChar})
Closest candidates are:
nc_get_var1_x(::NcVar{Float64,N,M} where M where N, ::Array{UInt64,1}, ::Type{Float64}) at /home/peter/.julia/dev/NetCDF/src/NetCDF.jl:352
nc_get_var1_x(::NcVar{Float32,N,M} where M where N, ::Array{UInt64,1}, ::Type{Float32}) at /home/peter/.julia/dev/NetCDF/src/NetCDF.jl:352
nc_get_var1_x(::NcVar{Int32,N,M} where M where N, ::Array{UInt64,1}, ::Type{Int32}) at /home/peter/.julia/dev/NetCDF/src/NetCDF.jl:352
...
Stacktrace:
[1] readvar(::NcVar{NetCDF.ASCIIChar,2,2}, ::Int64, ::Int64) at /home/peter/.julia/dev/NetCDF/src/NetCDF.jl:309
[2] getindex(::NcVar{NetCDF.ASCIIChar,2,2}, ::Int64, ::Int64) at /home/peter/.julia/dev/NetCDF/src/NetCDF.jl:173
[3] isassigned(::NcVar{NetCDF.ASCIIChar,2,2}, ::Int64, ::Int64) at ./abstractarray.jl:351
[4] alignment(::IOContext{REPL.Terminals.TTYTerminal}, ::NcVar{NetCDF.ASCIIChar,2,2}, ::Array{Int64,1}, ::UnitRange{Int64}, ::Int64, ::Int64, ::Int64) at ./arrayshow.jl:67
[5] print_matrix(::IOContext{REPL.Terminals.TTYTerminal}, ::NcVar{NetCDF.ASCIIChar,2,2}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at ./arrayshow.jl:186
[6] print_matrix at ./arrayshow.jl:159 [inlined]
[7] print_array at ./arrayshow.jl:308 [inlined]
[8] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::NcVar{NetCDF.ASCIIChar,2,2}) at ./arrayshow.jl:345
[9] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:131
[10] display(::REPL.REPLDisplay, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:135
[11] display(::Any) at ./multimedia.jl:287
[12] #invokelatest#1 at ./essentials.jl:742 [inlined]
[13] invokelatest at ./essentials.jl:741 [inlined]
[14] print_response(::IO, ::Any, ::Any, ::Bool, ::Bool, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:155
[15] print_response(::REPL.AbstractREPL, ::Any, ::Any, ::Bool, ::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:140
[16] (::getfield(REPL, Symbol("#do_respond#38")){Bool,getfield(REPL, Symbol("##48#57")){REPL.LineEditREPL},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::Any, ::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:714
[17] (::getfield(REPL, Symbol("##53#62")))(::Any, ::Any, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:977
[18] #invokelatest#1 at ./essentials.jl:742 [inlined]
[19] invokelatest at ./essentials.jl:741 [inlined]
[20] (::getfield(REPL.LineEdit, Symbol("##27#28")){getfield(REPL, Symbol("##53#62")),String})(::Any, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:1324
[21] prompt!(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2365
[22] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2268
[23] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
[24] run_repl(::REPL.AbstractREPL, ::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
[25] (::getfield(Base, Symbol("##734#736")){Bool,Bool,Bool,Bool})(::Module) at ./client.jl:362
[26] #invokelatest#1 at ./essentials.jl:742 [inlined]
[27] invokelatest at ./essentials.jl:741 [inlined]
[28] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./client.jl:346
[29] exec_options(::Base.JLOptions) at ./client.jl:284
[30] _start() at ./client.jl:436

Support scalar variables

It's possible to have a NetCDF file with variables of 0 dimensions (i.e. scalars). Currently, NetCDF.jl basically ignores their presence in a file (LoadError: NetCDF file test.nc does not have a variable named foo), and nccreate errors as well:

julia> nccreate("test.nc", "foo")
LoadError: MethodError: no method matching NetCDF.NcDim(::Void, ::Int64; atts=Dict{Any,Any}(Pair{Any,Any}("missval",-9999)), values=Float64[], unlimited=false)

Could not use nccreate in julia pro

I use Julia Pro under Windows 10 with version 1.01
Nowadays I find I cannot use the nccreate function in my code.
my code is :

filename = "myfile.nc" varname = "var1" attribs = Dict("units" => "mm/d", "data_min" => 0.0, "data_max" => 87.0) nccreate(filename, varname, "x1", 11:20, "t", 20, Dict("units"=>"s"), atts=attribs)

and output is

`
Unreachable reached at 000000001427D6F8

Please submit a bug report with steps to reproduce this fault and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ILLEGAL_INSTRUCTION at 0x1427d6f8 -- #NcVar#6 at C:\Users\zfh1997\AppData\Local
\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\NetCDF\ttVCN\src\NetCDF.jl:143
in expression starting at D:\Course\Atmospheric_modeling\hw2\julia_anal.jl:32
#NcVar#6 at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\NetCDF\ttVCN\src\N
etCDF.jl:143
unknown function (ip: 000000001427D778)
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184 [inlined]
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:56
Type at .\none:0
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
#nccreate#51 at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\NetCDF\ttVCN\s
rc\NetCDF.jl:879
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
jl_f__apply at /home/Administrator/buildbot/worker/package_win64/build/src\builtins.c:556
#nccreate at .\none:0
jl_fptr_trampoline at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:1831
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
do_call at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:324
eval_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:430
eval_stmt_value at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:363 [in
lined]
eval_body at /home/Administrator/buildbot/worker/package_win64/build/src\interpreter.c:678
jl_interpret_toplevel_thunk_callback at /home/Administrator/buildbot/worker/package_win64/build/src
interpreter.c:795
unknown function (ip: FFFFFFFFFFFFFFFE)
unknown function (ip: 000000000E54C2DF)
unknown function (ip: 0000000000000007)
jl_toplevel_eval_flex at /home/Administrator/buildbot/worker/package_win64/build/src\toplevel.c:813
jl_parse_eval_all at /home/Administrator/buildbot/worker/package_win64/build/src\ast.c:838
include_string at .\loading.jl:1005
#118 at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\Atom\WSz3k\src\eval.jl
:120
withpath at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\CodeTools\hB4Hy\sr
c\utils.jl:30
jl_invoke at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:42
withpath at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\Atom\WSz3k\src\eva
l.jl:46 [inlined]
#117 at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\Atom\WSz3k\src\eval.jl
:117 [inlined]
hideprompt at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\Atom\WSz3k\src\r
epl.jl:76
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
macro expansion at C:\Users\zfh1997\AppData\Local\JuliaPro-1.0.1.1\pkgs-1.0.1.1\packages\Atom\WSz3k
src\eval.jl:116 [inlined]
#116 at .\task.jl:85
jl_apply_generic at /home/Administrator/buildbot/worker/package_win64/build/src\gf.c:2184
jl_apply at /home/Administrator/buildbot/worker/package_win64/build/src\julia.h:1537 [inlined]
start_task at /home/Administrator/buildbot/worker/package_win64/build/src\task.c:268
Allocations: 60727552 (Pool: 60716750; Big: 10802); GC: 136
`
I run both Julia and Atom with Admin. Thanks a lot.

method deprecated

I get this error when showing the output of ncinfo. looks like a minor issue though.

WARNING: tty_size is deprecated. use `displaysize(io)` as a replacement
 in depwarn(::String, ::Symbol) at ./deprecated.jl:64
 in tty_size() at ./deprecated.jl:182
 in show(::IOContext{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::NetCDF.NcFile) at /home/proy/.julia/v0.5/NetCDF/src/NetCDF.jl:734
 in (::Juno.##1#2{NetCDF.NcFile})(::Base.AbstractIOBuffer{Array{UInt8,1}}) at /home/proy/.julia/v0.5/Juno/src/types.jl:11
 in #sprint#304(::Void, ::Function, ::Int64, ::Function) at ./strings/io.jl:37
 in Type at /home/proy/.julia/v0.5/Juno/src/types.jl:51 [inlined]
 in Type at /home/proy/.julia/v0.5/Juno/src/types.jl:52 [inlined]
 in render(::Juno.Console, ::NetCDF.NcFile) at /home/proy/.julia/v0.5/Juno/src/types.jl:16
 in (::Atom.##60#63)() at /home/proy/.julia/v0.5/Atom/src/eval.jl:41
 in withpath(::Atom.##60#63, ::Void) at /home/proy/.julia/v0.5/CodeTools/src/utils.jl:30
 in withpath(::Function, ::Void) at /home/proy/.julia/v0.5/Atom/src/eval.jl:45
 in macro expansion at /home/proy/.julia/v0.5/Atom/src/eval.jl:107 [inlined]
 in (::Atom.##59#62)() at ./task.jl:60

Allow ncclose to accept an NcFile

It would be useful to allow ncclose to accept an NcFile handle as an alternative to a filename. It would be more in line with most other netCDF implementations.

issue with libnetcdf not properly installed

Hi, I just installed Julia (version 1.0.5) on windows 10 and I am trying to use the NetCDF package. I did add NetCDF, but when coming to actually use the package I get the following error:

[ Info: Precompiling NetCDF [30363a11-5582-574a-97bb-aa9a979735b9] ERROR: LoadError: LoadError: libnetcdf not properly installed. Please run Pkg.build("NetCDF") Stacktrace: [1] error(::String) at .\error.jl:33 [2] top-level scope at C:\Users\david\.julia\packages\NetCDF\dmH61\src\netcdf_c.jl:7 [3] include at .\boot.jl:317 [inlined] [4] include_relative(::Module, ::String) at .\loading.jl:1044 [5] include at .\sysimg.jl:29 [inlined] [6] include(::String) at C:\Users\david\.julia\packages\NetCDF\dmH61\src\NetCDF.jl:1 [7] top-level scope at none:0 [8] include at .\boot.jl:317 [inlined] [9] include_relative(::Module, ::String) at .\loading.jl:1044 [10] include(::Module, ::String) at .\sysimg.jl:29 [11] top-level scope at none:2 [12] eval at .\boot.jl:319 [inlined] [13] eval(::Expr) at .\client.jl:393 [14] top-level scope at .\none:3 in expression starting at C:\Users\david\.julia\packages\NetCDF\dmH61\src\netcdf_c.jl:4 in expression starting at C:\Users\david\.julia\packages\NetCDF\dmH61\src\NetCDF.jl:6

I guess it has something to do with the netcdf libraries, but I really can't figure out what's wrong. I also tried
Conda.runconda(install -c anaconda libnetcdf)
but this did not work either.
May anybody help with some suggestions?
Thanks
Davide

Segfault on dummy input

julia> versioninfo()
Julia Version 1.3.0-rc4.1
Commit 8c4656b97a (2019-10-15 14:08 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, ivybridge)
Environment:
  JULIA_EDITOR = atom  -a
  JULIA_NUM_THREADS = 24

Status `~/.julia/environments/v1.3/Project.toml`
  [7d9fca2a] Arpack v0.3.1
  [c52e3926] Atom v0.11.3
  [6e4b80f9] BenchmarkTools v0.4.3
  [336ed68f] CSV v0.5.13
  [be33ccc6] CUDAnative v2.5.4
  [324d7699] CategoricalArrays v0.7.1
  [3a865a2d] CuArrays v1.4.4
  [5a033b19] CurveFit v0.3.2
  [a93c6f00] DataFrames v0.19.4
  [39dd38d3] Dierckx v0.4.1
  [aaf54ef3] DistributedArrays v0.6.4
  [186dfeec] FEniCS v0.3.2
  [9aa1b823] FastClosures v0.3.2
  [5789e2e9] FileIO v1.0.7
  [1a297f60] FillArrays v0.8.0
  [a98d9a8b] Interpolations v0.12.5
  [42fd0dbc] IterativeSolvers v0.8.1
  [033835bb] JLD2 v0.1.3
  [f80590ac] JuliaFEM v0.5.1
  [e5e0dc1b] Juno v0.7.2
  [b964fa9f] LaTeXStrings v1.0.3
  [30363a11] NetCDF v0.8.0
  [69de0a69] Parsers v0.3.8
  [91a5bcdd] Plots v0.27.0
  [c46f51b8] ProfileView v0.5.0
  [438e738f] PyCall v1.91.2
  [1fd47b50] QuadGK v2.1.1
  [276daf66] SpecialFunctions v0.8.0
  [2913bbd2] StatsBase v0.32.0
  [bd369af6] Tables v0.2.11
  [37b6cedf] Traceur v0.3.0
  [b8865327] UnicodePlots v1.1.0
  [37e2e46d] LinearAlgebra 

using NetCDF
function func(zi=820.0,HCeil=1400.0,source3d="/media/[username]/Asterix/boroksim2h_3d.001.nc",sourcepr = "/media/[username]/Asterix/boroksim2h_pr.001.nc")
    ztheta::Array{Float64,1} = ncread(sourcepr,"ztheta")
    return 1
end
loot = func()#1
loot = func(820.0,1400.0,"","")#segfault

Error building NetCDF

When I try to build NetCDF I get the following error

(esdl_sentinel) pkg> build NetCDF
  Building Conda ─→ `~/.julia/packages/Conda/kLXeC/deps/build.log`
  Building NetCDF → `~/.julia/packages/NetCDF/SFmQh/deps/build.log`
 Resolving package versions...
┌ Error: Error building `NetCDF`: 
│ ERROR: LoadError: IOError: could not spawn `ldconfig -p`: no such file or directory (ENOENT)
│ Stacktrace:
│  [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at ./process.jl:401
│  [2] setup_stdios(::getfield(Base, Symbol("##526#527")){Cmd}, ::Array{Any,1}) at ./process.jl:414
│  [3] _spawn at ./process.jl:413 [inlined]
│  [4] #eachline#532(::Bool, ::typeof(eachline), ::Cmd) at ./process.jl:619
│  [5] eachline at ./process.jl:618 [inlined]
│  [6] read_sonames() at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:398
│  [7] lookup_soname at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:431 [inlined]
│  [8] #_find_library#42(::Type, ::typeof(BinDeps._find_library), ::BinDeps.LibraryDependency) at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:708
│  [9] _find_library at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:643 [inlined]
│  [10] satisfy!(::BinDeps.LibraryDependency, ::Array{DataType,1}) at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:922 (repeats 2 times)
│  [11] top-level scope at /home/qe89hep/.julia/packages/BinDeps/eiJeV/src/dependencies.jl:977
│  [12] include at ./boot.jl:328 [inlined]
│  [13] include_relative(::Module, ::String) at ./loading.jl:1094
│  [14] include(::Module, ::String) at ./Base.jl:31
│  [15] include(::String) at ./client.jl:431
│  [16] top-level scope at none:5
│ in expression starting at /home/qe89hep/.julia/packages/NetCDF/SFmQh/deps/build.jl:15
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/Pkg/src/backwards_compatible_isolation.jl:647
'''
This is on julia version 1.2.0 and NetCDF version 0.8.
Is there anything I could try to make this work?

Add support for reading entire dataset

It seems ncread requires a variable argument name. It would be convenient, and more efficient in some cases, to be able to read the entire dataset rather than specified variables.

different result between ncread and matlab‘s ncread

Hi, I get different result between ncread and matlab‘s ncread. And it doesn't look like offset or scale_factor‘s problem.

julia>mss=ncread(MSS_str,"mss")

julia> mss
21602×10800 Array{Int32,2}:
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14972  14947  14924  14900  14876  14852  14817
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14955  14930  14907  14884  14859  14835  14801
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14928  14905  14882  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14928  14905  14882  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14928  14905  14882  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14954  14928  14905  14882  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14905  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14905  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14905  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14953  14929  14904  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14953  14928  14904  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14953  14928  14904  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14953  14928  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14953  14928  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14953  14928  14904  14881  14857  14833  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14954  14929  14905  14882  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14905  14882  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14905  14882  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14954  14928  14905  14882  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14954  14928  14905  14882  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28856  …  14953  14927  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14953  14927  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14953  14927  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14953  14927  14904  14881  14857  14833  14799
 -28914  -28904  -28894  -28885  -28875  -28866  -28856     14953  14927  14904  14881  14857  14833  14799
      ⋮                                       ⋮          ⋱                    ⋮                            
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14953  14929  14906  14882  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14953  14929  14906  14882  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857  …  14953  14929  14906  14882  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14953  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857  …  14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28895  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14858  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14834  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14954  14929  14906  14883  14859  14835  14800
 -28914  -28904  -28894  -28885  -28875  -28866  -28857  …  14972  14947  14924  14901  14877  14852  14817
 -28914  -28904  -28894  -28885  -28875  -28866  -28857     14955  14930  14907  14884  14860  14836  14801

% matlab
mss=ncread(MSS_str,'mss');

Here are some parts of matlab result

%matlab
>> ncdisp('./data/mss/DTU15MSS_1min.nc')
Source:
           xxx/data/mss/DTU15MSS_1min.nc
Format:
           classic
Global Attributes:
           Conventions = 'COARDS/CF-1.0'
           title       = 'DTU15MSS_1min.nc'
           source      = 'Danish National Space Center'
           node_offset = 1
Dimensions:
           lon = 21602
           lat = 10800
Variables:
    lon
           Size:       21602x1
           Dimensions: lon
           Datatype:   double
           Attributes:
                       long_name    = 'longitude'
                       units        = 'degrees_east'
                       actual_range = [-0.01666667      360.0167]
    lat
           Size:       10800x1
           Dimensions: lat
           Datatype:   double
           Attributes:
                       long_name    = 'latitude'
                       units        = 'degrees_north'
                       actual_range = [-90  90]
    mss
           Size:       21602x10800
           Dimensions: lon,lat
           Datatype:   int32
           Attributes:
                       long_name    = 'mean sea surface height'
                       units        = 'm'
                       scale_factor = 0.001
                       actual_range = [-105.568         86.76]

julia V1.2.0
julia NetCDF v0.8.0
matlab 2017b
macOS

build doesn't work on Windows

When I do Pkg.add("NetCDF") on Windows I get the following output:

   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.6 (2016-06-19 17:16 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-w64-mingw32

julia> Pkg.add("NetCDF")
INFO: Installing NetCDF v0.3.1
INFO: Building NetCDF
Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Fetching package metadata .......
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at C:\Users\anthoff\.julia\v0.4\Conda\deps\usr:
#
libnetcdf                 4.3.3.1                   vc9_5  [vc9]
Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

Get-Process : Cannot find a process with the name "Outlook". Verify the process name and call the cmdlet again.
At line:1 char:1
+ Get-Process Outlook
+ ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Outlook:String) [Get-Process], ProcessCommandException
    + FullyQualifiedErrorId : NoProcessFoundForGivenName,Microsoft.PowerShell.Commands.GetProcessCommand

===============================[ ERROR: NetCDF ]================================

LoadError: Provider BinDeps.PackageManager failed to satisfy dependency libnetcdf
while loading C:\Users\anthoff\.julia\v0.4\NetCDF\deps\build.jl, in expression starting on line 12

================================================================================

==========================================================================[ BUILD ERRORS ]==========================================================================

WARNING: NetCDF had build errors.

 - packages with build errors remain installed in C:\Users\anthoff\.julia\v0.4
 - build the package(s) and all dependencies with `Pkg.build("NetCDF")`
 - build a single package by running its `deps/build.jl` script

====================================================================================================================================================================
INFO: Package database updated

julia>

FreeBSD precompilation fails

I have a package, which fails the cirrus-ci tests on FreeBSD with Julia 1.0,1.2 as well as nightly due to its dependencies on NetCDF.jl, whereas Linux, osx, win pass (with Travis & appveyor).

 [1] error(::String) at ./error.jl:33
 [2] top-level scope at /.julia/packages/NetCDF/wnKH9/src/netcdf_c.jl:7
 [3] include at ./boot.jl:328 [inlined]
 [4] include_relative(::Module, ::String) at ./loading.jl:1094
 [5] include at ./Base.jl:31 [inlined]
 [6] include(::String) at /.julia/packages/NetCDF/wnKH9/src/NetCDF.jl:3
 [7] top-level scope at /.julia/packages/NetCDF/wnKH9/src/NetCDF.jl:8
...
in expression starting at /.julia/packages/NetCDF/wnKH9/src/netcdf_c.jl:4
in expression starting at /.julia/packages/NetCDF/wnKH9/src/NetCDF.jl:8

ldconfig not found

Building this package seems to trigger a call to 'ldconfig -p' since recently, which fails on my opensuse15.1 system as a normal user since ldconfig is in /sbin which is not in the path for a normal user. Adding /sbin to the path is a workaround, but perhaps it is more convenient/safer to fix this in the package?

Keep up the good work for this excellent package,
Martin

Illegal Instruction in Julia 1.0.1 (conda forge)

I am not sure if NetCDF.jl is supposed to work in 1.0.1 or not. I got error with the same code in the README part with Dict modified to 1.0 syntax. Hope this might help developing.

Code:

using NetCDF

filename   = "myfile.nc"
varname    = "var1"
attribs    = Dict("units"    => "mm/d",
              "data_min" => 0.0,
              "data_max" => 87.0)

nccreate(filename, varname, "x1", 11:20, "t", 20, Dict("units"=>"s"), atts=attribs)

Output:

Unreachable reached at 0x7fc9e2b25633

signal (4): Illegal instruction
in expression starting at /home/tienyiah/testjulia/testjulia.jl:9
#NcVar#6 at /home/tienyiah/.julia/packages/NetCDF/ttVCN/src/NetCDF.jl:143
unknown function (ip: 0x7fc9e2b256bb)
jl_fptr_trampoline at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:1831
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
Type at ./none:0
jl_fptr_trampoline at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:1831
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
#nccreate#51 at /home/tienyiah/.julia/packages/NetCDF/ttVCN/src/NetCDF.jl:879
jl_fptr_trampoline at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:1831
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
jl_apply at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/julia.h:1537 [inlined]
jl_f__apply at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/builtins.c:556
#nccreate at ./none:0
jl_fptr_trampoline at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:1831
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
do_call at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:324
eval_value at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:430
eval_stmt_value at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:363 [inlined]
eval_body at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:678
jl_interpret_toplevel_thunk_callback at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:795
unknown function (ip: 0xfffffffffffffffe)
unknown function (ip: 0x7fc9fd8d85af)
unknown function (ip: 0x7)
jl_interpret_toplevel_thunk at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/interpreter.c:804
jl_toplevel_eval_flex at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/toplevel.c:813
jl_parse_eval_all at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/ast.c:838
jl_load at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/toplevel.c:847
include at ./boot.jl:317 [inlined]
include_relative at ./loading.jl:1041
include at ./sysimg.jl:29
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
exec_options at ./client.jl:229
_start at ./client.jl:421
jl_apply_generic at /home/conda/feedstock_root/build_artifacts/julia_1539816087668/work/src/gf.c:2184
unknown function (ip: 0x401b61)
unknown function (ip: 0x40146b)
__libc_start_main at /lib64/libc.so.6 (unknown line)
unknown function (ip: 0x401504)
Allocations: 5453600 (Pool: 5452671; Big: 929); GC: 11
Illegal instruction (core dumped)

Environment Setup

I am using miniconda3 (conda 4.5.11) on CentOS Linux release 7.5.1804 (Core)

conda create -y -n julia_test
. activate julia_test
conda install -y libnetcdf
conda install -y -c conda-forge julia
julia -e 'using Pkg; Pkg.add("NetCDF")'

Versioninfo

Julia Version 1.0.1
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, haswell)

NcVar MethodError

I want to write out some radar variables to a NetCDF file with Julia V1.1.1while it produces the error below,
Running the netcdf example using dictionaries to replace the deprecated curly brackets with Julia V1.0.2 produces the error
Script

Write out the radar variables

function write_ncfile(filename,lat,lon,lev,times,varnames)
###Write to nc-file
println("Write to nc file ...")

ncvars = NcVar[]
xatts = Dict("long_name" => "x (longitude)", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
yatts = Dict("long_name" => "y (latitude)", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
zatts = Dict("long_name" => "z (eta)", "units" => "unitless", "missing_value" => -999.0, "_FillValue" => -999.0)
tatts = Dict("long_name" => "time (minutes)", "units" => "min", "missing_value" => -999.0, "_FillValue" => -999.0)
x_dim = NcDim("east_west",collect(1:length(lon[:,1,1])),xatts)
y_dim = NcDim("south_north",collect(1:length(lat[1,:,1])),yatts)
z_dim = NcDim("bottom_top",collect(1:length(lev[:,1])),zatts)
t_dim = NcDim("Time",collect(1:length(times)),tatts)

for varname in varnames
atts = Dict("long_name" => varname, "units" => "???", "missing_value" => -999.0, "_FillValue" => -999.0)
push!(ncvars,NcVar(varname,[x_dim,y_dim,z_dim,t_dim],atts,Float64))
end
atts = Dict("long_name" => "Latitude", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
push!(ncvars,NcVar("XLAT",[x_dim,y_dim,t_dim],atts,Float64))
atts = Dict("long_name" => "Longitude", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
push!(ncvars,NcVar("XLON",[x_dim,y_dim,t_dim],atts,Float64))
atts = Dict("long_name" => "Time", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
push!(ncvars,NcVar("XTIME",[t_dim],atts,Float64))
atts = Dict("long_name" => "Eta Levels", "units" => "deg", "missing_value" => -999.0, "_FillValue" => -999.0)
push!(ncvars,NcVar("ZNU",[z_dim,t_dim],atts,Float64))
nc = NetCDF.create(filename,ncvars)

NetCDF.putvar(nc,"REFL_10CM",refl)
NetCDF.putvar(nc,"ZDR",ZDR)
NetCDF.putvar(nc,"ZV",ZV)
NetCDF.putvar(nc,"ZH",ZH)
NetCDF.putvar(nc,"Rayleigh",DBZ)
NetCDF.putvar(nc,"Zdiff",Zdiff)
NetCDF.putvar(nc,"QRAIN",qrain)
NetCDF.putvar(nc,"QNRAIN",qnrain)
NetCDF.putvar(nc,"XLAT",lat)
NetCDF.putvar(nc,"XLON",lon)
NetCDF.putvar(nc,"XTIME",times)
NetCDF.putvar(nc,"ZNU",lev)

NetCDF.close(nc)
return 0
end

Result:
ERROR: LoadError: MethodError: no method matching NcVar(::String, ::Array{NcDim,1}, ::Dict{String,Any}, ::Type{Float64}) Closest candidates are: NcVar(::AbstractString, ::Union{Array{NcDim,1}, NcDim}; atts, t, compress, chunksize) at /home/.julia/packages/NetCDF/gaxgJ/src/NetCDF.jl:142
Stacktrace:
[1] write_ncfile(::String, ::Array{Float32,3}, ::Array{Float32,3}, ::Array{Float32,2}, ::Array{Float32,1}, ::Array{String,1}) at /home/julia111/test.jl:294
[2] top-level scope at none:0
[3] include at ./boot.jl:326 [inlined]
[4] include_relative(::Module, ::String) at ./loading.jl:1038
[5] include(::Module, ::String) at ./sysimg.jl:29
[6] exec_options(::Base.JLOptions) at ./client.jl:267
[7] _start() at ./client.jl:436
in expression starting at /home/julia111/test.jl:464

build failed in windows

When building NetCDF, julia was stuck for a long time. And got the message:

Pkg.build("NetCDF")
  Building Conda ─→ `C:\Users\kongdd\.julia\packages\Conda\kLXeC\deps\build.log`
  Building NetCDF → `C:\Users\kongdd\.julia\packages\NetCDF\wnKH9\deps\build.log`
ERROR: InterruptException:
Stacktrace:
 [1] poptaskref(::Base.InvasiveLinkedListSynchronized{Task}) at .\task.jl:564
 [2] wait() at .\task.jl:591
 [3] wait(::Base.GenericCondition{Base.AlwaysLockedST}) at .\condition.jl:104
 [4] stream_wait(::Base.Process, ::Base.GenericCondition{Base.AlwaysLockedST}) at .\stream                                                   .jl:47
 [5] wait at .\process.jl:956 [inlined]
 [6] success at .\process.jl:771 [inlined]
 [7] success(::Base.CmdRedirect) at .\process.jl:784
 [8] (::getfield(Pkg.Operations, Symbol("##45#48")){Bool,Cmd})(::IOStream) at C:\cygwin\ho                                                   me\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\b                                                   ackwards_compatible_isolation.jl:633
 [9] #open#312(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::t                                                   ypeof(open), ::getfield(Pkg.Operations, Symbol("##45#48")){Bool,Cmd}, ::String, ::Vararg{S                                                   tring,N} where N) at .\iostream.jl:375
 [10] open at .\iostream.jl:373 [inlined]
 [11] (::getfield(Pkg.Operations, Symbol("##44#47")){Pkg.Types.PackageSpec,Bool,String,Cmd                                                   })() at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\s                                                   tdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:632
 [12] #46 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\ju                                                   lia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:652 [inlined]
 [13] #32 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\ju                                                   lia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:568 [inlined]
 [14] withenv(::getfield(Pkg.Operations, Symbol("##32#39")){getfield(Pkg.Operations, Symbo                                                   l("##46#49")){getfield(Pkg.Operations, Symbol("##44#47")){Pkg.Types.PackageSpec,Bool,Strin                                                   g,Cmd}},Pkg.Types.Context}, ::Pair{String,String}, ::Vararg{Pair{String,B} where B,N} wher                                                   e N) at .\env.jl:161
 [15] (::getfield(Pkg.Operations, Symbol("##30#36")){Bool,getfield(Pkg.Operations, Symbol(                                                   "##46#49")){getfield(Pkg.Operations, Symbol("##44#47")){Pkg.Types.PackageSpec,Bool,String,                                                   Cmd}},Pkg.Types.Context,Pkg.Types.PackageSpec,Pkg.Types.Context,Set{Base.UUID},Set{Base.UU                                                   ID},Array{Pkg.Types.PackageSpec,1}})(::String) at C:\cygwin\home\Administrator\buildbot\wo                                                   rker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\backwards_compatible_isolatio                                                   n.jl:567
 [16] #mktempdir#15(::String, ::typeof(mktempdir), ::getfield(Pkg.Operations, Symbol("##30                                                   #36")){Bool,getfield(Pkg.Operations, Symbol("##46#49")){getfield(Pkg.Operations, Symbol("#                                                   #44#47")){Pkg.Types.PackageSpec,Bool,String,Cmd}},Pkg.Types.Context,Pkg.Types.PackageSpec,                                                   Pkg.Types.Context,Set{Base.UUID},Set{Base.UUID},Array{Pkg.Types.PackageSpec,1}}, ::String)                                                    at .\file.jl:584
 [17] mktempdir(::Function, ::String) at .\file.jl:582 (repeats 2 times)
 [18] #with_dependencies_loadable_at_toplevel#27(::Bool, ::typeof(Pkg.Operations.with_depe                                                   ndencies_loadable_at_toplevel), ::getfield(Pkg.Operations, Symbol("##46#49")){getfield(Pkg                                                   .Operations, Symbol("##44#47")){Pkg.Types.PackageSpec,Bool,String,Cmd}}, ::Pkg.Types.Conte                                                   xt, ::Pkg.Types.PackageSpec) at C:\cygwin\home\Administrator\buildbot\worker\package_win64                                                   \build\usr\share\julia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:516
 [19] (::getfield(Pkg.Operations, Symbol("#kw##with_dependencies_loadable_at_toplevel")))(                                                   ::NamedTuple{(:might_need_to_resolve,),Tuple{Bool}}, ::typeof(Pkg.Operations.with_dependen                                                   cies_loadable_at_toplevel), ::Function, ::Pkg.Types.Context, ::Pkg.Types.PackageSpec) at .                                                   \none:0
 [20] backwards_compat_for_build(::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, :                                                   :Bool, ::Bool, ::Int64) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\buil                                                   d\usr\share\julia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:650
 [21] #build_versions#87(::Bool, ::Bool, ::typeof(Pkg.Operations.build_versions), ::Pkg.Ty                                                   pes.Context, ::Array{Base.UUID,1}) at C:\cygwin\home\Administrator\buildbot\worker\package                                                   _win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\Operations.jl:795
 [22] #build_versions at .\none:0 [inlined]
 [23] build(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Bool) at C:\cygwin\ho                                                   me\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\O                                                   perations.jl:715
 [24] #build#73(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple                                                   {}}}, ::typeof(Pkg.API.build), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C                                                   :\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.                                                   2\Pkg\src\API.jl:399
 [25] build at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\                                                   julia\stdlib\v1.2\Pkg\src\API.jl:382 [inlined]
 [26] #build#72 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\sh                                                   are\julia\stdlib\v1.2\Pkg\src\API.jl:380 [inlined]
 [27] build at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\                                                   julia\stdlib\v1.2\Pkg\src\API.jl:380 [inlined]
 [28] #build#69 at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\sh                                                   are\julia\stdlib\v1.2\Pkg\src\API.jl:377 [inlined]
 [29] build(::String) at C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\                                                   usr\share\julia\stdlib\v1.2\Pkg\src\API.jl:377
 [30] top-level scope at REPL[5]:1

scale_factor, add_offset and _FillValue

In Python and Matlab/Octave, the NetCDF variables scale_factor and add_offset are applied when these attributes are defined, according to this simple rule:

variable2 = scale_factor * variable + add_offset

As far as I can see, this is also the default for the R package RNetCDF. Is it possible to add this behaviour to ncread and to getindex/setindex! functions? (I actually prefer the later API because indexing is much more natural).

In a similar vein, is it also possible to set values to NaN which are equal to _FillValue or maybe use DataArrays?

using NetCDF (enhancement manual)

For some people (like me) NetCDF.jl may be the first package they install. They will get an error because, when following the manual, they didn't actually load the package.

In README.md just after (or before) Quickstart add the command using NetCDF.

Maybe also in https://github.com/meggart/NetCDF.jl/blob/master/doc/NetCDF.md.

Julia NetCDF getting negative values from nc file

Hello!

I have been trying to read a NetCDF file with Julia, and this file has mostly positive values, however when I read the contents of a variable, the output is mostly negative. I have also tried reading the same file with R, Python and GRADS, and their output, which is positive, are the same. I am not sure if I am missing something.

The variables have the following dimensions: 23,376 time steps, longitude that range from 306.625 to 316 (X size = 75) and latitude that range from -26.625 to -19.625 (Y size = 48).

I tried both methods posted here in this git (high.jl and low.jl), and both give the same result.

Thank you in advance for any help!

What to do about threads in Julia 1.3

So Julia is getting a new threading model and I think packages are supposed to prepare becoming thread-safe. To my best knowledge, the NetCDF C library is not thread-safe and should not be called from several threads simultaneously, although there has been some discussion on the topic. https://www.unidata.ucar.edu/blogs/developer/entry/implementing-thread-safe-access-to

So I guess one would have to make sure that calls to any netcdf c function would be exclusive among threads by using locks from the Julia side. I think this will probably affect a lot of other wrapper packages around non-thread-safe libraries, so it might make sense to wait a bit and see solutions popping up. Just wanted to bring up the issue now, so people might reference existing solutions.

Can't install NetCDF Package at Julia 0.4.1 Windows 32-bit or 64-bit

in Julia i686-w64-mingw32

_ _ ()_ | A fresh approach to technical computing
() | () () | Documentation: http://docs.julialang.org
_ _ | | __ _ | Type "?help" for help.
| | | | | | |/ ` | |
| | || | | | (| | | Version 0.4.1 (2015-11-08 10:33 UTC)
/ |_'|||__'| | Official http://julialang.org/ release
|__/ | i686-w64-mingw32

I do this's
Pkg.init()

julia> Pkg.add("NetCDF")
INFO: Installing Formatting v0.1.4
INFO: Installing NetCDF v0.3.0
JULIA_PKGDIR=O:\Julia-0.4.1\bin..\share\julia\site
INFO: Building NetCDF
Fetching package metadata: ....
Solving package specifications: ...............

All requested packages already installed.

packages in environment at O:\Julia-0.4.1\share\julia\site\v0.4\Conda\deps\usr

libnetcdf 4.3.3.1 3
===============================[ ERROR: NetCDF ]================================

LoadError: Provider BinDeps.PackageManager failed to satisfy dependency libnetcd
f
while loading O:\Julia-0.4.1\share\julia\site\v0.4\NetCDF\deps\build.jl, in expr
ession starting on line 12

================================[ BUILD ERRORS ]================================

WARNING: NetCDF had build errors.

packages with build errors remain installed in O:\Julia-0.4.1\share\julia\sit
e\v0.4
build the package(s) and all dependencies with Pkg.build("NetCDF")
build a single package by running its deps/build.jl script

INFO: Package database updated
INFO: METADATA is out-of-date —— you may not have the latest version of NetCDF
INFO: Use Pkg.update() to get the latest versions of your packages

Writing values into unlimited dimension variable

If I create dimensions like

xdim = NcDim("x",nx,values=x)
ydim = NcDim("y",ny,values=y)
tdim = NcDim("t",0,unlimited=true)

and the variables as

uvar = NcVar("u",[xdim,ydim,tdim],t=Float32)
tvar = NcVar("t",tdim,t=Int64)

I can write the u variable for a given timestep i with

ncu = NetCDF.create("u.nc",[uvar,tvar],mode=NC_NETCDF4)
NetCDF.putvar(ncu,"u",Float32.(u),start=[1,1,i],count=[-1,-1,1])

However, no matter how I try to write the values into the unlimited dimension variable "t" (either all at once

NetCDF.putvar(ncu,"t",t)

where t is an integer array, or consequitively everytime I also write "u")

NetCDF.putvar(ncu,"t",[i],start=[i],count=[1])

the values do not get written into the nc file (but also no error is thrown)

error showing NcVar with no data

This is with NetCDF.jl master on Julia 1.4-rc2

julia> using NetCDF

julia> d = NcDim("Dim1", 1)
NcDim(-1, -1, -1, "Dim1", 0x0000000000000001, Any[], Dict{Any,Any}(), false)

julia> v = NcVar("v1", [d])
Disk Array with size 1


julia> @show v
ERROR: NetCDF error code -33:
        NetCDF: Not a valid ID
Stacktrace:
 [1] check at C:\Users\visser_mn\.julia\packages\NetCDF\SFmQh\src\netcdf_helpers.jl:22 [inlined]
 [2] nc_get_vara_double(::Int32, ::Int32, ::Array{UInt64,1}, ::Array{UInt64,1}, ::Array{Float64,1}) at C:\Users\visser_mn\.julia\packages\NetCDF\SFmQh\src\netcdf_c.jl:2486
 [3] nc_get_vara_x! at C:\Users\visser_mn\.julia\packages\NetCDF\SFmQh\src\NetCDF.jl:465 [inlined]
 [4] readvar!(::NcVar{Float64,1,6}, ::Array{Float64,1}; start::Array{Int64,1}, count::Array{Int64,1}) at C:\Users\visser_mn\.julia\packages\NetCDF\SFmQh\src\NetCDF.jl:343
 [5] readblock!(::NcVar{Float64,1,6}, ::Array{Float64,1}, ::UnitRange{Int64}) at C:\Users\visser_mn\.julia\packages\NetCDF\SFmQh\src\NetCDF.jl:218
 [6] getindex_disk at C:\Users\visser_mn\.julia\packages\DiskArrays\RJ0eT\src\DiskArrays.jl:36 [inlined]
 [7] getindex at C:\Users\visser_mn\.julia\packages\DiskArrays\RJ0eT\src\DiskArrays.jl:158 [inlined]
 [8] isassigned(::NcVar{Float64,1,6}, ::Int64) at .\abstractarray.jl:404
 [9] show_delim_array(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::NcVar{Float64,1,6}, ::Char, ::String, ::Char, ::Bool, ::Int64, ::Int64) at .\show.jl:712
 [10] show_delim_array at .\show.jl:705 [inlined]
 [11] show_vector(::Base.GenericIOBuffer{Array{UInt8,1}}, ::NcVar{Float64,1,6}, ::Char, ::Char) at .\arrayshow.jl:458
 [12] show_vector at .\arrayshow.jl:447 [inlined]
 [13] show(::Base.GenericIOBuffer{Array{UInt8,1}}, ::NcVar{Float64,1,6}) at .\arrayshow.jl:420
 [14] sprint(::Function, ::NcVar{Float64,1,6}; context::Nothing, sizehint::Int64) at .\strings\io.jl:105
 [15] #repr#339 at .\strings\io.jl:227 [inlined]
 [16] repr(::NcVar{Float64,1,6}) at .\strings\io.jl:227
 [17] top-level scope at show.jl:613

Reading data along chunked dimension does not scale linearly with amount of data

Super cool work on integrating DiskArrays.jl with NetCDF.jl! Looking forward to ditching xarray in favor of a pure Julia solution.

@visr helped me get up and running but we noticed that grabbing 2x as much data seems to take ~4x longer whereas I expected it to scale linearly. I am unfortunately interested in grabbing data along the dimension with chunk size 1...

julia> using NetCDF

julia> ds = NetCDF.open("/home/alir/cnhlab004/bsose_i122/bsose_i122_2013to2017_1day_Theta.nc", "THETA")
Disk Array with size 2160 x 588 x 52 x 1826

julia> NetCDF.getchunksize(ds)
(2160, 588, 19, 1)

julia> @time ds[100, 200, :, 300]
  0.012066 seconds (48 allocations: 2.500 KiB)

julia> @time ds[100, 200, :, 320:330]
  0.010111 seconds (55 allocations: 4.750 KiB)

julia> @time ds[100, 200, :, 300:400]
  5.256234 seconds (56 allocations: 23.016 KiB)

julia> @time ds[100, 200, :, 600:800]
 19.074392 seconds (56 allocations: 43.328 KiB)

Method Error with iscontiguous when nccreate is called

Running the netcdf example using dictionaries to replace the deprecated curly brackets with Julia V1.0.2 produces the error
ERROR: MethodError: no method matching iscontiguous(::UnitRange{Int64})
This error occurs wit h Julia 1.0.2 on Rhel7, MacOS Mojave, and Windows 10 OS's

Script:

using NetCDF
filename   = "myfile.nc"
varname    = "var1"
attribs = Dict("units" => "mm/d", "data_min" => 0.0, "data_max" => 87.0)
nccreate(filename, varname, "x1", 11:20, "t", 20, Dict("units"=>"s"), atts=attribs)

Result:

ERROR: MethodError: no method matching iscontiguous(::UnitRange{Int64})
Closest candidates are:
  iscontiguous(::SubArray) at subarray.jl:286
  iscontiguous(::Type{#s57} where #s57<:(SubArray{T,N,P,I,true} where I<:Tuple{AbstractUnitRange,Vararg{Any,N} where N} where P where N where T)) at subarray.jl:288
  iscontiguous(::Type{#s57} where #s57<:SubArray) at subarray.jl:287
Stacktrace:
 [1] #putvar#31(::Array{Int64,1}, ::Array{Int64,1}, ::Function, ::NcVar{Int64,1,10}, ::UnitRange{Int64}) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:437
 [2] (::getfield(NetCDF, Symbol("#kw##putvar")))(::NamedTuple{(:start, :count),Tuple{Array{Int64,1},Array{Int64,1}}}, ::typeof(NetCDF.putvar), ::NcVar{Int64,1,10}, ::UnitRange{Int64}) at ./none:0
 [3] #putvar#30(::Array{Int64,1}, ::Array{Int64,1}, ::Function, ::NcFile, ::String, ::UnitRange{Int64}) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:421
 [4] putvar(::NcFile, ::String, ::UnitRange{Int64}) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:421
 [5] #create#38(::Dict{Any,Any}, ::UInt16, ::Function, ::String, ::Array{NcVar,1}) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:634
 [6] #create at ./none:0 [inlined]
 [7] #create#39(::Dict{Any,Any}, ::UInt16, ::Function, ::String, ::NcVar{Float64,2,6}) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:641
 [8] (::getfield(NetCDF, Symbol("#kw##create")))(::NamedTuple{(:gatts, :mode),Tuple{Dict{Any,Any},UInt16}}, ::typeof(NetCDF.create), ::String, ::NcVar{Float64,2,6}) at ./none:0
 [9] #nccreate#51(::Dict{String,Any}, ::Dict{Any,Any}, ::Int64, ::Int64, ::UInt16, ::Tuple{Int64}, ::Function, ::String, ::String, ::String, ::Vararg{Any,N} where N) at /home/Jessica.Liptak/.julia/packages/NetCDF/61OwG/src/NetCDF.jl:920
 [10] (::getfield(NetCDF, Symbol("#kw##nccreate")))(::NamedTuple{(:atts,),Tuple{Dict{String,Any}}}, ::typeof(nccreate), ::String, ::String, ::String, ::Vararg{Any,N} where N) at ./none:0
 [11] top-level scope at none:0

Nondeterministic ncread in Julia0.5

I'm trying to read an ExodusII file with code I'd written for Julia0.4.x using the NetCDF package. Using ncread(filename,"coor_names") gives garbage values

I can reproduce the bug with the following.

$ wget https://github.com/idaholab/moose/raw/devel/examples/ex01_inputfile/mug.e
$ julia
julia> f = "mug.e"
julia> using NetCDF
julia> A = ncread(f,"coor_names")
julia> sum([ncread(f,"coor_names")==A for i in 1:1000])

Downloaded julia binary from http://julialang.org/downloads/ for linux 64bit. Was no issue in previous version of julia (can't remember exactly which).

Iterate protocol on Julia 0.7/1.0

Hello, I get the following error on Julia 0.7/1.0 when using NetCDF.jl

┌ Warning: The start/next/done iteration protocol is deprecated. Use `iterate` instead.
│   caller = firsti(::UnitRange{Int64}, ::UInt64) at NetCDF.jl:299
└ @ NetCDF ~/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:299
ERROR: LoadError: LoadError: MethodError: no method matching -(::Base.LegacyIterationCompat{UnitRange{Int64},Int64,Int64}, ::Int64)
Closest candidates are:
  -(::Complex{Bool}, ::Real) at complex.jl:298
  -(::Missing, ::Number) at missing.jl:93
  -(::Base.CoreLogging.LogLevel, ::Integer) at logging.jl:107
  ...
Stacktrace:
 [1] firsti(::UnitRange{Int64}, ::UInt64) at /home/proy/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:299
 [2] macro expansion at /home/proy/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:319 [inlined]
 [3] readvar!(::NetCDF.NcVar{Float32,3,5}, ::Array{Float32,3}, ::UnitRange{Int64}, ::UnitRange{Int64}, ::UnitRange{Int64}) at /home/proy/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:312
 [4] readvar(::NetCDF.NcVar{Float32,3,5}, ::UnitRange{Int64}, ::UnitRange{Int64}, ::Vararg{UnitRange{Int64},N} where N) at /home/proy/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:275
 [5] getindex at /home/proy/.julia/packages/NetCDF/KF3pG/src/NetCDF.jl:159 [inlined]

The faulty line:

firsti(r::UnitRange,l::Integer) = start(r)-1firsti(r::UnitRange,l::Integer) = start(r)-1

Not sure yet about the solution, I haven't had the chance to look it up (still trying to understand Pkg3!).

Regards

currentNcFiles memory usage

I just tracked down a large memory leak to the existence of currentNcFiles: while the fix is easy (calling ncclose after reading from each file), the overall approach of keeping a global reference to each opened file is a bit strange. Any reason why this is done?

ncinfo used to be a struct, now it is Nothing

In latest NetCDF package in Julia 1.2 the return type of the ncinfo() function has changed. Previously it was a struct with fields for global attributes, variable names, etc. Now it is of type Nothing.

Was this change intended?

Do you know the revision number where this was changed?

reading char-array using the low-level interface

I have some troubles reading a character arrays using the low-level interface. Here is what I tried:

using NetCDF
fname = "argo-profiles-6900659.nc";
nc = NetCDF.open(fname);
julia> temp_qc = nc["TEMP_QC"][:];
ERROR: NetCDF error code -56:
	NetCDF: Attempt to convert between text & numbers
 in check at /home/abarth/.julia/v0.5/NetCDF/src/netcdf_helpers.jl:22 [inlined]
 in nc_get_var1_uchar(::Int32, ::Int32, ::Array{UInt64,1}, ::Array{UInt8,1}) at /home/abarth/.julia/v0.5/NetCDF/src/netcdf_c.jl:789
 in nc_get_var1_x at /home/abarth/.julia/v0.5/NetCDF/src/NetCDF.jl:267 [inlined]
 in macro expansion at /home/abarth/.julia/v0.5/NetCDF/src/NetCDF.jl:231 [inlined]
 in readvar(::NetCDF.NcVar{UInt8,2,2}, ::Int64, ::Int64) at /home/abarth/.julia/v0.5/NetCDF/src/NetCDF.jl:227
 in _unsafe_getindex at ./reshapedarray.jl:128 [inlined]
 in getindex at ./reshapedarray.jl:119 [inlined]
 in macro expansion at ./multidimensional.jl:352 [inlined]
 in macro expansion at ./cartesian.jl:64 [inlined]
 in macro expansion at ./multidimensional.jl:350 [inlined]
 in _unsafe_getindex! at ./multidimensional.jl:342 [inlined]
 in macro expansion at ./multidimensional.jl:300 [inlined]
 in _unsafe_getindex(::Base.LinearSlow, ::Base.ReshapedArray{UInt8,1,NetCDF.NcVar{UInt8,2,2},Tuple{Base.MultiplicativeInverses.SignedMultiplicativeInverse{Int64}}}, ::Colon) at ./multidimensional.jl:293
 in getindex(::NetCDF.NcVar{UInt8,2,2}, ::Colon) at ./abstractarray.jl:760

Using the ncread command works however fine:

qc = ncread(fname,"TEMP_QC");

Just the type is a bit surprising (Array{UInt8,2}). With a convert(Array{Char,2},qc), one get the expected type.

Maybe this work-around is useful for other users of the package.

The NetCDF file to reproduce this issue:
http://modb.oce.ulg.ac.be/mediawiki/upload/Alex/argo-profiles-6900659.nc

I am using Julia 0.5.2, the NetCDF package 0.4.1 and the NetCDF library of version 4.1.3 on Linux.

Properly support unlimited dimensions

It is propably not too much work, I personally just don't use unlimited dimensions so I kind of ignored them so far. This needs to change...

Error when adding a variable to a closed file

I have an old netcdf file, to which I want to add a new variable, like follows:

nccreate(file, "time_array", "ymdh", 4, "dim_time")

This raises the following error:

       nccreate(file, "time_array", "ymdh", 4, "dim_time")
reopening file in WRITE mode
ERROR: MethodError: no method matching open(::String, ::UInt16)
Closest candidates are:
  open(::AbstractString; mode, readdimvar) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:654
  open(::AbstractString, ::AbstractString; mode, readdimvar) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:638
Stacktrace:
 [1] #nccreate#46(::Dict{Any,Any}, ::Dict{Any,Any}, ::Int64, ::Int64, ::UInt16, ::Tuple{Int64}, ::Function, ::String, ::String, ::String, ::Vararg{Any,N} where N) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:858
 [2] nccreate(::String, ::String, ::String, ::Vararg{Any,N} where N) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:847

This error disappears if I change the open file statement on line 858 from:

open(fil,NC_WRITE)

to:

open(fil,mode=NC_WRITE)

Did I do something wrong from the beginning, or should there be a keyword in the open statement?

Thanks for any help.
Jan

ReadOnlyMemoryError on calling ncclose()

Can't quite figure this out, seen on Win64:

julia> Pkg.test("NetCDF")
INFO: Testing NetCDF
ERROR: LoadError: LoadError: ReadOnlyMemoryError()
Stacktrace:
 [1] nc_close(::Int32) at C:\Users\visser_mn\.julia\v0.6\NetCDF\src\netcdf_c.jl:541
 [2] close(::NetCDF.NcFile) at C:\Users\visser_mn\.julia\v0.6\NetCDF\src\NetCDF.jl:533
 [3] ncclose(::String) at C:\Users\visser_mn\.julia\v0.6\NetCDF\src\NetCDF.jl:441
 [4] ncclose() at C:\Users\visser_mn\.julia\v0.6\NetCDF\src\NetCDF.jl:449
 [5] include_from_node1(::String) at .\loading.jl:569
 [6] include(::String) at .\sysimg.jl:14
 [7] include_from_node1(::String) at .\loading.jl:569
 [8] include(::String) at .\sysimg.jl:14
 [9] process_options(::Base.JLOptions) at .\client.jl:305
 [10] _start() at .\client.jl:371
while loading C:\Users\visser_mn\.julia\v0.6\NetCDF\test\high.jl, in expression starting on line 71
while loading C:\Users\visser_mn\.julia\v0.6\NetCDF\test\runtests.jl, in expression starting on line 13

Sometimes it passes test\high.jl and crashes the next time it is called in test\chunks.jl

No NetCDF errors detected with new C interface

@visr
I just realized that since the upgrade no errors are thrown anymore when ccalls return an error code, so far the interface relied on this to catch errors like trying to open a non-existing file. This is probably my bad for not testing if errors are thrown if a file does not exist. How complicated would it be to change the Clang wrapper so that instead of

function nc_open(path,mode::Integer,ncidp)
    ccall((:nc_open,libnetcdf),Cint,(Ptr{UInt8},Cint,Ptr{Cint}),path,mode,ncidp)
end

something like

function nc_open(path,mode::Integer,ncidp)
    ret = ccall((:nc_open,libnetcdf),Cint,(Ptr{UInt8},Cint,Ptr{Cint}),path,mode,ncidp)
    ret == NC_NOERR || error("NetCDF library error thrown: Error Code $ret")
end

would be generated. If it is too complicated, I would do the checks on the nc_helpers.jl side. Just let me know....

Error when getting attribute

I get the following error when trying to load an attribute of a netcdf that I haven´t read anything from before:

julia> ncgetatt("tair_1km.nc", "dim_space", "id")
ERROR: MethodError: no method matching open(::String, ::UInt16)
Closest candidates are:
  open(::AbstractString; mode, readdimvar) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:654
  open(::AbstractString, ::AbstractString; mode, readdimvar) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:638
Stacktrace:
 [1] ncgetatt(::String, ::String, ::String) at /home/jmg/.julia/v0.6/NetCDF/src/NetCDF.jl:792

If I have read some information (for example done an ncinfo) from the netcdf before I try to load the attribute, everything works fine.

Slicing of NcVar does not work

Following code throws the error UndefVarError: start not defined:

    ds = NetCDF.open(file_name);
    a = ds[varname][1:2, 1]    # given that length(ds[varname].dim == 2) and size(ds[varname])[1] >= 2

However, reading the whole array and then slicing does work (obviously)

    ds = NetCDF.open(file_name);
    a = ds[varname][:, 1][1:2, :]

NetCDF.open(... , readdimvar = true) does not work.

julia> import NetCDF
julia> tmp = NetCDF.open(filename,readdimvar = true)
ERROR: UndefVarError: _readdimvars not defined

but tests pass and readdimvar=false works fine.

ncinfo(filename) showing no data

Hello!
Trying to read a NetCDF4 file, using NetCDF.jl, and although I can read it in Python with netCDF4 package (so I conclude the file is ok), in Julia I can’t even get general information with ncinfo(filename), as my print shows. Any guess on the problem?

(any file from here can be used for testing)

my environment:

macOS Catalina (10.15.5)
Julia v1.4.2
Atom v0.12.15
Juno v0.8.2
NetCDF v0.10.2

HDF5 error

With v0.10.3 on Julia 1.5.2 I'm currently producing this error

using NetCDF

x = Array{Float32}(1:10)
y = Array{Float32}(1:10)

xdim = NcDim("x",length(x),values=x)
ydim = NcDim("y",length(y),values=y)
tdim = NcDim("t",0,unlimited=true)

var = NcVar("u",[xdim,ydim,tdim],t=Float32)
tvar = NcVar("t",tdim,t=Int32)

nc = NetCDF.create("u.nc",[var,tvar],mode=NC_NETCDF4)
NetCDF.putatt(nc,"u",Dict("units"=>"m/s","long_name"=>"velocity"))

for i in 1:10
    NetCDF.putvar(nc,"u",rand(Float32,length(x),length(y)),start=[1,1,i],count=[-1,-1,1])
end

The file exists, and also the variable nc in the same repl looks fine

julia> nc

##### NetCDF File #####

u.nc

##### Dimensions #####

Name                                                Length                    
--------------------------------------------------------------------------------
t                                                   UNLIMITED (10 currently)  
x                                                   10                        
y                                                   10                        

##### Variables #####

Name                            Type            Dimensions                      
--------------------------------------------------------------------------------
t                               INT             t                               
x                               FLOAT           x                               
u                               FLOAT           x y t                           
y                               FLOAT           y                               

##### Attributes #####

Variable            Name                Value                                   
--------------------------------------------------------------------------------
u                   units               m/s                                     
u                   long_name           velocity

However, if I try to NetCDF.open the file again I get

julia> u = NetCDF.open("u.nc")
ERROR: NetCDF error code -101:
	NetCDF: HDF error

And similarly with ncdump, which returns

milan@milanair ~ % ncdump -h u.nc
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 4371963328:
  #000: H5F.c line 509 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: H5Fint.c line 1400 in H5F__open(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #002: H5Fint.c line 1615 in H5F_open(): unable to lock the file
    major: File accessibilty
    minor: Unable to open file
  #003: H5FD.c line 1640 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Can't update object
  #004: H5FDsec2.c line 941 in H5FD_sec2_lock(): unable to lock file, errno = 35, error message = 'Resource temporarily unavailable'
    major: File accessibilty
    minor: Bad file ID accessed
ncdump: u.nc: u.nc: NetCDF: HDF error

Can anyone reproduce this error?

off by one error in nc_get_att?

I'm finding the last character of attributes is being cut off, and errors are raised if the attribute is a 0-length string. I think this is an off by one error, since the following seems to fix it.

line 155 of netcdf_helpers is currently:
valsa=Array(nctype2jltype[attype],attlen)
change this to:
valsa=Array(nctype2jltype[attype],attlen+1)

ncsync and ncputatt issues after upgrade from 0.3 to 0.4

I get one warning and an error following update from 0.3 to 0.4.

The warning:
WARNING: deprecated syntax "{a=>b, ...}" at /home/emason/julia/make_chl_loess.jl:97. Use "Dict{Any,Any}(a=>b, ...)" instead.
relates to this line in my code:
ncputatt(filename, "loess_span", {"loess_span" => loess_span})

The error:
while loading /home/emason/julia/make_chl_loess.jl, in expression starting on line 121 ERROR: LoadError: UndefVarError: _nc_sync_c not defined in ncsync at /home/emason/.julia/v0.4/NetCDF/src/NetCDF.jl:369
relates to this:

ncsync()

Thanks

Document unlimited dimensions

Creation and writing to unlimited dimensions is still undocumented.

Trying to use Conda as a binary provider under Linux and Windows

It would be cool if we could use Conda.jl (many thanks to @Luthaf) to provide binaries under Windows and Linux without root access. What works so far is

using Conda
Conda.add("libnetcdf")

this will install the Conda libnetcdf package and put some netcdf.ddl/so in .julia//v0.4/Conda/deps/usr/pkgs/libnetcdf-4.3.3.1-1/lib under Linux or /v0.4/Conda/deps/usr/pkgs/libnetcdf-4.3.3.1-1/bin when I am on Windows.

However Libdl.find_library does not automatically detect the files in these paths.
@Luthaf could you help/explain how we can make this work?

Error tagging new release

The tag name "0.7.2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @meggart