Giter VIP home page Giter VIP logo

Comments (8)

JalonSolov avatar JalonSolov commented on July 29, 2024 1

The main difference is that Java has a runtime, so extra checks like this can be done at runtime. If you want this with V, you will need to do the checks yourself, since V doesn't have a runtime.

If you really need to work with huge arrays like this, it would be best to declare a struct, with the @[heap] attribute on it to ensure it's never created on the stack. Put the array in the struct and always access it from there.

Otherwise, you can try increasing values with the V command line option -thread-stack-size <number of bytes> when you build/run your code, until you give a large enough value to make it work.

from v.

tzSharing avatar tzSharing commented on July 29, 2024

The main difference is that Java has a runtime, so extra checks like this can be done at runtime. If you want this with V, you will need to do the checks yourself, since V doesn't have a runtime.

If you really need to work with huge arrays like this, it would be best to declare a struct, with the @[heap] attribute on it to ensure it's never created on the stack. Put the array in the struct and always access it from there.

Otherwise, you can try increasing values with the V command line option -thread-stack-size <number of bytes> when you build/run your code, until you give a large enough value to make it work.

A brief description of my usage scenario at that time may help to understand. I tried to write a file transfer tool, using the API about network IO in the v standard library. I have implemented the file transfer function, and tested the transfer of files from several Mb to hundreds of Mb in size. Everything is normal. But when I tested a movie file transfer of about 2Gb in size, I waited about 30 minutes and still did not complete the transfer. I realized that there may be a problem, because using the operating system's Ctrl + V only takes about 30 seconds. At first I thought my code was not efficient, but after several hours of investigation, I found that the cause was the thread blocking caused by the addition of element code to the v array.
In addition, I also made a new discovery:
Extract part of the code:

...
file_path := 'C:\\Users\\TZ\\Desktop\\星际穿越.rmvb'
mut file := os.open( file_path ) or { eprintln( err.str() ) return }
defer { file.close() }
bytes := file.read_bytes( int( os.file_size( file_path ) ) )  //Module: os  method signature:`pub fn (f &File) read_bytes(size int) []u8`
println('file bytes len=${bytes.len}') //file bytes len=1911250979
...

The array length of the final printout is 1911250979, which obviously exceeds 1073741824. pub fn (f & File) read_bytes (size int) [] u8 in Module: os completely reads all the data and put a u8 [], and there are no blocking threads or out of memory or other problems. Prove that the array length is allowed to exceed 1073741824. As for the question I raised, it is most likely due to a bug in the implementation of the add element method (arr < < element) of the array.

from v.

kbkpbot avatar kbkpbot commented on July 29, 2024

It seems that the array length can't exceed 1073741824, #17958

from v.

spytheman avatar spytheman commented on July 29, 2024

Yes, it is a known problem, caused by V using 32 bit integers for the .len and .cap fields of its dynamic arrays, and the default strategy, for growing a dynamic array (doubling the allocated memory, when the capacity is reached).

from v.

spytheman avatar spytheman commented on July 29, 2024

In the read_bytes's case, the resulting array's capacity is known beforehand (by the file size), thus it is allocated right away, without needing to reallocate anything after that, and so the array can reach up to 2^31 - 1 = 2147483647 bytes in that way, without overflowing its .cap and .len .

from v.

spytheman avatar spytheman commented on July 29, 2024

As for the behavior, it should indeed better panic in that case, with a stacktrace, instead of looping indefinitely due to the overflow of the .cap field.
I have a local patch to do it, but I have to run more tests before making a PR:
image

from v.

medvednikov avatar medvednikov commented on July 29, 2024

I'm making int 64 bit on 64 bit systems right now. So this issue will be fixed soon.

from v.

tzSharing avatar tzSharing commented on July 29, 2024

Okay, I see.

from v.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.