Giter VIP home page Giter VIP logo

Comments (2)

bodgit avatar bodgit commented on May 23, 2024

Try something like this instead:

func extractFile(file *sevenzip.File, target string) error {
	_f, err := os.OpenFile(target, os.O_CREATE, file.Mode())
	if err != nil {
		return err
	}
	defer _f.Close()

	fReader, err := file.Open()
	if err != nil {
		return err
	}
	defer fReader.Close()

	_, err = io.Copy(_f, fReader)

        return err
}

func extract7zArchive(archive string, path string, password string) error {
	reader, err := sevenz.OpenReaderWithPassword(archive, password)
	if err != nil {
		return err
	}
	defer reader.Close()

	for _, f := range reader.File {
		target := PathAppend(path, f.Name)
		// seems no flag to check if f is directory, but it will end with / if is directory
		if IsStrEndWith(f.Name, "/", true) {
			if err := os.MkdirAll(target, f.Mode()); err != nil {
				return err
			}
		} else {
                	pre := time.Now().UnixMilli()
                	if err = extractFile(f, target); err != nil {
                                return err
                        }
                	LogInfo("%d ms cost open %s", time.Now().UnixMilli()-pre, f.Name)
		}
	}

	return nil
}

I've not compile-tested it but you should be able to get the idea.

The point is to close the io.ReadCloser as soon as you finished extracting a file before extracting the next one. The way your code is currently structured means that all of the io.ReadClosers are deferred until extract7zArchive() completes before closing all of them.

Internally, to read file n in an archive, you have to start at the beginning of the stream and read (and discard) the data for files 0 to n-1, so for increasing values of n it will get slower and slower as you have to read and discard more and more.

There's an optimisation in my library that will reuse the stream reader but that only works if you call Close() on the file reader before opening the next one.

You're also doing the same with your filehandles returned from os.OpenFile() so you can potentially run out of filehandles available to your process when dealing with very large archives.

from sevenzip.

hiroyukki avatar hiroyukki commented on May 23, 2024

Thanks a lot, I close reader before open next file item, it works now

from sevenzip.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.