Giter VIP home page Giter VIP logo

mem's Introduction

Go Memory Management

Analysing

  1. Benchmark (component memory analysing of heap allocations, memory and cpu usage)
  • Run benchmarks with go test -bench=..
  • To see heap allocations use:
    • b.ReportAllocs() function;
    • benchmem flag: go test -bench=. -benchmem.
  • For memory profile use:
    • -memprofile flag: go test -bench=. -memprofile mem.out;
    • add heap profile server with file writting (see Profiler)
  • For cpu profile use:
    • -cpuprofile flag: go test -bench=. -cpuprofile cpu.out;
    • add cpu profile server with file writting (see Profiler)
  1. Runtime Statistics (detailed memory stats to look for possible memory leaks)
  • Use runtime.ReadMemStats(&r), where r is var r runtime.MemStats (see Runtime Statistics)
  • Investigate runtime/mstats.go fir additional information about stats fields.
  1. Escape Analysis (identifies allocation on heap, function complexity and inlining)
  • Use go build -gcflags="-m" . to enable escape analysis logs.
  • Use additional -m: -gcflags="-m -m" to have higher analisys mode.
  1. GC Trace
  • Use export GODEBUG=gctrace=1 and run app, or GODEBUG=gctrace=1 go run main.go (see GC Trace)

Development

  1. Avoid slices (string, []byte)
  2. Use StringBuffer/StringBuilder for string concatenations.
  3. Avoid string as map key type, it is better to use int (~44% less memory usage).
  4. Use pointers for big arrays.
  5. Use sync.Pool for data that can be reused.
  6. Align struct fields (use libs e.g. https://github.com/essentialkaos/aligo).
  7. Use runtime/debug.SetMemoryLimit or GOMEMLIMIT env (Go 1.19, https://tip.golang.org/doc/go1.19).

Profiler

  1. Memory Profiler (with lib)
import "github.com/pkg/profile"

defer profile.Start(profile.MemProfile, profile.ProfilePath(".")).Stop()
  1. CPU Profiler (with lib)
import "github.com/pkg/profile"

defer profile.Start(profile.CPUProfile, profile.ProfilePath(".")).Stop()

Runtime Statistics

package main

import (
	"fmt"
	"runtime"
	"time"
)

func main() {
	go func() {
		for {
			var r runtime.MemStats
			runtime.ReadMemStats(&r)
			fmt.Println("\nTime: ", time.Now())
			fmt.Println("Runtime MemStats Sys: ", r.Sys) // 
			fmt.Println("Runtime Heap Allocation: ", r.HeapAlloc)
			fmt.Println("Runtime Heap Idle: ", r.HeapIdle)
			fmt.Println("Runtime Heap In Use: ", r.HeapInuse)
			fmt.Println("Runtime Heap HeapObjects: ", r.HeapObjects)
			fmt.Println("Runtime Heap Released: ", r.HeapReleased)
			time.Sleep(1 * time.Second)
		}
	}()

	time.Sleep(1 * time.Second)
}

GC Trace

gc 6 @48.155s 15%: 0.093+12360+0.32 ms clock,
0.18+7720/21356/3615+0.65 ms cpu, 11039->13278->6876 MB, 14183 MB goal, 8 P
  • @48.155s since program start
  • 15% of time spent in GC since program start
  • 0.093+12360+0.32 ms clock stop-the-world (STW) sweep termination + concurrent mark and scan + and STW mark termination
  • 0.18+7720/21356/3615+0.65 ms cpu (GC performed in line with allocation), background GC time, and idle GC time:
    • 0.18 sweepTermCpu,
    • 7720 gcController.assistTime,
    • 21356 gcController.dedicatedMarkTime + gcController.fractionalMarkTime,
    • 3615 gcController.idleMarkTime,
    • 0.65 markTermCpu
  • 11039->13278->6876 MB heap size at GC start, at GC end, and live heap 8 P number of processors used

Resources

  1. https://www.ardanlabs.com/blog/2018/12/garbage-collection-in-go-part1-semantics.html
  2. https://tip.golang.org/doc/go1.19

mem's People

Contributors

ykholm avatar

Stargazers

 avatar

Watchers

 avatar

Forkers

alizufan

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.