Common and useful utils for the Go project development.
- Only rely on the Go standard package
- Functions or lightweight packages
- Non-business related general tools
- BitSet A bit set
- Calendar Chinese Lunar Calendar, Solar Calendar and cron time rules
- CoarseTime Current time truncated to the nearest 100ms
- Errors Improved errors package.
- Graceful Shutdown or reboot current process gracefully.
- GoPool Goroutines' pool
- ResPool Resources' pool
- Workshop Non-blocking asynchronous multiplex resource pool
- Password Check password
- Various Various small functions
A bit set.
-
import it
"github.com/luoyeaichifan/goutil/bitset"
-
New creates a bit set object.
func New(init ...byte) *BitSet
-
NewFromHex creates a bit set object from hex string.
func NewFromHex(s string) (*BitSet, error)
-
Not returns ^b.
func (b *BitSet) Not() *BitSet
-
And returns all the "AND" bit sets.
Notes:
If the bitSets are empty, returns b.func (b *BitSet) And(bitSets ...*BitSet) *BitSet
-
Or returns all the "OR" bit sets.
Notes:
If the bitSets are empty, returns b.func (b *BitSet) Or(bitSet ...*BitSet) *BitSet
-
Xor returns all the "XOR" bit sets.
Notes:
If the bitSets are empty, returns b.func (b *BitSet) Xor(bitSet ...*BitSet) *BitSet
-
AndNot returns all the "&^" bit sets.
Notes:
If the bitSets are empty, returns b.func (b *BitSet) AndNot(bitSet ...*BitSet) *BitSet
-
Set sets the bit bool value on the specified offset, and returns the value of before setting.
Notes:
0 means the 1st bit, -1 means the bottom 1th bit, -2 means the bottom 2th bit and so on;
If offset>=len(b.set), automatically grow the bit set;
If the bit offset is out of the left range, returns error.func (b *BitSet) Set(offset int, value bool) (bool, error)
-
Get gets the bit bool value on the specified offset.
Notes:
0 means the 1st bit, -1 means the bottom 1th bit, -2 means the bottom 2th bit and so on;
If offset>=len(b.set), returns false.func (b *BitSet) Get(offset int) bool
-
Range calls f sequentially for each bit present in the bit set.
If f returns false, range stops the iteration.func (b *BitSet) Range(f func(offset int, truth bool) bool)
-
Count counts the amount of bit set to 1 within the specified range of the bit set.
Notes:
0 means the 1st bit, -1 means the bottom 1th bit, -2 means the bottom 2th bit and so on.func (b *BitSet) Count(start, end int) int
-
Clear clears the bit set.
func (b *BitSet) Clear()
-
Size returns the bits size.
func (b *BitSet) Size() int
-
Bytes returns the bit set copy bytes.
func (b *BitSet) Bytes() []byte
-
Binary returns the bit set by binary type.
Notes:
Paramter sep is the separator between chars.func (b *BitSet) Binary(sep string) string
-
String returns the bit set by hex type.
func (b *BitSet) String() string
-
Sub returns the bit subset within the specified range of the bit set.
Notes:
0 means the 1st bit, -1 means the bottom 1th bit, -2 means the bottom 2th bit and so on.func (b *BitSet) Sub(start, end int) *BitSet
Chinese Lunar Calendar, Solar Calendar and cron time rules.
-
import it
"github.com/luoyeaichifan/goutil/calendar"
The current time truncated to the nearest second.
-
import it
"github.com/luoyeaichifan/goutil/coarsetime"
-
FloorTimeNow returns the current time from the range (now-100ms,now]. This is a faster alternative to time.Now().
func FloorTimeNow() time.Time
-
CeilingTimeNow returns the current time from the range [now,now+100ms). This is a faster alternative to time.Now().
func CeilingTimeNow() time.Time
Errors is improved errors package.
-
import it
"github.com/luoyeaichifan/goutil/errors"
-
New returns an error that formats as the given text.
func New(text string) error
-
Errorf formats according to a format specifier and returns the string as a value that satisfies error.
func Errorf(format string, a ...interface{}) error
-
Merge merges multi errors.
func Merge(errs ...error) error
-
Append appends multiple errors to the error.
func Append(err error, errs ...error) error
Shutdown or reboot current process gracefully.
-
import it
"github.com/luoyeaichifan/goutil/graceful"
-
GraceSignal open graceful shutdown or reboot signal.
func GraceSignal()
-
SetShutdown sets the function which is called after the process shutdown, and the time-out period for the process shutdown. If 0<=timeout<5s, automatically use 'MinShutdownTimeout'(5s). If timeout<0, indefinite period. 'firstSweepFunc' is first executed. 'beforeExitingFunc' is executed before process exiting.
func SetShutdown(timeout time.Duration, firstSweepFunc, beforeExitingFunc func() error)
-
Shutdown closes all the frame process gracefully. Parameter timeout is used to reset time-out period for the process shutdown.
func Shutdown(timeout ...time.Duration)
-
Reboot all the frame process gracefully.
Notes:
Windows system are not supported!func Reboot(timeout ...time.Duration)
-
AddInherited adds the files and envs to be inherited by the new process.
Notes:
Only for reboot!
Windows system are not supported!func AddInherited(procFiles []*os.File, envs []*Env)
-
Logger logger interface
Logger interface { Infof(format string, v ...interface{}) Errorf(format string, v ...interface{}) }
-
SetLog resets logger
func SetLog(logger Logger)
GoPool is a Goroutines pool. It can control concurrent numbers, reuse goroutines.
-
import it
"github.com/luoyeaichifan/goutil/pool"
-
GoPool executes concurrently incoming function via a pool of goroutines in FILO order, i.e. the most recently stopped goroutine will execute the next incoming function. Such a scheme keeps CPU caches hot (in theory).
type GoPool struct { // Has unexported fields. }
-
NewGoPool creates a new *GoPool. If maxGoroutinesAmount<=0, will use default value. If maxGoroutineIdleDuration<=0, will use default value.
func NewGoPool(maxGoroutinesAmount int, maxGoroutineIdleDuration time.Duration) *GoPool
-
Go executes the function via a goroutine. If returns non-nil, the function cannot be executed because exceeded maxGoroutinesAmount limit.
func (gp *GoPool) Go(fn func()) error
-
TryGo tries to execute the function via goroutine. If there are no concurrent resources, execute it synchronously.
func (gp *GoPool) TryGo(fn func())
-
Stop starts GoPool. If calling 'Go' after calling 'Stop', will no longer reuse goroutine.
func (gp *GoPool) Stop()
ResPool is a high availability/high concurrent resource pool, which automatically manages the number of resources. So it is similar to database/sql's db pool.
-
import it
"github.com/luoyeaichifan/goutil/pool"
-
ResPool is a pool of zero or more underlying avatar(resource). It's safe for concurrent use by multiple goroutines. ResPool creates and frees resource automatically; it also maintains a free pool of idle avatar(resource).
type ResPool interface { // Name returns the name. Name() string // Get returns a object in Resource type. Get() (Resource, error) // GetContext returns a object in Resource type. // Support context cancellation. GetContext(context.Context) (Resource, error) // Put gives a resource back to the ResPool. // If error is not nil, close the avatar. Put(Resource, error) // Callback callbacks your handle function, returns the error of getting resource or handling. // Support recover panic. Callback(func(Resource) error) error // Callback callbacks your handle function, returns the error of getting resource or handling. // Support recover panic and context cancellation. CallbackContext(context.Context, func(Resource) error) error // SetMaxLifetime sets the maximum amount of time a resource may be reused. // // Expired resource may be closed lazily before reuse. // // If d <= 0, resource are reused forever. SetMaxLifetime(d time.Duration) // SetMaxIdle sets the maximum number of resources in the idle // resource pool. // // If SetMaxIdle is greater than 0 but less than the new MaxIdle // then the new MaxIdle will be reduced to match the SetMaxIdle limit // // If n <= 0, no idle resources are retained. SetMaxIdle(n int) // SetMaxOpen sets the maximum number of open resources. // // If MaxIdle is greater than 0 and the new MaxOpen is less than // MaxIdle, then MaxIdle will be reduced to match the new // MaxOpen limit // // If n <= 0, then there is no limit on the number of open resources. // The default is 0 (unlimited). SetMaxOpen(n int) // Close closes the ResPool, releasing any open resources. // // It is rare to close a ResPool, as the ResPool handle is meant to be // long-lived and shared between many goroutines. Close() error // Stats returns resource statistics. Stats() ResPoolStats }
-
NewResPool creates ResPool.
func NewResPool(name string, newfunc func(context.Context) (Resource, error)) ResPool
-
Resource is a resource that can be stored in the ResPool.
type Resource interface { // SetAvatar stores the contact with resPool // Do not call it yourself, it is only called by (*ResPool).get, and will only be called once SetAvatar(*Avatar) // GetAvatar gets the contact with resPool // Do not call it yourself, it is only called by (*ResPool).Put GetAvatar() *Avatar // Close closes the original source // No need to call it yourself, it is only called by (*Avatar).close Close() error }
-
Avatar links a Resource with a mutex, to be held during all calls into the Avatar.
type Avatar struct { // Has unexported fields. }
-
Free releases self to the ResPool. If error is not nil, close it.
func (avatar *Avatar) Free(err error)
-
ResPool returns ResPool to which it belongs.
func (avatar *Avatar) ResPool() ResPool
-
ResPools stores ResPool.
type ResPools struct { // Has unexported fields. }
-
NewResPools creates a new ResPools.
func NewResPools() *ResPools
-
Clean delects and close all the ResPools.
func (c *ResPools) Clean()
-
Del delects ResPool by name, and close the ResPool.
func (c *ResPools) Del(name string)
-
Get gets ResPool by name.
func (c *ResPools) Get(name string) (ResPool, bool)
-
GetAll gets all the ResPools.
func (c *ResPools) GetAll() []ResPool
-
Set stores ResPool. If the same name exists, will close and cover it.
func (c *ResPools) Set(pool ResPool)
Non-blocking asynchronous multiplex resource pool.
Conditions of Use:
- Limited resources
- Resources can be multiplexed non-blockingly and asynchronously
- Typical application scenarios, such as connection pool for asynchronous communication
Performance:
-
The longer the business is, the more obvious the performance improvement.
If the service is executed for 1ms each time, the performance is improved by about 4 times;
If the business is executed for 10ms each time, the performance is improved by about 28 times -
The average time spent on each operation will not change significantly,
but the overall throughput is greatly improved -
import it
"github.com/luoyeaichifan/goutil/pool"
-
Type definition
type ( // Worker woker interface // Note: Worker can not be implemented using empty structures(struct{})! Worker interface { Health() bool Close() error } // Workshop working workshop Workshop struct { // Has unexported fields. } )
-
NewWorkshop creates a new workshop(non-blocking asynchronous multiplex resource pool).
If maxQuota<=0, will use default value.
If maxIdleDuration<=0, will use default value.
Notes:
Worker can not be implemented using empty structures(struct{})!func NewWorkshop(maxQuota int, maxIdleDuration time.Duration, newWorkerFunc func() (Worker, error)) *Workshop
-
Close wait for all the work to be completed and close the workshop.
func (w *Workshop) Close()
-
Callback assigns a healthy worker to execute the function.
func (w *Workshop) Callback(fn func(Worker) error) error
-
Fire marks the worker to reduce a job.
If the worker does not belong to the workshop, close the worker.func (w *Workshop) Fire(worker Worker)
-
Hire hires a healthy worker and marks the worker to add a job.
func (w *Workshop) Hire() (Worker, error)
-
Stats returns the current workshop stats.
func (w *Workshop) Stats() *WorkshopStats
Password check password.
-
import it
"github.com/luoyeaichifan/goutil/password"
-
CheckPassword checks if the password matches the format requirements.
func CheckPassword(pw string, flag Flag, minLen int, maxLen ...int) bool
Various small functions.
-
import it
"github.com/luoyeaichifan/goutil"
-
BytesToString convert []byte type to string type.
func BytesToString(b []byte) string
-
StringToBytes convert string type to []byte type.
NOTE: panic if modify the member value of the []byte.func StringToBytes(s string) []byte
-
SpaceInOne combines multiple consecutive space characters into one.
func SpaceInOne(s string) string
-
NewRandom creates a new padded Encoding defined by the given alphabet string.
func NewRandom(alphabet string) *Random
-
RandomBytes returns securely generated random bytes. It will panic if the system's secure random number generator fails to function correctly.
func RandomBytes(n int) []byte
-
URLRandomString returns a URL-safe, base64 encoded securely generated
random string. It will panic if the system's secure random number generator
fails to function correctly.
The length n must be an integer multiple of 4, otherwise the last character will be padded with=
.func URLRandomString(n int) string
-
CamelString converts the accepted string to a camel string (xx_yy to XxYy)
func CamelString(s string) string
-
SnakeString converts the accepted string to a snake string (XxYy to xx_yy)
func SnakeString(s string) string
-
ObjectName gets the type name of the object
func ObjectName(obj interface{}) string
-
GetCallLine gets caller line information.
func GetCallLine(calldepth int) string
-
JsQueryEscape escapes the string in javascript standard so it can be safely placed inside a URL query.
func JsQueryEscape(s string) string
-
JsQueryUnescape does the inverse transformation of JsQueryEscape, converting %AB into the byte 0xAB and '+' into ' ' (space).
It returns an error if any % is not followed by two hexadecimal digits.func JsQueryUnescape(s string) (string, error)
-
Map is a concurrent map with loads, stores, and deletes.
It is safe for multiple goroutines to call a Map's methods concurrently.type Map interface { // Load returns the value stored in the map for a key, or nil if no // value is present. // The ok result indicates whether value was found in the map. Load(key interface{}) (value interface{}, ok bool) // Store sets the value for a key. Store(key, value interface{}) // LoadOrStore returns the existing value for the key if present. // Otherwise, it stores and returns the given value. // The loaded result is true if the value was loaded, false if stored. LoadOrStore(key, value interface{}) (actual interface{}, loaded bool) // Range calls f sequentially for each key and value present in the map. // If f returns false, range stops the iteration. Range(f func(key, value interface{}) bool) // Random returns a pair kv randomly. // If exist=false, no kv data is exist. Random() (key, value interface{}, exist bool) // Delete deletes the value for a key. Delete(key interface{}) // Clear clears all current data in the map. Clear() // Len returns the length of the map. Len() int }
-
RwMap creates a new concurrent safe map with sync.RWMutex.
The normal Map is high-performance mapping under low concurrency conditions.func RwMap(capacity ...int) Map
-
AtomicMap creates a concurrent map with amortized-constant-time loads, stores, and deletes.
It is safe for multiple goroutines to call a atomicMap's methods concurrently. From go v1.9 sync.Map.func AtomicMap() Map
-
SelfPath gets compiled executable file absolute path.
func SelfPath() string
-
SelfDir gets compiled executable file directory.
func SelfDir() string
-
RelPath gets relative path.
func RelPath(targpath string) string
-
SelfChdir switch the working path to my own path.
func SelfChdir()
-
FileExists reports whether the named file or directory exists.
func FileExists(name string) bool
-
SearchFile Search a file in paths.
This is often used in search config file in/etc
~/
func SearchFile(filename string, paths ...string) (fullpath string, err error)
-
GrepFile like command grep -E.
For example: GrepFile(^hello
, "hello.txt").\n
is striped while readfunc GrepFile(patten string, filename string) (lines []string, err error)
-
WalkDirs traverses the directory, return to the relative path.
You can specify the suffix.func WalkDirs(targpath string, suffixes ...string) (dirlist []string)
-
IsExportedOrBuiltinType is this type exported or a builtin?
func IsExportedOrBuiltinType(t reflect.Type) bool
-
IsExportedName is this an exported - upper case - name?
func IsExportedName(name string) bool
-
PanicTrace trace panic stack info.
func PanicTrace(kb int) []byte
-
ExtranetIP get external IP addr.
NOTE: Query IP information from the service API: http://pv.sohu.com/cityjson?ie=utf-8func ExtranetIP() (ip string, err error)
-
IntranetIP get internal IP addr.
func IntranetIP() (string, error)
-
Md5 returns the MD5 checksum string of the data.
func Md5(b []byte) string
-
AESEncrypt encrypts a piece of data.
The cipherkey argument should be the AES key, either 16, 24, or 32 bytes to select AES-128, AES-192, or AES-256.func AESEncrypt(cipherkey, src []byte) []byte
-
AESDecrypt decrypts a piece of data.
The cipherkey argument should be the AES key, either 16, 24, or 32 bytes to select AES-128, AES-192, or AES-256.func AESDecrypt(cipherkey, ciphertext []byte) ([]byte, error)
-
WritePidFile writes the current PID to the specified file.
func WritePidFile(pidFile ...string)
-
SetToStrings sets a element to the string set.
func SetToStrings(set []string, a string) []string
-
RemoveFromStrings removes a element from the string set.
func RemoveFromStrings(set []string, a string) []string
-
RemoveAllFromStrings removes all the a element from the string set.
func RemoveAllFromStrings(set []string, a string) []string
-
SetToInts sets a element to the int set.
func SetToInts(set []int, a int) []int
-
RemoveFromInts removes a element from the int set.
func RemoveFromInts(set []int, a int) []int
-
RemoveAllFromInts removes all the a element from the int set.
func RemoveAllFromInts(set []int, a int) []int
-
SetToInt32s sets a element to the int32 set.
func SetToInt32s(set []int32, a int32) []int32
-
RemoveFromInt32s removes a element from the int32 set.
func RemoveFromInt32s(set []int32, a int32) []int32
-
RemoveAllFromInt32s removes all the a element from the int32 set.
func RemoveAllFromInt32s(set []int32, a int32) []int32
-
SetToInt64s sets a element to the int64 set.
func SetToInt64s(set []int64, a int64) []int64
-
RemoveFromInt64s removes a element from the int64 set.
func RemoveFromInt64s(set []int64, a int64) []int64
-
RemoveAllFromInt64s removes all the a element from the int64 set.
func RemoveAllFromInt64s(set []int64, a int64) []int64
-
SetToInterfaces sets a element to the interface{} set.
func SetToInterfaces(set []interface{}, a interface{}) []interface{}
-
RemoveFromInterfaces removes a element from the interface{} set.
func RemoveFromInterfaces(set []interface{}, a interface{}) []interface{}
-
RemoveAllFromInterfaces removes all the a element from the interface{} set.
func RemoveAllFromInterfaces(set []interface{}, a interface{}) []interface{}
-
GetFirstGopath gets the first $GOPATH value.
func GetFirstGopath(allowAutomaticGuessing bool) (goPath string, err error)