gopcp / example.v2 Goto Github PK
View Code? Open in Web Editor NEWAn example project for book 'Go Programming & Concurrency in Practice, 2nd edition' (《Go并发编程实战》第2版).
License: Apache License 2.0
An example project for book 'Go Programming & Concurrency in Practice, 2nd edition' (《Go并发编程实战》第2版).
License: Apache License 2.0
操作系统:win10,64位
go版本:1.10.3 windows/amd64
代码:书中“1.5 问候程序”->"代码清单 1-2 hello.go"
我运行的代码如下:
package main
import(
"bufio"
"fmt"
"os"
)
func main() {
inputReader := bufio.NewReader(os.Stdin)
fmt.Println("Please input your name:")
input, err := inputReader.ReadString('\n')
if err != nil {
fmt.Printf("An error occurred: %s\n", err)
os.Exit(1)
}else {
input = input[:len(input)-1]
fmt.Println(input)
fmt.Printf("Hello, %s!", input)
}
}
运行结果:
Please input your name:
Robert
Robert
!ello, Robert
以上代码与书中基本一致,书中给出的运行结果为:
Please input your name:
Robert
Hello, Robert!
请问为什么会出现不同的结果呢?
Run command ps aux | grep apipe
:
Error: Couldn't wait for the second command: exit status 1
go version
go version go1.13.4 darwin/amd64
func (analyzer *myAnalyzer) Analyze(
resp *module.Response) (dataList []module.Data, errorList []error) {
analyzer.ModuleInternal.IncrHandlingNumber()
defer analyzer.ModuleInternal.DecrHandlingNumber()
analyzer.ModuleInternal.IncrCalledCount()
if resp == nil {
errorList = append(errorList,
genParameterError("nil response"))
return
}
httpResp := resp.HTTPResp()
if httpResp == nil {
errorList = append(errorList,
genParameterError("nil HTTP response"))
return
}
httpReq := httpResp.Request
if httpReq == nil {
errorList = append(errorList,
genParameterError("nil HTTP request"))
return
}
var reqURL = httpReq.URL
if reqURL == nil {
errorList = append(errorList,
genParameterError("nil HTTP request URL"))
return
}
analyzer.ModuleInternal.IncrAcceptedCount()
respDepth := resp.Depth()
logger.Infof("Parse the response (URL: %s, depth: %d)... \n",
reqURL, respDepth)
// 解析HTTP响应。
if httpResp.Body != nil {
defer httpResp.Body.Close()
}
multipleReader, err := reader.NewMultipleReader(httpResp.Body)
if err != nil {
errorList = append(errorList, genError(err.Error()))
return
}
dataList = []module.Data{}
for _, respParser := range analyzer.respParsers {
httpResp.Body = multipleReader.Reader()
pDataList, pErrorList := respParser(httpResp, respDepth)
if pDataList != nil {
for _, pData := range pDataList {
if pData == nil {
continue
}
dataList = appendDataList(dataList, pData, respDepth)
}
}
if pErrorList != nil {
for _, pError := range pErrorList {
if pError == nil {
continue
}
errorList = append(errorList, pError)
}
}
}
if len(errorList) == 0 {
analyzer.ModuleInternal.IncrCompletedCount()
}
return dataList, errorList
}
// 解析HTTP响应。
if httpResp.Body != nil {
defer httpResp.Body.Close()
}
multipleReader, err := reader.NewMultipleReader(httpResp.Body)
if err != nil {
errorList = append(errorList, genError(err.Error()))
return
}
dataList = []module.Data{}
for _, respParser := range analyzer.respParsers {
httpResp.Body = multipleReader.Reader()
pDataList, pErrorList := respParser(httpResp, respDepth)
.....
}
httpResp.Body = multipleReader.Reader()
这里httpResp.Body赋了一个新值,是否会导致原来的Body没有关闭从而造成内存泄漏cmap.segments = make([]Segment, concurrency);
假如 concurrency为6
那么int(keyHash32 >> 1) % 5的值为 0 到 4
那么有一个 segment 是不是会一直取不到
这个包 import "golang.org/x/net/..." 要到哪里下载啊,我到墙外也没找到下载链接啊
apipe.go
ref: https://golang.org/pkg/io/#Reader
// Callers should always process the n > 0 bytes returned before
// considering the error err. Doing so correctly handles I/O errors
// that happen after reading some bytes and also both of the
// allowed EOF behaviors.
for {
tempOutput := make([]byte, 5)
n, err := stdout0.Read(tempOutput)
if err != nil {
if err == io.EOF {
break
} else {
fmt.Printf("Error: Couldn't read data from the pipe: %s\n", err)
return
}
}
if n > 0 {
outputBuf0.Write(tempOutput[:n])
}
}
for {
tempOutput := make([]byte, 5)
n, err := stdout0.Read(tempOutput)
if n > 0 {
outputBuf0.Write(tempOutput[:n])
}
if err != nil {
if err == io.EOF {
break
} else {
fmt.Printf("Error: Couldn't read data from the pipe: %s\n", err)
return
}
}
}
应该使用无重复的Pair,保证获取FirstPair时正确。
testCases:=genNoRepeatTestPairs(number)
老师您好,在看第五章Cmap实战的时候有一个问题,bucket delete方法用新链表替换旧链表的时候,会出现新链表和旧链表同时存在的状态,请问旧链表的内存回收是否会出现问题?
如果有线程阻塞在旧链表的某处,这时候旧链表某节点被回收肯定会引发错误,请问go的垃圾回收机制如何知道适当的回收时机呢?
这三行应该是多余的吧
书中,loadgen的并发设计是根据过期时间/时间间隔的方式~
想请问:为何不直接使用for循环concurrency次+timeout的形式来处理呢?这样不是免去了tick的使用?
func (s *segment) GetWithHash(key string, keyHash uint64) Pair {
s.lock.Lock()
b := s.buckets[int(keyHash%uint64(s.bucketsLen))]
s.lock.Unlock()
return b.Get(key) // 存在并发的风险,可能在get的过程中有Delete和Put操作
}
chapter3/socket/tcp_socket.go 中,server 端的 wg.Done() 放在 handleConn() 中,会导致 server 端的 listener.Close() 无法被执行吧?
package analyzer
....
// New 用于创建一个分析器实例。
func New(
mid module.MID,
respParsers []module.ParseResponse,
scoreCalculator module.CalculateScore) (module.Analyzer, error) {
moduleBase, err := stub.NewModuleInternal(mid, scoreCalculator)
if err != nil {
return nil, err
}
if respParsers == nil {
return nil, genParameterError("nil response parsers")
}
if len(respParsers) == 0 {
return nil, genParameterError("empty response parser list")
}
var innerParsers []module.ParseResponse
for i, parser := range respParsers {
if parser == nil {
return nil, genParameterError(fmt.Sprintf("nil response parser[%d]", i))
}
innerParsers = append(innerParsers, parser)
}
return &myAnalyzer{
ModuleInternal: moduleBase,
respParsers: innerParsers,
}, nil
}
var innerParsers []module.ParseResponse
for i, parser := range respParsers {
if parser == nil {
return nil, genParameterError(fmt.Sprintf("nil response parser[%d]", i))
}
innerParsers = append(innerParsers, parser)
}
respParsers
参数,需要把值赋值到一个新的变量innerParsers
里面去.不赋值给新的变量我感觉也没什么问题啊!也不存在并发问题.如果参数出错了就直接return了.Pipline
里面也是看到这样的代码.老师您好,在cmap里面bucket
的Delete
方法是否可以简化一下 .我们可以直接记录删除要删除节点的pre节点和next节点.把pre.next = next节点
书里面的示例代码:
func (b *bucket) Delete(key string, lock sync.Locker) bool {
if lock != nil {
lock.Lock()
defer lock.Unlock()
}
firstPair := b.GetFirstPair()
if firstPair == nil {
return false
}
var prevPairs []Pair
var target Pair
var breakpoint Pair
for v := firstPair; v != nil; v = v.Next() {
if v.Key() == key {
target = v
breakpoint = v.Next()
break
}
prevPairs = append(prevPairs, v)
}
if target == nil {
return false
}
newFirstPair := breakpoint
for i := len(prevPairs) - 1; i >= 0; i-- {
pairCopy := prevPairs[i].Copy()
pairCopy.SetNext(newFirstPair)
newFirstPair = pairCopy
}
if newFirstPair != nil {
b.firstValue.Store(newFirstPair)
} else {
b.firstValue.Store(placeholder)
}
atomic.AddUint64(&b.size, ^uint64(0))
return true
}
func (b *bucket) Delete(key string, locker sync.Locker) bool {
if locker != nil {
locker.Lock()
defer locker.Unlock()
}
firstPair := b.GetFirstPair()
if firstPair == nil {
return false
}
var prePair IPair
var target IPair
var nextPair IPair
for v := firstPair; v != nil; v = v.Next() {
if v.Key() == key {
target = v
nextPair = v.Next()
break
}
prePair = v
}
if target == nil {
return false
}
// bucket只存在一个元素
if prePair == nil {
b.firstValue.Store(placeholder)
} else {
prePair.SetNext(nextPair)
}
atomic.AddUint64(&b.size, ^uint64(0))
return true
}
书上的 chanbase1.go 的运行结果与实际运行结果(win10,go 1.8.1)不同。书上的结果 Sent: d[sender]
在实际运行时会出现在 Received: d
之后。我想容易对读者出现困扰。是否可以在示例的
书上的 Sent :d
会在Received 之前出现。与实际运行结果不符合(win10 go 1.8,1)可以考虑在示例中
for {
if elem, ok := <-strChan; ok {
fmt.Println("Received:", elem, "[Receiver]")
time.Sleep(time.Second) //这里
} else {
break
}
}
多加一个 time.Second
这样的话可以让读者更好地理解
具体部分是bucket 实现这里。
假如不提供锁参数,单靠原子更新表头,会不会无法保证线程安全?
举个例子,bucket.delete
这里,您的实现是copy被删除元素的之前元素链,和之后元素,然后重新构建一条新的单向链表,原子更新firstValue,链表头。
问题就出在这里,假如Copy组成新链表的同一时刻,暂未更新链表头。这时候别的线程操作,更新,或者删除这个bucket链表段中的元素,那么当前copy的这段链表段就不是最新的数据了,然后用这条脏的链表段,更新表头,会覆盖了之前的操作,导致之前别的线程操作的丢失。
问题复现:
bucket.go
文件中,L135之前,添加time.Sleep(time.Second)
放大模拟一个删除延时。
bucket_test.go
func TestBucketDelaySecDeleteNoLocker(t *testing.T) {
b := newBucket()
wg := sync.WaitGroup{}
wg.Add(2)
p, _ := newPair("A", "a")
b.Put(p, nil)
p, _ = newPair("B", "b")
b.Put(p, nil)
p, _ = newPair("C", "c")
b.Put(p, nil)
//c b a
t.Logf("before bucket:\n%v\n", b)
go func() {
defer wg.Done()
p, _ := newPair("D", "d")
_, err := b.Put(p, nil)
if err != nil {
t.Fatalf("An error occurs when putting a pair to the bucket: %s (pair: %#v)", b, p)
}
// bucket should be D C B A
t.Logf("after put item D, bucket:\n%v\n", b)
}()
go func() {
defer wg.Done()
// delete item C
b.Delete("C", nil)
}()
wg.Wait()
expected := []string{"D", "B", "A"}
actual := make([]string, 0)
for curr := b.GetFirstPair(); curr != nil; curr = curr.Next() {
actual = append(actual, curr.Key())
}
t.Logf("after delete C , bucket:\n%v\n", b)
if !reflect.DeepEqual(actual, expected) {
t.Fatalf("expected %v , but %v", expected, actual)
}
}
=== RUN TestBucketDelaySecDeleteNoLocker
bucket_test.go:323: before bucket:
[ pair{key:C, hash:67, element:c, nextKey:B} pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
bucket_test.go:333: after put item D, bucket:
[ pair{key:D, hash:68, element:d, nextKey:C} pair{key:C, hash:67, element:c, nextKey:B} pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
bucket_test.go:349: after delete C , bucket:
[ pair{key:B, hash:66, element:b, nextKey:A} pair{key:A, hash:65, element:a, nextKey:} ]
bucket_test.go:352: expected [D B A] , but [B A]
--- FAIL: TestBucketDelaySecDeleteNoLocker (1.01s)
FAIL
让我想起那个 笑话
四块
十块?
不是十块 而是四块
二十四块?
不是二十四块 就是四块
九十四块?
这几本编程书下来
以后买书会有个标准
大于300页的烂书率高
Run command echo -n "My first command comes from golang."
:
My first command comes from golang.
Run command ps aux | grep apipe
:
Error: Couldn't wait for the second command: exit status 1
cow.go 41- 44 行
`func (array *intArray) Set(index uint32, elem int) (err error) {
if err = array.checkIndex(index); err != nil {
return
}
if err = array.checkValue(); err != nil {
return
}
// 不要这样做!否则会形成竞态条件!
// oldArray := array.val.Load().([]int)
// oldArray[index] = elem
// array.val.Store(oldArray)
newArray := make([]int, array.length)
copy(newArray, array.val.Load().([]int))
newArray[index] = elem
array.val.Store(newArray)
return
}`
chapter5/value/cow/cow.go
中Set
方法:
newArray := make([]int, array.length)
copy(newArray, array.val.Load().([]int))
newArray[index] = elem
array.val.Store(newArray)
这样实现是否会有并发问题?复制和设值这段是否需要加锁?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.