teh-cmc / go-internals Goto Github PK
View Code? Open in Web Editor NEWA book about the internals of the Go programming language.
License: Other
A book about the internals of the Go programming language.
License: Other
The section
The NOP instruction just before the CALL exists so that the prologue doesn't jump directly onto a
CALL instruction. On some platforms, doing so can lead to very dark places; it's common pratice to
set-up a noop instruction right before the actual call and land on this NOP instead.
has generated some questions on reddit and #performance on the Gophers Slack. Could you please expand on this more?
It's quite possible this is covered in Chapter 2 or elsewhere, and I missed it, because I am lazy. If so, apologies!
Based on my C++ background, I believe I completely understand how Go handles your type-assertion example here. It can just, as you say, compare Eface._type
with type.uint32
.
But I cannot fathom how the following code works!
type Craft interface { Float() }
type Car struct {}
type Boat struct {}
func (c Boat) Drive() {}
func (c Boat) Float() {}
func (c Car) Drive() {}
var vface interface { Drive() }
var cface interface { Float() }
func main() {
b := Boat{}
vface = b
cface = vface.(Craft)
}
On the line cface = vface.(Craft)
, all the compiler knows about vface
is that it is some kind of Vehicle
. (Incidentally, its _type
compares equal to type.Boat
; but nobody said anything about Boat
on this line, so that can't be relevant to anything.) So how does the runtime know how to convert an arbitrary Vehicle
into a Craft
?
If you initialize b := Car{}
instead, then you get panic: interface conversion: main.Car is not main.Craft: missing method Float
at runtime — which is exactly what I would expect — but, I don't understand how the runtime figured that out. Does the _type
structure for Car
contain a list of the names and signatures of every one of its methods, and then the interface-conversion code walks through that whole list at runtime to collect the needed methods to populate a Craft
itab (or panic)?
In C++ terms, your Eface.(uint32)
example is simply a std::any_cast
— relatively cheap — but the vface.(Craft)
example seems much wilder, much more dynamic and costly — so costly that it can't be done at all in C++. Is that right?
That this reordering is a net win for us in this particular case is nothing but mere luck, AFAICT. In fact, if you take the time to experiment a bit with type-switches, especially ones with more than two cases, you'll find that the compiler always shuffles the cases using some kind of deterministic heuristics.
What those heuristics are, I don't know (but as always, I'd love to if you do).
Type-switches have sorted and runtime use a Binary search to find the result , so the time complexity is log2(n).
if necessary, i am glad to pull request to this repo about how the golang compile do
Like most recent compilers, the Go tool suite always references argument and locals using offsets from the stack-pointer directly in the code it generates. This allows for the frame-pointer to be used as an extra general-purpose register on platform with fewer registers (e.g. x86).
But it doesn't, starting with 1.8 (or was it 1.7 or 1.9?) the compiler emits instructions to use the frame pointer for its intended purpose, to help dtrace-like debugging tools. Just like you say later in the chapter. Maybe I'm misunderstanding.
title.
Side issue: As chapter 2 talks a lot about indirect calls: I would be interested in your opinion on Spectre variant 2 (AKA CVE-2017-5715 AKA Branch Target Injection) attacks on Go code.
There's also not much talk about retpoline support for the Go compiler.
Do you have any thoughts on this topic that you want to share?
First of all, great work @teh-cmc . I really enjoy your book, it help me to understand Go better.
Since version 1.17, Go has changed its argument passing convention from solely stack based to register based Go 1.17 release note.
Could you consider update your book or insert a warning to notice readers about this change?
Thanks.
If required I would even like to help with a few PR's so that the book's structure conforms to that of gitbook. Although I think markdown should work right-away there too
After so many years, seems go compiler optimization have changed a lot. One problem is in chapter 2, iface.go, chapter 2. To make the assembly code call the function through itab func pointer,
func main() {
m := Mather(Adder{id: 6754})
// This call just makes sure that the interface is actually used.
// Without this call, the linker would see that the interface defined above
// is in fact never used, and thus would optimize it out of the final
// executable.
m.Add(10, 32)
}
should be changed to
func main() {
var m Mather
m = Adder{id: 6754}
// This call just makes sure that the interface is actually used.
// Without this call, the linker would see that the interface defined above
// is in fact never used, and thus would optimize it out of the final
// executable.
m.Add(10, 32)
}
Or the assembly code would CALL "".Adder.Add(SB) directly.
have read the first two chapters, it's really amazing!
any plans for writing compiling part as well as runtime?
You asked why go.itabs are dupok. I believe the fundamental reason is that Go can be statically creating iface structures outside of either the package that defines the type or the package that defines the interface.
To make this concrete, here's a scenario where I believe that Go has to create duplicate ones. Suppose that we have four packages, A, B, C, D, and a main package. A defines a concrete type T, B defines an interface I (and perhaps some functions that take it), and both C and D import A and B and statically create B.I instances from A.T instances. Now we use both C and D in our main program.
Package A and B don't know about each other, so neither can define the iface<B.I, A.T> structure. Since C and D may be used by themselves, each must define this separately; they both need it and they have no guaranteed source of it outside themselves. Then when we use both of them together in our main program, we have duplicate definitions of iface<B.I, A.T>, one from each package, so Go must make such duplicates harmless.
We can also use nm
in binutils
to dump symbol information
nm --print-size ./iface |grep "go.itab.main.Adder,main.Mather"
00000000004767a0 0000000000000028 R go.itab.main.Adder,main.Mather
Though no difference ...
It would be helpful to use different term than "top" to refer to the first "element" in the stack that would be popped off. Maybe, tip or head?
Here's a sentence that felt confusing:
"".b+12(SP) and "".a+8(SP) respectively refer to the addresses 12 bytes and 8 bytes below the top of the stack (remember: it grows downwards!).
From this sentence, I thought the stack looked like as follows (here the stack is growing downwards):
RETURN ADDRESS 0(SP)
Argument A 8(SP)
Argument B 12(SP)
However, the diagram in the book is as follows:
| +-------------------------+ <-- 32(SP)
| | |
G | | |
R | | |
O | | main.main's saved |
W | | frame-pointer (BP) |
S | |-------------------------| <-- 24(SP)
| | [alignment] |
D | | "".~r3 (bool) = 1/true | <-- 21(SP)
O | |-------------------------| <-- 20(SP)
W | | |
N | | "".~r2 (int32) = 42 |
W | |-------------------------| <-- 16(SP)
A | | |
R | | "".b (int32) = 32 |
D | |-------------------------| <-- 12(SP)
S | | |
| | "".a (int32) = 10 |
| |-------------------------| <-- 8(SP)
| | |
| | |
| | |
\ | / | return address to |
\|/ | main.main + 0x30 |
- +-------------------------+ <-- 0(SP) (TOP OF STACK)
Looking at this diagram I see that the top of the stack refers to the bottom of the stack, but it might be nice to use different terminology.
Hello, i am a native chinese speaker, and i am devoted to studying Golang also, and found that there is
little information about golang internals. your project seems like get this point. So let me work with you.
Let more chinese programmer understanding golang better.
$ echo 'obase=2;137438953482' | bc
10000000000000000000000000000000001010
\_____/\_____________________________/
32 10
should be
$ echo 'obase=2;137438953482' | bc
10000000000000000000000000000000001010
\____/\______________________________/
32 10
Question: Whats the difference between the output provided by the -S compile switch to the one that go tool objdump
provides?
The Go compiler never generates instructions from the PUSH/POP family: the stack is grown or shrunk by respectively decrementing or incrementing the virtual stack pointer SP.
The asm code in this chapter is generated by the compiler, I think Go compiler(1.10) will not generate code that refer to virtual register SP, it should be the hardware SP.
I actually got surprised that type switches are O(N).
"how else could it work?"
I would assume that since the list of hashes is known at compile time, the compiler could find a perfect hashing function and just do a few arithmetic ops to get the address on which to jump1. At least if the size of switch exceeds some threshold.
If finding perfect hash would be too time-consuming, it could at least sort the hashes and do a binary search.
I guess the answer is that this is one of the optimizations Go is still awaiting...
EDIT: 1 Actually it would need to get just an index and lookup the target address; compiler knows the list of target hashes but not all input hashes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.