In a tri-color collector, every object is either white, grey, or black and we view the heap as a graph of connected objects.
接下来,就是具体的三色标记法的工作了:
At the start of a GC cycle all objects are white. The GC visits all roots, which are objects directly accessible by the application such as globals and things on the stack, and colors these grey. The GC then chooses a grey object, blackens it, and then scans it for pointers to other objects. When this scan finds a pointer to a white object, it turns that object grey. This process repeats until there are no more grey objects. At this point, white objects are known to be unreachable and can be reused.
Go’s write barrier colors the now-reachable object grey if it is currently white, ensuring that the garbage collector will eventually scan it for pointers.
var writeBarrier struct { enabled bool// compiler emits a check of this before calling write barrier pad [3]byte// compiler uses 32-bit load for "enabled" field needed bool// whether we need a write barrier for current GC phase cgo bool// whether we need a write barrier for a cgo check alignme uint64// guarantee alignment so that compiler can use a 32 or 64-bit load }
temporary spikes in memory usage should be handled by increasing CPU costs, not by aborting.
Basically if the GC sees memory pressure it informs the application that it should shed load. Once things are back to normal the GC informs the application that it can go back to its regular load.
这段话包含了Go语言的GC,在面对CPU和内存压力下的决策:
Go程序很少会OOM
这句话有一定前提,即内存设置是合理的,代码也没有明显的内存泄露问题
至于具体原因,我们看下文
业务高峰时内存使用率过高,应该通过提升CPU能力来解决,而不是中止程序
自动GC是需要CPU的计算资源做支持,来清理无用内存
要保证内存资源能支持程序的正常运行,有两个思路:
减少已有内存 - 通过GC来回收无用的内存
限制新增内存 - 即运行时尽可能地避免新内存的分配,最简单的方法就是不运行代码
显然,中止程序对业务的影响很大,我们更倾向于通过GC去回收内存,腾出新的空间
GC压力高时,通知应用减少负载;而当恢复正常后,GC再通知应用可以恢复到正常模式了
我们可以将上述分为两类工作
业务逻辑的Goroutine
GC的Goroutine
这两类Goroutine都会消耗CPU资源,区别在于:
运行业务逻辑往往会增加内存
GC是回收内存
这里就能体现出Go运行时的策略
内存压力高时,GC线程更容易抢占到CPU资源,进行内存回收
代价是业务处理逻辑会有一定性能损耗,被分配的计算资源减少
GC最直观的影响就体现在延迟上。尤其是在STW - Stop The World情况下,程序会暂停所有非GC的工作,进行全量的垃圾回收。即便整个GC只花费了1s,所有涉及到这个程序的业务调用,都会增加1s延迟;在微服务场景下,这个问题会变得尤为复杂。
type Reader struct { r io.Reader pad int64// Amount of padding (ignored) after current file entry curr fileReader // Reader for current file entry blk block // Buffer to use as temporary local storage
// err is a persistent error. // It is only the responsibility of every exported method of Reader to // ensure that this error is sticky. err error }
// children goroutine funcSubFoo(ctx context.Context) { for { select { case <-ctx.Done(): return case <-dataCh: go LongLogic() case <-finishedCh: fmt.Println("LongLogic finished") } } }
funcSubFoo(ctx context.Context) { for { select { case <-ctx.Done(): return case <-dataCh: // logic包内部保证 logic.Run() case result := <-logic.Finish(): fmt.Println("result", result) } } }
而logic包中的大致框架如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
package logic
var finishedCh = make(chanstruct{})
funcRun() { // 在这里引入排队机制 gofunc() { // long time process <-finishedCh }() }
funcSubFoo(ctx context.Context) { for { select { case <-ctx.Done(): return case <-dataCh: // logic包内部保证 logic.Run() case result := <-logic.Finish(): fmt.Println("result", result) } } }
而logic包中的大致框架如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
package logic
var finishedCh = make(chanstruct{})
funcRun() { // 在这里引入排队机制 gofunc() { // long time process <-finishedCh }() }
多维度数据Dimensional data - Prometheus implements a highly dimensional data model. Time series are identified by a metric name and a set of key-value pairs.
强力的查询Powerful queries - PromQL allows slicing and dicing of collected time series data in order to generate ad-hoc graphs, tables, and alerts.
很棒的可视化Great visualization - Prometheus has multiple modes for visualizing data: a built-in expression browser, Grafana integration, and a console template language.
高效存储Efficient storage - Prometheus stores time series in memory and on local disk in an efficient custom format. Scaling is achieved by functional sharding and federation.
简单操作Simple operation - Each server is independent for reliability, relying only on local storage. Written in Go, all binaries are statically linked and easy to deploy.
精确告警Precise alerting - Alerts are defined based on Prometheus’s flexible PromQL and maintain dimensional information. An alertmanager handles notifications and silencing.
很多客户端库Many client libraries - Client libraries allow easy instrumentation of services. Over ten languages are supported already and custom libraries are easy to implement.
大量现有集成Many integrations - Existing exporters allow bridging of third-party data into Prometheus. Examples: system statistics, as well as Docker, HAProxy, StatsD, and JMX metrics.
GitOps is a way of managing your infrastructure and applications so that whole system is described declaratively and version controlled (most likely in a Git repository), and having an automated process that ensures that the deployed environment matches the state specified in a repository.
Source源,包括期望状态与获取的途径。
A Source defines the origin of a repository containing the desired state of the system and the requirements to obtain it (e.g. credentials, version selectors).
Reconciliation refers to ensuring that a given state (e.g. application running in the cluster, infrastructure) matches a desired state declaratively defined somewhere (e.g. a Git repository).
// children goroutine funcSubFoo(ctx context.Context) { for { select { case <-ctx.Done(): return case <-dataCh: go LongLogic() case <-finishedCh: fmt.Println("LongLogic finished") } } }