owned this note
owned this note
Published
Linked with GitHub
RWBY
===
<!-- .slide: data-background="pink" -->
<!-- .slide: data-transition="zoom" -->
Reader/Writer/Close :dizzy:
> [name=郭學聰 Hsueh-Tsung Kuo] [time=Thu, 27 Dec 2018] [color=red]
---
<!-- .slide: data-transition="convex" -->
## who am I?
----
<!-- .slide: data-transition="convex" -->
* programmer from Rayark, a game company in Taiwan
* backend engineer, focus on common service
* usually develop something related to my work in Python, Ruby, Golang, C#
* built almost entire VOEZ game server by myself only
* supported Sdorica backend development
---
<!-- .slide: data-transition="convex" -->
## outline
----
<!-- .slide: data-transition="convex" -->
* Reader/Writer spec & definition
* Reader/Writer interface variation
* why ioutil?
* defer & Close()
* customize Reader/Writer/Closer
* from block cipher to streaming
---
<!-- .slide: data-transition="convex" -->
## RWBY?
----
<!-- .slide: data-transition="convex" -->
---
<!-- .slide: data-transition="convex" -->
## Reader/Writer spec & definition
* Reader https://golang.org/pkg/io/#Reader
* Writer https://golang.org/pkg/io/#Writer
* Closer https://golang.org/pkg/io/#Closer
----
<!-- .slide: data-transition="convex" -->
### Reader
```go=
type Reader interface {
Read(p []byte) (n int, err error)
}
```
----
<!-- .slide: data-transition="convex" -->
### Reader
Key point:
* Callers should always process the `n > 0` bytes returned before considering the error err.
----
<!-- .slide: data-transition="convex" -->
:o:
```go=
func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error) {
if len(buf) < min {
return 0, ErrShortBuffer
}
for n < min && err == nil {
var nn int
nn, err = r.Read(buf[n:])
n += nn
}
if n >= min {
err = nil
} else if n > 0 && err == EOF {
err = ErrUnexpectedEOF
}
return
}
```
https://golang.org/src/io/io.go?s=10817:10883#L294
----
<!-- .slide: data-transition="convex" -->
:o:
```go=
func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
b.lastRead = opInvalid
for {
i := b.grow(MinRead)
b.buf = b.buf[:i]
m, e := r.Read(b.buf[i:cap(b.buf)])
if m < 0 {
panic(errNegativeRead)
}
b.buf = b.buf[:i+m]
n += int64(m)
if e == io.EOF {
return n, nil // e is EOF, so return nil explicitly
}
if e != nil {
return n, e
}
}
}
```
https://golang.org/src/bytes/buffer.go?s=6926:6985#L191
----
<!-- .slide: data-transition="convex" -->
### Writer
```go=
type Writer interface {
Write(p []byte) (n int, err error)
}
```
----
<!-- .slide: data-transition="convex" -->
### Writer
Key point:
* Write must return a non-nil error if it returns `n < len(p)`.
----
<!-- .slide: data-transition="convex" -->
### Closer
```go=
type Closer interface {
Close() error
}
```
----
<!-- .slide: data-transition="convex" -->
### Closer
Key point:
* The behavior of Close after the first call is undefined.
---
<!-- .slide: data-transition="convex" -->
## Reader/Writer interface variation
See: https://golang.org/pkg/io/
----
<!-- .slide: data-transition="convex" -->
### ReaderAt
```go=
type ReaderAt interface {
ReadAt(p []byte, off int64) (n int, err error)
}
```
----
<!-- .slide: data-transition="convex" -->
### ReaderAt
Key point:
* When ReadAt returns `n < len(p)`, it returns a non-nil error explaining why more bytes were not returned.
https://golang.org/pkg/io/#ReaderAt
---
<!-- .slide: data-transition="convex" -->
## why io & io/ioutil?
* Key point of Reader
----
<!-- .slide: data-transition="convex" -->
to ensure Read() fill buf
```go=
func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error)
func ReadFull(r Reader, buf []byte) (n int, err error)
```
https://golang.org/pkg/io/#ReadAtLeast
https://golang.org/pkg/io/#ReadFull
----
<!-- .slide: data-transition="convex" -->
or use utility function to read/write all
```go=
var Discard io.Writer = devNull(0)
func NopCloser(r io.Reader) io.ReadCloser
func ReadAll(r io.Reader) ([]byte, error)
func ReadDir(dirname string) ([]os.FileInfo, error)
func ReadFile(filename string) ([]byte, error)
func TempDir(dir, prefix string) (name string, err error)
func TempFile(dir, pattern string) (f *os.File, err error)
func WriteFile(filename string, data []byte, perm os.FileMode) error
```
https://golang.org/pkg/io/ioutil/
---
<!-- .slide: data-transition="convex" -->
## defer & Close()
----
<!-- .slide: data-transition="convex" -->
### may miss error
```go=
func RWBY() (err error) {
defer f.Close()
}
```
----
<!-- .slide: data-transition="convex" -->
### don't miss any error
```go=
func RWBY() (err error) {
defer func() {
cerr := f.Close()
if err == nil {
err = cerr
}
}()
}
```
----
<!-- .slide: data-transition="convex" -->
### don't miss any error
```go=
func safeClose(c io.Closer, err *error) {
cerr := c.Close()
if *err == nil {
*err = cerr
}
}
func RWBY() (err error) {
defer safeClose(f, &err)
}
```
---
<!-- .slide: data-transition="convex" -->
## customize Reader/Writer/Closer
----
<!-- .slide: data-transition="convex" -->
### WriterCounter
```go=
type WriterCounter struct {
writer io.Writer
count int64
}
func (wc *WriterCounter) Write(p []byte) (n int, err error) {
n, err = wc.writer.Write(p)
atomic.AddInt64(&wc.count, int64(n))
return
}
func (wc *WriterCounter) Count() int64 {
return atomic.LoadInt64(&wc.count)
}
func NewWriterCounter(w io.Writer) *WriterCounter {
return &WriterCounter{writer: w}
}
```
----
<!-- .slide: data-transition="convex" -->
### WriteNopCloser
```go=
type writeNopCloser struct {
io.Writer
}
func (_ writeNopCloser) Close() error {
return nil
}
func WriteNopCloser(w io.Writer) io.WriteCloser {
return writeNopCloser{Writer: w}
}
```
----
<!-- .slide: data-transition="convex" -->
### ChainCloseReadCloser
```go=
type ChainCloseReadCloser struct {
io.ReadCloser
underlyingReadClosers []io.ReadCloser
}
func (chainCloseReadCloser *ChainCloseReadCloser) Close() error {
err := chainCloseReadCloser.ReadCloser.Close()
for _, underlyingReadCloser := range chainCloseReadCloser.underlyingReadClosers {
cerr := underlyingReadCloser.Close()
// always return first close error
if err == nil {
err = cerr
}
}
return err
}
func NewChainCloseReadCloser(readCloser io.ReadCloser, underlyingReadClosers ...io.ReadCloser) io.ReadCloser {
return &ChainCloseReadCloser{ReadCloser: readCloser, underlyingReadClosers: underlyingReadClosers}
}
```
---
<!-- .slide: data-transition="convex" -->
## from block cipher to streaming
----
<!-- .slide: data-transition="convex" -->
* ???
---
<!-- .slide: data-transition="convex" -->
## conclusion
----
<!-- .slide: data-transition="convex" -->
> :hash: Reader/Writer/Closer 博大精深
> [name=Hsueh-Tsung Kuo] [time=Thu, 26 Apr 2018] [color=red]
----
<!-- .slide: data-transition="convex" -->
### special thanks
* komod
---
<!-- .slide: data-transition="zoom" -->
## Q&A
---
<style>
.reveal {
background: #FFDFEF;
color: black;
}
.reveal h2,
.reveal h3,
.reveal h4 {
color: black;
}
.reveal code {
font-size: 14px !important;
line-height: 1.2;
}
.rightpart{
float:right;
width:50%;
}
.leftpart{
margin-right: 50% !important;
height:50%;
}
.reveal section img { background:none; border:none; box-shadow:none; }
p.blo {
font-size: 50px !important;
background:#B6BDBB;
border:1px solid silver;
display:inline-block;
padding:0.5em 0.75em;
border-radius: 10px;
box-shadow: 5px 5px 5px #666;
}
p.blo1 {
background: #c7c2bb;
}
p.blo2 {
background: #b8c0c8;
}
p.blo3 {
background: #c7cedd;
}
p.bloT {
font-size: 60px !important;
background:#B6BDD3;
border:1px solid silver;
display:inline-block;
padding:0.5em 0.75em;
border-radius: 8px;
box-shadow: 1px 2px 5px #333;
}
p.bloA {
background: #B6BDE3;
}
p.bloB {
background: #E3BDB3;
}
.slide-number{
margin-bottom:10px !important;
width:100%;
text-align:center;
font-size:25px !important;
background-color:transparent !important;
}
iframe.myclass{
width:100px;
height:100px;
bottom:0;
left:0;
position:fixed;
border:none;
z-index:99999;
}
h1.raw {
color: #fff;
background-image: linear-gradient(90deg,#f35626,#feab3a);
-webkit-background-clip: text;
-webkit-text-fill-color: transparent;
animation: hue 5s infinite linear;
}
@keyframes hue {
from {
filter: hue-rotate(0deg);
}
to {
filter: hue-rotate(360deg);
}
}
.progress{
height:14px !important;
}
.progress span{
height:14px !important;
background: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAMCAIAAAAs6UAAAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAyJpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNiAoV2luZG93cykiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6QUNCQzIyREQ0QjdEMTFFMzlEMDM4Qzc3MEY0NzdGMDgiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6QUNCQzIyREU0QjdEMTFFMzlEMDM4Qzc3MEY0NzdGMDgiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDpBQ0JDMjJEQjRCN0QxMUUzOUQwMzhDNzcwRjQ3N0YwOCIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDpBQ0JDMjJEQzRCN0QxMUUzOUQwMzhDNzcwRjQ3N0YwOCIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/PovDFgYAAAAmSURBVHjaYvjPwMAAxjMZmBhA9H8INv4P4TPM/A+m04zBNECAAQBCWQv9SUQpVgAAAABJRU5ErkJggg==") repeat-x !important;
}
.progress span:after,
.progress span.nyancat{
content: "";
background: url('data:image/gif;base64,R0lGODlhIgAVAKIHAL3/9/+Zmf8zmf/MmZmZmf+Z/wAAAAAAACH/C05FVFNDQVBFMi4wAwEAAAAh/wtYTVAgRGF0YVhNUDw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iIHhtcE1NOk9yaWdpbmFsRG9jdW1lbnRJRD0ieG1wLmRpZDpDMkJBNjY5RTU1NEJFMzExOUM4QUM2MDAwNDQzRERBQyIgeG1wTU06RG9jdW1lbnRJRD0ieG1wLmRpZDpCREIzOEIzMzRCN0IxMUUzODhEQjgwOTYzMTgyNTE0QiIgeG1wTU06SW5zdGFuY2VJRD0ieG1wLmlpZDpCREIzOEIzMjRCN0IxMUUzODhEQjgwOTYzMTgyNTE0QiIgeG1wOkNyZWF0b3JUb29sPSJBZG9iZSBQaG90b3Nob3AgQ1M2IChXaW5kb3dzKSI+IDx4bXBNTTpEZXJpdmVkRnJvbSBzdFJlZjppbnN0YW5jZUlEPSJ4bXAuaWlkOkM1QkE2NjlFNTU0QkUzMTE5QzhBQzYwMDA0NDNEREFDIiBzdFJlZjpkb2N1bWVudElEPSJ4bXAuZGlkOkMyQkE2NjlFNTU0QkUzMTE5QzhBQzYwMDA0NDNEREFDIi8+IDwvcmRmOkRlc2NyaXB0aW9uPiA8L3JkZjpSREY+IDwveDp4bXBtZXRhPiA8P3hwYWNrZXQgZW5kPSJyIj8+Af/+/fz7+vn49/b19PPy8fDv7u3s6+rp6Ofm5eTj4uHg397d3Nva2djX1tXU09LR0M/OzczLysnIx8bFxMPCwcC/vr28u7q5uLe2tbSzsrGwr66trKuqqainpqWko6KhoJ+enZybmpmYl5aVlJOSkZCPjo2Mi4qJiIeGhYSDgoGAf359fHt6eXh3dnV0c3JxcG9ubWxramloZ2ZlZGNiYWBfXl1cW1pZWFdWVVRTUlFQT05NTEtKSUhHRkVEQ0JBQD8+PTw7Ojk4NzY1NDMyMTAvLi0sKyopKCcmJSQjIiEgHx4dHBsaGRgXFhUUExIREA8ODQwLCgkIBwYFBAMCAQAAIfkECQcABwAsAAAAACIAFQAAA6J4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93bqCA7f7TFaYoIFAMMwczB5EkTzJllEUttmIGoG5bfPBjDawD7CsJC67uWcv2CRov929C/q2ZpcBbYBmLGk6W1BRY4MUDnMvJEsBAXdlknk2fCeRk2iJliAijpBlEmigjR0plKSgpKWvEUheF4tUZqZID1RHjEe8PsDBBwkAIfkECQcABwAsAAAAACIAFQAAA6B4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93TqS40XiKSYgTLBgIBAMqE/zmQSaZEzns+jQ9pC/5dQJ0VIv5KMVWxqb36opxHrNvu9ptPfGbmsBbgSAeRdydCdjXWRPchQPh1hNAQF4TpM9NnwukpRyi5chGjqJEoSOIh0plaYsZBKvsCuNjY5ptElgDyFIuj6+vwcJACH5BAkHAAcALAAAAAAiABUAAAOfeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GKifWaddvNQAtszXYCxgR/Zy5jYTFeXmSDiIZGdQEBd06QSBQ5e4cEkE9nnZQaG2J4F4MSLx8rkqUSZBeurhlTUqsLsi60DpZxSWBJugcJACH5BAkHAAcALAAAAAAiABUAAAOgeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GuifWaddvNwMkZtmY7AWMEgGcKY2ExXl5khFMVc0Z1AQF3TpJShDl8iASST2efloV5JTyJFpgOch8dgW9KZxexshGNLqgLtbW0SXFwvaJfCQAh+QQJBwAHACwAAAAAIgAVAAADoXi63P7wmUmrnVGOzbvfRsYYXGGe6MmF4kEOaSGYMwq2LizHfDGwIlLPNKGZfi6gZmggEAy2iVPZEKZqzakq+1xUFFYe90lxTsHmim6HGpvf3eR7skYJ3PC5tyystc0AboFnVXQ9XFJTZIQOYUYFTQEBeWaSVF4bbCeRk1meBJYSL3WbaReMIxQfHXh6jaYXsbEQni6oaF21ERR7l0ksvA0JACH5BAkHAAcALAAAAAAiABUAAAOeeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMfFlA4hTITEMxkIBMOuADwmhzqeM6mashTCXKw2TVKQyKuTRSx2wegnNkyJ1ozpOFiMLqcEU8BZHx6NYW8nVlZefQ1tZgQBAXJIi1eHUTRwi0lhl48QL0sogxaGDhMlUo2gh14fHhcVmnOrrxNqrU9joX21Q0IUElm7DQkAIfkECQcABwAsAAAAACIAFQAAA6J4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93bqCA7f7TFaYoIFAMMwczB5EkTzJllEUttmIGoG5bfPBjDawD7CsJC67uWcv2CRov929C/q2ZpcBbYBmLGk6W1BRY4MUDnMvJEsBAXdlknk2fCeRk2iJliAijpBlEmigjR0plKSgpKWvEUheF4tUZqZID1RHjEe8PsDBBwkAIfkECQcABwAsAAAAACIAFQAAA6B4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93TqS40XiKSYgTLBgIBAMqE/zmQSaZEzns+jQ9pC/5dQJ0VIv5KMVWxqb36opxHrNvu9ptPfGbmsBbgSAeRdydCdjXWRPchQPh1hNAQF4TpM9NnwukpRyi5chGjqJEoSOIh0plaYsZBKvsCuNjY5ptElgDyFIuj6+vwcJACH5BAkHAAcALAAAAAAiABUAAAOfeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GKifWaddvNQAtszXYCxgR/Zy5jYTFeXmSDiIZGdQEBd06QSBQ5e4cEkE9nnZQaG2J4F4MSLx8rkqUSZBeurhlTUqsLsi60DpZxSWBJugcJACH5BAkHAAcALAAAAAAiABUAAAOgeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GuifWaddvNwMkZtmY7AWMEgGcKY2ExXl5khFMVc0Z1AQF3TpJShDl8iASST2efloV5JTyJFpgOch8dgW9KZxexshGNLqgLtbW0SXFwvaJfCQAh+QQJBwAHACwAAAAAIgAVAAADoXi63P7wmUmrnVGOzbvfRsYYXGGe6MmF4kEOaSGYMwq2LizHfDGwIlLPNKGZfi6gZmggEAy2iVPZEKZqzakq+1xUFFYe90lxTsHmim6HGpvf3eR7skYJ3PC5tyystc0AboFnVXQ9XFJTZIQOYUYFTQEBeWaSVF4bbCeRk1meBJYSL3WbaReMIxQfHXh6jaYXsbEQni6oaF21ERR7l0ksvA0JACH5BAkHAAcALAAAAAAiABUAAAOeeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMfFlA4hTITEMxkIBMOuADwmhzqeM6mashTCXKw2TVKQyKuTRSx2wegnNkyJ1ozpOFiMLqcEU8BZHx6NYW8nVlZefQ1tZgQBAXJIi1eHUTRwi0lhl48QL0sogxaGDhMlUo2gh14fHhcVmnOrrxNqrU9joX21Q0IUElm7DQkAOw==') !important;
width: 34px !important;
height: 21px !important;
border: none !important;
float:right;
margin-top:-7px;
margin-right:-10px;
}
</style>
Google Drive
Gist
Clipboard
Sign in with Wallet
