Skip to content

proposal: x/exp/mmap: implement io.WriterTo and io.ReadSeeker #56422

Open
golang/exp
#70
@costela

Description

@costela

A naive implementation of io.WriterTo in mmap.ReaderAt seems to bring the following performance improvement when using code that relies on io.Copy (e.g. http.ServeContent).

name        old time/op    new time/op    delta
MmapCopy-8     126µs ± 7%     106µs ±16%  -16.09%  (p=0.000 n=8+10)

name        old speed      new speed      delta
MmapCopy-8  16.9GB/s ± 6%  20.4GB/s ±15%  +20.30%  (p=0.000 n=8+10)

name        old alloc/op   new alloc/op   delta
MmapCopy-8    33.7kB ± 0%     0.2kB ±14%  -99.40%  (p=0.000 n=9+10)

name        old allocs/op  new allocs/op  delta
MmapCopy-8      2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=10+10)
Benchmark code
func BenchmarkMmapCopy(b *testing.B) {
	// mmap some big-ish file; will only work on unix-like OSs.
	f, err := Open("/proc/self/exe")
	if err != nil {
		b.Fatalf("Open: %v", err)
	}

	type sectionReaderWithWriterTo struct {
		*io.SectionReader
		*ReaderAt
	}
	reader := &sectionReaderWithWriterTo{
		SectionReader: io.NewSectionReader(f, 0, int64(f.Len())),
		ReaderAt:      f,
	}

	// Sanity check: ensure we will run into the io.Copy optimization when using the NEW code above.
	var _ io.WriterTo = (*sectionReaderWithWriterTo)(nil)

	buf := &bytes.Buffer{}
	// "Hide" the ReaderFrom interface by wrapping the writer.
	// Otherwise we skew the results by optimizing the wrong side.
	writer := struct{ io.Writer }{buf}

	b.ReportAllocs()
	b.ResetTimer()

	for i := 0; i < b.N; i++ {
		_, _ = reader.Seek(0, io.SeekStart)
		buf.Reset()

		n, _ := io.Copy(writer, reader)
		b.SetBytes(n)
	}
}

This proposal would probably also solve #20642, as suggested.

Update 2022-10-27: since repeated uses of WriteTo() should behave like other readers (i.e.: keep state to avoid re-writing the whole mmap.ReaderAt), this proposal also implies an implementation of io.ReadSeeker.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    • Status

      Incoming

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions