recyclable

package module
v2.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 14, 2026 License: BSD-3-Clause Imports: 4 Imported by: 4

README

recyclable

import "github.com/cognusion/go-recyclable/v2"

Overview

Package recyclable provides the recyclable.BufferPool, which is a never-ending font of recyclable.Buffer, a multiuse buffer that is very reusable, supports re-reading the contained buffer, and when Close()d, will return home to its BufferPool for reuse. Buffer also supports safe concurrent WriteAt and ReadAt.

Since v2, Buffer is a cleaner fusion of a [bytes.Buffer] and [bytes.Reader], with large swaths of code reused from the Go standard library. The resulting Write performance gains are massive.

Migrating from v1 should be quite seamless with two caveats:

- Buffer is now missing ReadRune() and UnreadRune() to simplify accounting, so is no longer an [io.RuneScanner]
- Read operations no longer do a `Seek(0,0)` after every operation, so the caller needs to do so themselves if they wish to re-Read the Buffer again.
Example :

HOWTO implement a goro-safe BufferPool for Buffers

// BufferPool allows us to have a never-ending font of Buffers.
// If the Pool is empty, a new one is created. If there is one someone put
// back, then it is returned. Saves on allocs like crazy. <3
rPool := NewBufferPool()

// Let's grab a Buffer
    rb := rPool.Get()

    // And immediately reset the value, as we can't trust it to be empty
rb.Reset([]byte("Hello World"))

// Unlike most buffers, we can re-read it:
for range 10 {
    if string(rb.Bytes()) != "Hello World" {
        panic("OMG! Can't reread?!!!")
    }
}

// Or get the string value, if you prefer (and know it's safe)
    for range 10 {
        if rb.String() != "Hello World" {
            panic("OMG! Can't reread?!!!")
        }
    }

    // Appending to it as an io.Writer works as well
    io.WriteString(rb, ", nice day?")
    if string(rb.Bytes()) != "Hello World, nice day?" {
        panic("OMG! Append failed?!")
    }

    // Lastly, when you're all done, just close it.
    rb.Close() // and it will go back into the Pool.
    // Please don't use it anymore. Get a fresh one.

    rb = rPool.Get() // See, not hard?
    defer rb.Close() // Just remember to close it, unless you're passing it elsewhere
    rb.Empty()       // Ready to go

    /* HINTS:
    * Makes awesome ``http.Request.Body``s, especially since they get automatically ``.Close()``d when done with
    * Replaces ``bytes.Buffer`` and ``bytes.Reader`` for most uses
    * Isa Stringer and an error
    * As a Writer and a Reader can be used in pipes and elsewhere
      * You can also pipe them to themselves, but that is a very bad idea unless you love watching OOMs
    */

Index

Examples
Package files

buffer.go bufferpool.go

Constants

const (
    // ErrTooLarge is returned when ResetFromLimitedReader is used and the supplied Reader writes too much
    ErrTooLarge = Error("read byte count too large")
    // ErrPointlessClose is returned when a Buffer is Close()d, but has no home to return to.
    // Safely ignorable if you understand what the previous sentence means.
    ErrPointlessClose = Error("closing a Buffer with no home")
    // ErrInvalidWhence is returned when the whence provided to Seek is not one of the io.Seek* constants.
    ErrInvalidWhence = Error("invalid whence")
    // ErrNegativePosition is returned when an index value would be below zero.
    ErrNegativePosition = Error("negative position")
    // ErrNegativeOffset is returned when an offset value would be below zero.
    ErrNegativeOffset = Error("negative offset")
    // ErrBeginning is returned when `UnreadByte` is called, but the index is already at the beginning.
    ErrBeginning = Error("at the beginning")
)

type Buffer

type Buffer struct {
    // contains filtered or unexported fields
}

Buffer is an io.Reader, io.ReadCloser, io.ReaderAt, io.Writer, io.WriterAt, io.WriteCloser, io.WriterTo, io.Seeker, io.ByteScanner, fmt.Stringer, error, and more (see [Test_BufferInterfacesOhMy] for the interfaces we assure we implement)!

It's designed to work in coordination with a BufferPool for recycling, and it's .Close() method puts itself back in the Pool it came from.

It is safe to use outside of a BufferPool, and a referenced zero value can be used organically as well (e.g. b := &Buffer{}).

func NewBuffer
func NewBuffer(home *BufferPool, bytes []byte) *Buffer

NewBuffer returns a Buffer with a proper home. Generally calling BufferPool.Get() is preferable to calling this directly. Calling NewBuffer with a nil home is perfectly safe.

func NewBufferString
func NewBufferString(home *BufferPool, s string) *Buffer

NewBufferString returns a Buffer with a proper home. Generally calling BufferPool.Get() is preferable to calling this directly. Calling NewBufferString with a nil home is perfectly safe.

func (*Buffer) Available
func (r *Buffer) Available() int

Available returns how many bytes are unused in the buffer.

func (*Buffer) AvailableBuffer
func (r *Buffer) AvailableBuffer() []byte

AvailableBuffer returns an empty buffer with b.Available() capacity. This buffer is intended to be appended to and passed to an immediately succeeding [Buffer.Write] call. The buffer is only valid until the next write operation on b.

func (*Buffer) Bytes
func (r *Buffer) Bytes() []byte

Bytes returns a slice of length b.Len() holding the unread portion of the buffer. The slice is valid for use only until the next buffer modification (that is, only until the next call to a method like [Buffer.Read], [Buffer.Write], [Buffer.Reset], or [Buffer.Truncate]). The slice aliases the buffer content at least until the next buffer modification, so immediate changes to the slice will affect the result of future reads.

func (*Buffer) Cap
func (r *Buffer) Cap() int

Cap returns the capacity of the buffer's underlying byte slice, that is, the total space allocated for the buffer's data.

func (*Buffer) Close
func (r *Buffer) Close() error

Close puts itself back into the [BufferPool] it came from. A harmless error is returned if Close is called but no BufferPool was specified at creation. This should absolutely never be called more than once per Buffer life. Implements [io.Closer] et. al.

func (*Buffer) Empty
func (r *Buffer) Empty()

Empty resets the buffer to be empty, but it retains the underlying storage for use by future writes.

func (*Buffer) Error
func (r *Buffer) Error() string

Error returns the contents of the buffer as a string. Implements [error].

func (*Buffer) Grow
func (r *Buffer) Grow(n int)

Grow grows the buffer's capacity, if necessary, to guarantee space for another n bytes. After Grow(n), at least n bytes can be written to the buffer without another allocation. If n is negative, Grow will panic. If the buffer can't grow it will panic with [ErrTooLarge].

func (*Buffer) Len
func (r *Buffer) Len() int

Len returns the number of bytes used in the buffer's underlying slice.

func (*Buffer) Next
func (r *Buffer) Next(n int) []byte

Next returns a slice containing the next n bytes from the buffer, advancing the buffer as if the bytes had been returned by [Buffer.Read]. If there are fewer than n bytes in the buffer, Next returns the entire buffer. The slice is only valid until the next call to a read or write method.

func (*Buffer) Read
func (r *Buffer) Read(b []byte) (n int, err error)

Read implements the [io.Reader] interface.

func (*Buffer) ReadAt
func (r *Buffer) ReadAt(b []byte, off int64) (n int, err error)

ReadAt implements the [io.ReaderAt] interface.

func (*Buffer) ReadByte
func (r *Buffer) ReadByte() (byte, error)

ReadByte implements the [io.ByteReader] interface.

func (*Buffer) ReadBytes
func (r *Buffer) ReadBytes(delim byte) (line []byte, err error)

ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter. If ReadBytes encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often [io.EOF]). ReadBytes returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) ReadFrom
func (r *Buffer) ReadFrom(reader io.Reader) (n int64, err error)

ReadFrom reads data from r until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except [io.EOF] encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with [ErrTooLarge].

func (*Buffer) ReadString
func (r *Buffer) ReadString(delim byte) (line string, err error)

ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often [io.EOF]). ReadString returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) Reset
func (r *Buffer) Reset(b []byte)

Reset resets the Buffer to be reading from b.

func (*Buffer) ResetFromLimitedReader
func (r *Buffer) ResetFromLimitedReader(reader io.Reader, max int64) error

ResetFromLimitedReader performs a [Buffer.Reset] using the contents of the supplied Reader as the new content, up to at most max bytes, returning ErrTooLarge if it's over. The error is not terminal, and the buffer may continue to be used, understanding the contents will be limited

func (*Buffer) ResetFromReader
func (r *Buffer) ResetFromReader(reader io.Reader)

ResetFromReader performs a [Buffer.Reset] using the contents of the supplied Reader as the new content

func (*Buffer) Seek
func (r *Buffer) Seek(offset int64, whence int) (int64, error)

Seek implements the [io.Seeker] interface.

func (*Buffer) Size
func (r *Buffer) Size() int64

Size returns the original length of the underlying byte slice. Size is the number of bytes available for reading.

func (*Buffer) String
func (r *Buffer) String() string

String returns the contents of the buffer as a string, and sets the seek pointer back to the beginning

func (*Buffer) Truncate
func (r *Buffer) Truncate(n int)

Truncate discards all but the first n unread bytes from the buffer but continues to use the same allocated storage. It panics if n is negative or greater than the length of the buffer.

func (*Buffer) UnreadByte
func (r *Buffer) UnreadByte() error

UnreadByte complements [Buffer.ReadByte] in implementing the [io.ByteScanner] interface. If UnreadByte is used outside of normal ByteReading, the result is undefined.

func (*Buffer) Write
func (r *Buffer) Write(p []byte) (n int, err error)

Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with [ErrTooLarge].

func (*Buffer) WriteAt
func (r *Buffer) WriteAt(p []byte, pos int64) (n int, err error)

WriteAt allows for asynchronous writes at various locations within a buffer. Mixing Write and WriteAt is unlikely to yield the results one expects. Implements [io.WriterAt].

func (*Buffer) WriteByte
func (r *Buffer) WriteByte(c byte) error

WriteByte appends the byte c to the buffer, growing the buffer as needed. The returned error is always nil, but is included to match [bufio.Writer]'s WriteByte. If the buffer becomes too large, WriteByte will panic with [ErrTooLarge].

func (*Buffer) WriteRune
func (r *Buffer) WriteRune(p rune) (n int, err error)

WriteRune appends the UTF-8 encoding of Unicode code point r to the buffer, returning its length and an error, which is always nil but is included to match [bufio.Writer]'s WriteRune. The buffer is grown as needed; if it becomes too large, WriteRune will panic with [ErrTooLarge].

func (*Buffer) WriteString
func (r *Buffer) WriteString(s string) (n int, err error)

WriteString appends the contents of s to the buffer, growing the buffer as needed. The return value n is the length of s; err is always nil. If the buffer becomes too large, WriteString will panic with [ErrTooLarge].

func (*Buffer) WriteTo
func (r *Buffer) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements the [io.WriterTo] interface.

type BufferPool

type BufferPool struct {
    // contains filtered or unexported fields
}

BufferPool is a self-managing pool of Buffers

func NewBufferPool
func NewBufferPool() *BufferPool

NewBufferPool returns an initialized BufferPool

func (*BufferPool) Get
func (p *BufferPool) Get() *Buffer

Get will return an existing Buffer or a new one if the pool is empty. REMEMBER to Reset or Empty the Buffer and don't just start using it, as it may very well have old data in it!

func (*BufferPool) Put
func (p *BufferPool) Put(b *Buffer)

Put returns a Buffer to the pool. Generally calling Buffer.Close() is preferable to calling this directly.

type Error

type Error string

Error is an error type

func (Error) Error
func (e Error) Error() string

Error returns the stringified version of Error


Generated by godoc2md

Documentation

Overview

Package recyclable provides the recyclable.BufferPool, which is a never-ending font of recyclable.Buffer, a multiuse buffer that is very reusable, supports re-reading the contained buffer, and when Close()d, will return home to its BufferPool for reuse. Buffer also supports safe concurrent WriteAt and ReadAt.

Since v2, Buffer is a cleaner fusion of a bytes.Buffer and bytes.Reader, with large swaths of code reused from the Go standard library. The resulting Write performance gains are massive.

Migrating from v1 should be quite seamless with two caveats:

  • Buffer is now missing ReadRune() and UnreadRune() to simplify accounting, so is no longer an io.RuneScanner
  • Read operations no longer do a `Seek(0,0)` after every operation, so the caller needs to do so themselves if they wish to re-Read the Buffer again.
Example

HOWTO implement a goro-safe BufferPool for Buffers

// BufferPool allows us to have a never-ending font of Buffers.
// If the Pool is empty, a new one is created. If there is one someone put
// back, then it is returned. Saves on allocs like crazy. <3
rPool := NewBufferPool()

// Let's grab a Buffer
rb := rPool.Get()

// And immediately reset the value, as we can't trust it to be empty
rb.Reset([]byte("Hello World"))

// Unlike most buffers, we can re-read it:
for range 10 {
	if string(rb.Bytes()) != "Hello World" {
		panic("OMG! Can't reread?!!!")
	}
}

// Or get the string value, if you prefer (and know it's safe)
for range 10 {
	if rb.String() != "Hello World" {
		panic("OMG! Can't reread?!!!")
	}
}

// Appending to it as an io.Writer works as well
io.WriteString(rb, ", nice day?")
if string(rb.Bytes()) != "Hello World, nice day?" {
	panic("OMG! Append failed?!")
}

// Lastly, when you're all done, just close it.
rb.Close() // and it will go back into the Pool.
// Please don't use it anymore. Get a fresh one.

rb = rPool.Get() // See, not hard?
defer rb.Close() // Just remember to close it, unless you're passing it elsewhere
rb.Empty()       // Ready to go

/* HINTS:
* Makes awesome ``http.Request.Body``s, especially since they get automatically ``.Close()``d when done with
* Replaces ``bytes.Buffer`` and ``bytes.Reader`` for most uses
* Isa Stringer and an error
* As a Writer and a Reader can be used in pipes and elsewhere
  * You can also pipe them to themselves, but that is a very bad idea unless you love watching OOMs
*/

Index

Examples

Constants

View Source
const (
	// ErrTooLarge is returned when ResetFromLimitedReader is used and the supplied Reader writes too much
	ErrTooLarge = Error("read byte count too large")
	// ErrPointlessClose is returned when a Buffer is Close()d, but has no home to return to.
	// Safely ignorable if you understand what the previous sentence means.
	ErrPointlessClose = Error("closing a Buffer with no home")
	// ErrInvalidWhence is returned when the whence provided to Seek is not one of the io.Seek* constants.
	ErrInvalidWhence = Error("invalid whence")
	// ErrNegativePosition is returned when an index value would be below zero.
	ErrNegativePosition = Error("negative position")
	// ErrNegativeOffset is returned when an offset value would be below zero.
	ErrNegativeOffset = Error("negative offset")
	// ErrBeginning is returned when `UnreadByte` is called, but the index is already at the beginning.
	ErrBeginning = Error("at the beginning")
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Buffer

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer is an io.Reader, io.ReadCloser, io.ReaderAt, io.Writer, io.WriterAt, io.WriteCloser, io.WriterTo, io.Seeker, io.ByteScanner, fmt.Stringer, error, and more (see [Test_BufferInterfacesOhMy] for the interfaces we assure we implement)!

It's designed to work in coordination with a BufferPool for recycling, and it's `.Close()` method puts itself back in the Pool it came from.

It is safe to use outside of a BufferPool, and a referenced zero value can be used organically as well (e.g. `b := &Buffer{}`).

func NewBuffer

func NewBuffer(home *BufferPool, bytes []byte) *Buffer

NewBuffer returns a Buffer with a proper home. Generally calling BufferPool.Get() is preferable to calling this directly. Calling NewBuffer with a `nil` home is perfectly safe.

func NewBufferString

func NewBufferString(home *BufferPool, s string) *Buffer

NewBufferString returns a Buffer with a proper home. Generally calling BufferPool.Get() is preferable to calling this directly. Calling NewBufferString with a `nil` home is perfectly safe.

func (*Buffer) Available

func (r *Buffer) Available() int

Available returns how many bytes are unused in the buffer.

func (*Buffer) AvailableBuffer

func (r *Buffer) AvailableBuffer() []byte

AvailableBuffer returns an empty buffer with b.Available() capacity. This buffer is intended to be appended to and passed to an immediately succeeding Buffer.Write call. The buffer is only valid until the next write operation on b.

func (*Buffer) Bytes

func (r *Buffer) Bytes() []byte

Bytes returns a slice of length b.Len() holding the unread portion of the buffer. The slice is valid for use only until the next buffer modification (that is, only until the next call to a method like Buffer.Read, Buffer.Write, Buffer.Reset, or Buffer.Truncate). The slice aliases the buffer content at least until the next buffer modification, so immediate changes to the slice will affect the result of future reads.

func (*Buffer) Cap

func (r *Buffer) Cap() int

Cap returns the capacity of the buffer's underlying byte slice, that is, the total space allocated for the buffer's data.

func (*Buffer) Close

func (r *Buffer) Close() error

Close puts itself back into the BufferPool it came from. A harmless error is returned if Close is called but no BufferPool was specified at creation. This should absolutely **never** be called more than once per Buffer life. Implements io.Closer et. al.

func (*Buffer) Empty

func (r *Buffer) Empty()

Empty resets the buffer to be empty, but it retains the underlying storage for use by future writes.

func (*Buffer) Error

func (r *Buffer) Error() string

Error returns the contents of the buffer as a string. Implements [error].

func (*Buffer) Grow

func (r *Buffer) Grow(n int)

Grow grows the buffer's capacity, if necessary, to guarantee space for another n bytes. After Grow(n), at least n bytes can be written to the buffer without another allocation. If n is negative, Grow will panic. If the buffer can't grow it will panic with ErrTooLarge.

func (*Buffer) Len

func (r *Buffer) Len() int

Len returns the number of bytes used in the buffer's underlying slice.

func (*Buffer) Next

func (r *Buffer) Next(n int) []byte

Next returns a slice containing the next n bytes from the buffer, advancing the buffer as if the bytes had been returned by Buffer.Read. If there are fewer than n bytes in the buffer, Next returns the entire buffer. The slice is only valid until the next call to a read or write method.

func (*Buffer) Read

func (r *Buffer) Read(b []byte) (n int, err error)

Read implements the io.Reader interface.

func (*Buffer) ReadAt

func (r *Buffer) ReadAt(b []byte, off int64) (n int, err error)

ReadAt implements the io.ReaderAt interface.

func (*Buffer) ReadByte

func (r *Buffer) ReadByte() (byte, error)

ReadByte implements the io.ByteReader interface.

func (*Buffer) ReadBytes

func (r *Buffer) ReadBytes(delim byte) (line []byte, err error)

ReadBytes reads until the first occurrence of delim in the input, returning a slice containing the data up to and including the delimiter. If ReadBytes encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadBytes returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) ReadFrom

func (r *Buffer) ReadFrom(reader io.Reader) (n int64, err error)

ReadFrom reads data from r until EOF and appends it to the buffer, growing the buffer as needed. The return value n is the number of bytes read. Any error except io.EOF encountered during the read is also returned. If the buffer becomes too large, ReadFrom will panic with ErrTooLarge.

func (*Buffer) ReadString

func (r *Buffer) ReadString(delim byte) (line string, err error)

ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim.

func (*Buffer) Reset

func (r *Buffer) Reset(b []byte)

Reset resets the Buffer to be reading from b.

func (*Buffer) ResetFromLimitedReader

func (r *Buffer) ResetFromLimitedReader(reader io.Reader, max int64) error

ResetFromLimitedReader performs a Buffer.Reset using the contents of the supplied Reader as the new content, up to at most max bytes, returning ErrTooLarge if it's over. The error is not terminal, and the buffer may continue to be used, understanding the contents will be limited

func (*Buffer) ResetFromReader

func (r *Buffer) ResetFromReader(reader io.Reader)

ResetFromReader performs a Buffer.Reset using the contents of the supplied Reader as the new content

func (*Buffer) Seek

func (r *Buffer) Seek(offset int64, whence int) (int64, error)

Seek implements the io.Seeker interface.

func (*Buffer) Size

func (r *Buffer) Size() int64

Size returns the original length of the underlying byte slice. Size is the number of bytes available for reading.

func (*Buffer) String

func (r *Buffer) String() string

String returns the contents of the buffer as a string, and sets the seek pointer back to the beginning

func (*Buffer) Truncate

func (r *Buffer) Truncate(n int)

Truncate discards all but the first n unread bytes from the buffer but continues to use the same allocated storage. It panics if n is negative or greater than the length of the buffer.

func (*Buffer) UnreadByte

func (r *Buffer) UnreadByte() error

UnreadByte complements Buffer.ReadByte in implementing the io.ByteScanner interface. If UnreadByte is used outside of normal ByteReading, the result is undefined.

func (*Buffer) Write

func (r *Buffer) Write(p []byte) (n int, err error)

Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with ErrTooLarge.

func (*Buffer) WriteAt

func (r *Buffer) WriteAt(p []byte, pos int64) (n int, err error)

WriteAt allows for asynchronous writes at various locations within a buffer. Mixing Write and WriteAt is unlikely to yield the results one expects. Implements io.WriterAt.

func (*Buffer) WriteByte

func (r *Buffer) WriteByte(c byte) error

WriteByte appends the byte c to the buffer, growing the buffer as needed. The returned error is always nil, but is included to match bufio.Writer's WriteByte. If the buffer becomes too large, WriteByte will panic with ErrTooLarge.

func (*Buffer) WriteRune

func (r *Buffer) WriteRune(p rune) (n int, err error)

WriteRune appends the UTF-8 encoding of Unicode code point r to the buffer, returning its length and an error, which is always nil but is included to match bufio.Writer's WriteRune. The buffer is grown as needed; if it becomes too large, WriteRune will panic with ErrTooLarge.

func (*Buffer) WriteString

func (r *Buffer) WriteString(s string) (n int, err error)

WriteString appends the contents of s to the buffer, growing the buffer as needed. The return value n is the length of s; err is always nil. If the buffer becomes too large, WriteString will panic with ErrTooLarge.

func (*Buffer) WriteTo

func (r *Buffer) WriteTo(w io.Writer) (n int64, err error)

WriteTo implements the io.WriterTo interface.

type BufferPool

type BufferPool struct {
	// contains filtered or unexported fields
}

BufferPool is a self-managing pool of Buffers

func NewBufferPool

func NewBufferPool() *BufferPool

NewBufferPool returns an initialized BufferPool

func (*BufferPool) Get

func (p *BufferPool) Get() *Buffer

Get will return an existing Buffer or a new one if the pool is empty. REMEMBER to Reset or Empty the Buffer and don't just start using it, as it may very well have old data in it!

func (*BufferPool) Put

func (p *BufferPool) Put(b *Buffer)

Put returns a Buffer to the pool. Generally calling Buffer.Close() is preferable to calling this directly.

type Error

type Error string

Error is an error type

func (Error) Error

func (e Error) Error() string

Error returns the stringified version of Error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址