Giter VIP home page Giter VIP logo

cyotek / cyotek.collections.generic.circularbuffer Goto Github PK

View Code? Open in Web Editor NEW
50.0 10.0 21.0 793 KB

The CircularBuffer<T> class is a data structure that uses a single, fixed-size buffer that behaves as if it were connected end-to-end. You can use it as a first-in, first-out collection of objects using a fixed buffer and automatic overwrite support.

License: Other

C# 97.91% Batchfile 2.09%
c-sharp circular-buffers hacktoberfest

cyotek.collections.generic.circularbuffer's Introduction

Icon CircularBuffer<T> Class

Build status NuGet Donate

Demo

The CircularBuffer<T> class is a data structure that uses a single, fixed-size buffer that behaves as if it were connected end-to-end. You can use it as collection of objects with automatic overwrite support and no array resizing or allocations. The design of the buffer allows it to be used as both a first-in, first-out queue, or a first-in, last-out stack.

You can drop the class directly into your projects to use as-is, or reference the assembly. A NuGet package is also available. The class has no external dependencies aside from a reference to System.dll.

View the change log for updates to this library.

Overwrite support

By default, the contents of the buffer automatically wrap, so for example if you create a buffer with a maximum capacity of 10, but then attempt to add 11 items, the oldest item in the buffer will be automatically overwritten.

Alternatively, you can set the AllowOverwrite property to false, in which case attempting to add that eleventh item would throw an exception.

Performance

The internal buffer of the class is created whenever the Capacity property is set. Generally, this means it will be created once for the lifetime of the class, unless for some reason you want to dynamically manipulate the capacity. Internally, CircularBuffer<T> has Head and Tail properties which represent the start and end of the buffer, so as you Put and Get items, these values will be adjusted accordingly. No resizing of buffers or reallocation.

Note: Calling the Clear method currently also reallocates the internal buffer rather than looping all the items and setting them to default(T).

The PeekLast(count), GetLast(count), Get(count) and ToArray methods will all create and return a new array. With the exception of ToArray (as there is already CopyTo), the other methods have overloads that allow you to specify an existing array to be populated.

Using the class

The CircularBuffer<T> mostly acts as a FIFO queue. You can use the Put method to put one or more items into the buffer, and then retrieve one or more items using one of the Get methods. However, you can also use it as FILO stack by using the GetLast method.

Note: When you retrieve an item (or items), references to the items still remain in the internal buffer. The Head and Size properties are adjusted so that you'll never get that item again no matter what methods you call. I'm not sure yet whether that is an acceptable approach, or if I should reset the entry to default(T).

To retrieve the next item without removing it from the buffer, you can use the Peek method. Or, to retrieve (again without removing) the last item in the buffer, you can use PeekLast. To round off peeking, there is also a PeekAt method which can retrieve an item from anywhere in the buffer.

Calling Get, GetLast, Peek or PeekLast on an empty buffer will thrown an exception. You can use IsEmpty to check if these actions will succeed. Similarly, calling Put on a full buffer with overwriting disabled will also throw an exception. You can use IsFull to check if this is the case.

The Size property allows you to see how many items you've added to the buffer. The Capacity property returns the maximum number of items the buffer can hold before the oldest items will be overwritten.

The ToArray method will return all queued items, or you can use CopyTo as a more advanced alternative.

The CircularBuffer<T> class implements IEnumerable<T> and IEnumerable, so you can happily iterate over the items - this won't remove them from the buffer. It also implements ICollection<T> and ICollection although calling ICollection<T>.Remove is not supported and will throw an exception.

Finally, the Clear method will reset the buffer to an empty state.

Although I don't think they'll be needed much in real-world use, the Head property represents the internal index of the next item to be read from the buffer. The Tail property represents the index of the next item to be written.

Examples

This first example creates a CircularBuffer<T>, adds four items, then retrieves the first item. The comments describe how the internal state of the buffer changes with each call.

  CircularBuffer<string> target;
  string firstItem;
  string[] items;

  target = new CircularBuffer<string>(10); // Creates a buffer for storing up to 10 items
  target.Put("Alpha");                     // Head is 0, Tail is 1, Size is 1
  target.Put("Beta");                      // Head is 0, Tail is 2, Size is 2
  target.Put("Gamma");                     // Head is 0, Tail is 3, Size is 3
  target.Put("Delta");                     // Head is 0, Tail is 4, Size is 4

  firstItem = target.Get();                // firstItem is Alpha. Head is 1, Tail is 4, Size is 3
  items = target.ToArray();                // items are Beta, Gamma, Delta. Head, Tail and Size are unchanged.

This second example shows how the buffer will automatically overwrite the oldest items when full.

  CircularBuffer<string> target;
  string firstItem;
  string[] items;

  target = new CircularBuffer<string>(3);  // Creates a buffer for storing up to 3 items
  target.Put("Alpha");                     // Head is 0, Tail is 1, Size is 1
  target.Put("Beta");                      // Head is 0, Tail is 2, Size is 2
  target.Put("Gamma");                     // Head is 0, Tail is 3, Size is 3
  target.Put("Delta");                     // Head is 1, Tail is 1, Size is 3

  firstItem = target.Get();                // firstItem is Beta. Head is 2, Tail is 1, Size is 2
  items = target.ToArray();                // items are Gamma, Delta. Head, Tail and Size are unchanged.

This final example shows how the buffer is unchanged when peeking.

  CircularBuffer<string> target;
  string firstItem;
  string lastItem;

  target = new CircularBuffer<string>(10); // Creates a buffer for storing up to 10 items
  target.Put("Alpha");                     // Head is 0, Tail is 1, Size is 1
  target.Put("Beta");                      // Head is 0, Tail is 2, Size is 2
  target.Put("Gamma");                     // Head is 0, Tail is 3, Size is 3
  target.Put("Delta");                     // Head is 0, Tail is 4, Size is 4

  firstItem = target.Peek();               // firstItem is Alpha. Head, Tail and Size are unchanged.
  lastItem = target.PeekLast();            // lastItem is Delta. Head, Tail and Size are unchanged.

For more examples, see the test class CircularBufferTests as this has tests which cover all the code paths. Except for ICollection.SyncRoot anyway!

Requirements

.NET Framework 2.0 or later.

Pre-built binaries are available via a signed NuGet package containing the following targets.

  • .NET 3.5
  • .NET 4.0
  • .NET 4.5.2
  • .NET 4.6.2
  • .NET 4.7.2
  • .NET 4.8
  • .NET Standard 2.0
  • .NET Standard 2.1
  • .NET Core 2.1
  • .NET Core 2.2
  • .NET Core 3.1

Is there a target not on this list you'd like to see? Raise an issue, or even better, a pull request.

Acknowledgements

The CircularBuffer<T> class was originally taken from Circular Buffer for .NET, however I've fixed a number of bugs and added a few improvements. Unfortunately it didn't occur to me to keep a list of all the bugs I fixed.

Syntax-wise, I don't remember changing any method signatures so they should work the same. I did rename the AllowOverflow property to AllowOverwrite which seems to make more sense to me.

The only thing the original has that this version does not is localization support - the original version read exception messages from a resource file, whereas here they are just string literals.

See CONTRIBUTORS.md for further details of updates to the CircularBuffer<T> class.

License

The code is licensed under the New BSD License (BSD) as per the original source this implementation is based upon. See LICENSE.txt for details.

cyotek.collections.generic.circularbuffer's People

Contributors

cyotek avatar domasm avatar vbfox avatar x0nn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cyotek.collections.generic.circularbuffer's Issues

.PeekLast(count)

It would be interesting to implement a .PeekLast() for a n number of last elements, similiar to .Peek() that you can Peek(n) n elements.

IndexOutOfRangeException (incorrect Head) after using Get(count) to empty Full buffer

I encountered this problem in v1.1.0 after using the Get(int count) overload to read a buffer (that has AllowOverwrite=false) 1 byte at a time when the buffer is completely full. When using the Get<T>() instead, it works fine. The problem seems to be an incorrect Head value after the Get() loop, see comment inside the code.

See this code that shows the problem:

using Cyotek.Collections.Generic;

namespace ConsoleApp12
{
    class Program
    {
        static void Main(string[] args)
        {
            Test(true);
            Test(false);
        }

        const int Size = 10;
        private static void Test(bool mode)
        {
            var buffer = new CircularBuffer<byte>(Size, false);

            var fullSizeInput = new byte[Size];
            buffer.Put(fullSizeInput);

            while (!buffer.IsEmpty)
            {
                if (mode)
                {
                    var val = buffer.Get();
                }
                else
                {
                    var array = buffer.Get(1);
                }
            }

            // At this point, for mode = true, Head=0, Tail=0, Size=0
            // At this point, for mode = false, Head=10 (!), Tail=0, Size=0

            var smallInput = new byte[1];
            buffer.Put(smallInput);

            // This throws and Index out of range exception for mode=false
            var fail = buffer.Peek();
        }

    }
}

Update documentation to note allocations

Setting the Capacity property or calling Get(count), Peek(count), ToArray() and Clear() all cause new allocations, this should be noted in the documentation.

GetEnumerator() reads past Tail

I noticed that GetEnumerator() doesn't seem to be working for me, it looks like it's reading past Tail. Should bufferIndex reset to 0 when either Tail or Capacity is reached, not just Capacity?

    public IEnumerator<T> GetEnumerator()
    {
      int bufferIndex;

      bufferIndex = this.Head;

      for (int i = 0; i < this.Size; i++, bufferIndex++)
      {
        if (bufferIndex == this.Capacity)
        {
          bufferIndex = 0;
        }

        yield return _buffer[bufferIndex];
      }
    }

Question to Skip() behavior

After Skip(int count), should we reduce the element count of the circular buffer as well?

If I understand it correctly, it is intended to 'drop' some elements, which means the actual size of elements should be reduced to reflect that.

Feel free to correct me if I'm wrong.

Usage for "number of messages per x" scenarios

Hi there,

I would like to know if your circular buffer could be used for the following situation:
-provide a count of objects "processed" in the last minute, hour or day
-be thread safe
How large do I size the buffer to address this? 24 * 60 * 60? so one per second?
When a message is processed, send it to the buffer (but send what exactly? the second?)
Queries to the buffer could be from multiple clients, possible at the same time.
Is that possible?
Cheers

Compilation warning with the NuGet package

Hi and thanks for your work!

I installed the NuGet package of your library within a .NET Standard 2.0 project, and the compiler gives me this warning:

Package 'Cyotek.CircularBuffer 1.0.3' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.

immagine

Reading the README.MD it seems that you library is fully compatible with .NET Standard 2.0!

Did I do something wrong?

Thanks!

.NET Standard Support

Whenever you make your next build, please consider migrating to .NET Standard. There is an example here. Thanks!

Enumerate backwards

Hi,

Sorry if I missed this in the tests - is there a way to enumerate backwards? I have data being added, but I'd like to look backwards through the buffer until I find a certain "thing". I still want the buffer to be added on to. I might have 300/400 items in there, but I don't know how far back I might need to go.

Any suggestions on the most efficient way of accomplishing this?

PeekLast(...,count) raises an IndexOutOfRangeException when buffer is overwritten

Hi.
If buffer has room, the method works as expected (there is an unit test in the test suite for this use case).
But if some elements are overwritten, an exception occurs. (not covered in test suite).
In fact, the internal method GetTailIndex(i) is the culprit, producing some negative indexes.
On my side, an additional if (bufferIndex < 0) bufferIndex += _capacity; fixes the index value

  private int GetTailIndex(int index)
    {
      int bufferIndex;

      if (_tail == 0)
      {
        bufferIndex = _size - (index + 1);
      }
      else
      {
        bufferIndex = _tail - (index + 1);
        if (bufferIndex < 0) bufferIndex += _capacity;
      }

      return bufferIndex;
    }

On my side

      var buffer = new CircularBuffer<string>(5);
      buffer.Put(new[] { "a", "b", "c", "d", "e" });
      CollectionAssert.AreEqual(new[] { "d", "e" }, buffer.PeekLast(2));
      CollectionAssert.AreEqual(new[] { "a", "b" }, buffer.Peek(2));
  //overwrite
      buffer.Put(new[] { "f", "g" });
      CollectionAssert.AreEqual(new[] { "e", "f", "g"}, buffer.PeekLast(3));
      CollectionAssert.AreEqual(new[] { "c", "d", "e"}, buffer.Peek(3));

Add T this[int index]

Feature request, could an index operator be added? It'd be nice to be able to use circularBuffer[index]. This would also allow IReadOnlyList to be implemented, if that's desirable.

        public T this[int index]
        {
            get
            {
                if (index < 0 || index >= Size)
                    throw new IndexOutOfRangeException();

                var bufferIndex = Head + index;
                if (bufferIndex >= Capacity)
                    bufferIndex -= Capacity;

                return _buffer[bufferIndex];
            }
        }

Copying Large buffer

I'm currently using the original CircularBuffer for one of my projects. Since it seemed to be "mostly dead" I searched for a follow-up and found this. Good work!
I wonder if for copying large buffers into/out the Circular Buffer it will perhaps be better to call static method Array.Copy, rather than loop on the array members?
What is do you think?

Nuget package ?

I'd love to reference this through a nuget package, would it be possible to create one ? You already have a build running on AppVeyor so it should be quite easy to push packages from there.

Thanks for this implentation, the original on codeplex has indeed quite a few bugs and I was about to fix it when I found this one :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.