Class ParallelBulkWriter<TData>

Namespace

Hi.Common.ParallelBulkUtils

Assembly

HiGeom.dll

Parallel bulk writer that efficiently processes and writes data in parallel. The writing data is buffered and processed on a separate task to improve performance. A write that throws is reported and the next batch is written as usual; whatever ends the working task opens the gate Wait() waits on and keeps it open, so no caller waits for a task that is gone, and Dispose() reports instead of throwing.

public class ParallelBulkWriter<TData> : IDisposable where TData : class

Type Parameters

TData

The type of data to write.

Inheritance

object

ParallelBulkWriter<TData>

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

DuplicateUtil.TryDuplicate<TSelf>(TSelf, params object[])

InvokeUtil.SelfInvoke<TSrc>(TSrc, Action<TSrc>)

InvokeUtil.SelfInvoke<TSrc, TDst>(TSrc, Func<TSrc, TDst>)

ColorUtil.GetGloomyColor(object, double, double)

KindNameUtil.KindName(object)

NameUtil.GetSelectionName(object)

StringUtil.GetPropertyStringIfToStringNotOverloaded(object, bool, bool)

LockUtil.Lock(object)

Constructors

ParallelBulkWriter(Action<List<TData>>, int, int, Action<Exception>)

Initializes a new instance of the ParallelBulkWriter<TData> class.

public ParallelBulkWriter(Action<List<TData>> addAllFunc, int writingBufferCap = 131072, int earlyWriteCount = 0, Action<Exception> reportException = null)

Parameters

addAllFunc Action<List<TData>>

The function used to add all data items to the destination.

writingBufferCap int

The capacity of the writing buffer. Default is 1024 * 128.

earlyWriteCount int

Before each write the working task waits until a second has passed since its previous wait ended; waiting for the first item and writing count toward that second. With this set, the wait also ends as soon as this many items are buffered and the next second counts from there, so a fast machine writes smaller batches than a second's worth. 0 (the default) never ends the wait early.

reportException Action<Exception>

Receives an exception a write throws (that batch is dropped, the next is written as usual) and the exception that ends the working task, if one does; null (the default) drops them.

Methods

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

Dispose(bool)

Releases the unmanaged resources used by the ParallelBulkWriter<TData> and optionally releases the managed resources.

protected virtual void Dispose(bool disposing)

Parameters

disposing bool

true to release both managed and unmanaged resources; false to release only unmanaged resources.

Enqueue(TData)

Adds a data item to the writing buffer for processing.

public void Enqueue(TData step)

Parameters

step TData

The data item to add.

Exceptions

ObjectDisposedException

The working task has ended: the writer was disposed, or the task's loop ended on an exception, which was reported.

OperationCanceledException

The working task ended while this call was blocked on a full buffer.

Wait()

Waits for all data items in the buffer to be processed: the working task's wait for the rest of its period is ended, so the buffered items go out now, and begins again once they have. Returns at once when the working task has ended.

public void Wait()