Table of Contents

Class SqliteStepStorage

Namespace
Hi.SqliteUtils
Assembly
HiNc.dll

The step cache: an SQLite file holding one StepLuggage row a step, a column a kind (StepLuggageTableName), and one StepBrief row a published step, a column a brief (StepBriefTableName: the MillingForceBrief and the SeqPhysicsState, the brief-into-DB chain's C3). The play writes whole rows in batches, a step's two rows in one transaction; the step-physics recompute amends what it rewrote, a chunk at a time in one transaction (AmendStepPhysics(IReadOnlyList<StepPhysicsAmendment>): the force column and the briefs' row); every consumer reads what it uses – one kind of one step, one kind over an index range, the whole rows for the recompute and the HardNc NC optimization, the briefs' rows (StepBrief) for the recompute and the NC optimization's solver, or which kinds the rows hold – straight from the file, with no cache in front of it. The step variables written onto the steps (the chain's C4c) are two more tables of the same file: a key table (StepVariableKeyTableName) and a value table, a row a (key, step) (StepVariableTableName); a play's writes land in the consignment's transaction, a mapping's, a training's and a script's batches in one transaction a batch (WriteStepVariables(IReadOnlyList<StepVariableWrite>)), and a reader asks one value of one step, one key over a range, or every key over a range. Two luggage columns are stored in a form of the file's own and decoded back on every read: the no-cut force record as its 8-byte header, the substraction's contours deflated behind the 13-byte header.

public class SqliteStepStorage : IDisposable
Inheritance
SqliteStepStorage
Implements
Inherited Members
Extension Methods

Constructors

SqliteStepStorage(string, ILogger)

Initializes a new instance of the SqliteStepStorage class.

public SqliteStepStorage(string databasePath = null, ILogger logger = null)

Parameters

databasePath string

The path to the SQLite database file. If null, uses the default per-user path (LocalApplicationData/HiNC/StepCache/step_cache.db) — callers hosting several instances under one account must pass a per-instance path.

logger ILogger

Optional logger instance.

Fields

EveryKeyRangeSql

Internal Use Only. The statement of FindStepVariables(int, int): the key table outer (a CROSS JOIN keeps it so), the value table searched by its primary key under each key – a plain join lets the planner scan the whole value table once a segment, there being no index on the step index alone. The tests pin its query plan.

public const string EveryKeyRangeSql = "\r\n\t\t\tSELECT v.StepIndex, k.\"Key\", v.Value\r\n\t\t\tFROM StepVariableKey k CROSS JOIN StepVariable v\r\n\t\t\tON v.KeyId = k.KeyId AND v.StepIndex >= @BeginIndex AND v.StepIndex < @EndIndex"

Field Value

string

LayerMillingEngagementColumn

The LayerMillingEngagement column (its WriteBin bytes).

public const string LayerMillingEngagementColumn = "LayerMillingEngagementData"

Field Value

string

MillingForceBriefColumn

The MillingForceBrief column: the tri-state record WriteBin(BinaryWriter, MillingForceBrief?) writes, the null brief included (its 2-byte record); never NULL.

public const string MillingForceBriefColumn = "MillingForceBriefData"

Field Value

string

MillingForceLuggageColumn

The MillingForceLuggage column: its WriteBin bytes, or the 8-byte header alone for the no-cut record of an untouched step under active physics whose no-cut brief the writer had at hand: a column of 8 bytes is always such a step, such a step not always a column of 8 bytes.

public const string MillingForceLuggageColumn = "MillingForceLuggageData"

Field Value

string

SeqPhysicsStateColumn

The SeqPhysicsState column (its WriteBin bytes); never NULL.

public const string SeqPhysicsStateColumn = "SeqPhysicsStateData"

Field Value

string

StepBriefTableName

The briefs' table: a row a published step, a column a brief; its row type is StepBrief.

public const string StepBriefTableName = "StepBrief"

Field Value

string

StepLuggageTableName

The one table: a row a step, a column a kind of StepLuggage.

public const string StepLuggageTableName = "StepLuggage"

Field Value

string

StepVariableKeyColumn

The key column of StepVariableKeyTableName: the step variable's key, unique.

public const string StepVariableKeyColumn = "Key"

Field Value

string

StepVariableKeyTableName

The key table of the written step variables: a row a key, its KeyId the value table's key.

public const string StepVariableKeyTableName = "StepVariableKey"

Field Value

string

StepVariableTableName

The value table of the written step variables: a row a (key, step), its value REAL (a double) or TEXT (a string), never NULL.

public const string StepVariableTableName = "StepVariable"

Field Value

string

StepVariableValueColumn

The value column of StepVariableTableName.

public const string StepVariableValueColumn = "Value"

Field Value

string

SubstractionColumn

The Substraction column: its WriteBin bytes with the contours deflated behind the 13-byte header, the contour count written negated as the marker; an untouched step's 13 bytes and a touched step's 17 bytes without a contour as they are.

public const string SubstractionColumn = "SubstractionData"

Field Value

string

Properties

CommittedStepIndex

The writer's watermark: the highest step index whose consignment was committed since the file was last reset (InsertStepConsignments(IReadOnlyList<StepConsignment>); -1 before the first batch). The batches commit in step order, so every consignment at or below it is in the file – its luggage, its briefs and the values written onto it – and a reader that pins a segment's written rows reads the watermark before the rows: a step at or below it without a row has no value, while a step above it may have had its rows committed, and its consignment freed, since the read (the brief-into-DB chain's C4c). A batch write of values (WriteStepVariables(IReadOnlyList<StepVariableWrite>)) is no consignment and leaves it. Moves back to -1 with the reset generation.

public int CommittedStepIndex { get; }

Property Value

int

DatabasePath

Gets the database file path.

public string DatabasePath { get; }

Property Value

string

Default

Gets or sets the default SQLite step storage instance.

public static SqliteStepStorage Default { get; set; }

Property Value

SqliteStepStorage

IsDefaultInit

Gets a value indicating whether the default storage has been initialized.

public static bool IsDefaultInit { get; }

Property Value

bool

StepVariableRangeReads

Internal Use Only. The range reads of the value table made so far (the two FindStepVariables): a diagnostic counter, for the tests.

public int StepVariableRangeReads { get; }

Property Value

int

StepVariableRowReads

Internal Use Only. The single-row reads of the value table made so far (TryGetStepVariable(int, string, out object)): a diagnostic counter, for the tests.

public int StepVariableRowReads { get; }

Property Value

int

WrittenStepVariableKeys

The keys the key table holds, in no particular order (a snapshot).

public IReadOnlyCollection<string> WrittenStepVariableKeys { get; }

Property Value

IReadOnlyCollection<string>

Methods

AmendStepPhysics(IReadOnlyList<StepPhysicsAmendment>)

Amends the rows of the steps a step-physics recompute rewrote (StepPhysicsAmendment), in one transaction for the list – a chunk's amendments are committed together or not at all: for every amendment with a force luggage, the force column of the step's luggage row alone is updated (UPDATE, in the column's stored form – the header alone when the amendment's brief is the no-cut record and the array is zeros; the substraction and the engagement columns are not touched or re-serialised, unlike the whole-row InsertStepLuggages(IEnumerable<StepLuggage>); a step without a luggage row gets none – the recompute's pre-scan refuses a force rerun on such a step); for every amendment, the step's StepBrief row is written (INSERT OR REPLACE: the force brief's tri-state record, a null brief the 2-byte null record, and the sequential physics state's record), created for a step that had none. The blobs go through the one buffer of the batch, as the consignments'. The store does not check that the step is published – a row for exactly the published steps is the callers' contract (the recompute hands it published steps; the tests write what they need). The brief-into-DB chain's C4a.

public void AmendStepPhysics(IReadOnlyList<StepPhysicsAmendment> amendments)

Parameters

amendments IReadOnlyList<StepPhysicsAmendment>

The amendments of one chunk.

ClearStepBriefs()

Deletes all StepBrief rows from the database (the luggage rows stay: ClearStepLuggages()).

public void ClearStepBriefs()

ClearStepLuggages()

Deletes all step luggage rows from the database (the briefs' rows stay: ClearStepBriefs()).

public void ClearStepLuggages()

ClearStepVariables()

Deletes every row of the two step-variable tables in one transaction, and the mirrored keys (the luggage and the briefs' rows stay).

public void ClearStepVariables()

CountStepBriefs()

Gets the count of StepBrief rows in the database: the published steps.

public int CountStepBriefs()

Returns

int

The count of records.

CountStepLuggages()

Gets the count of step luggage rows in the database.

public int CountStepLuggages()

Returns

int

The count of records.

CountStepVariables()

Gets the count of value rows (StepVariableTableName).

public int CountStepVariables()

Returns

int

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

public void Dispose()

Dispose(bool)

Releases the resources used by the storage: the pooled connections to the database file, which hold the file open and their page caches.

protected virtual void Dispose(bool disposing)

Parameters

disposing bool

Whether this method is being called from Dispose.

FindLayerMillingEngagements(int, int)

Finds the engagements of the rows within the range; see FindMillingForceLuggages(int, int).

public Dictionary<int, LayerMillingEngagement> FindLayerMillingEngagements(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, LayerMillingEngagement>

The step index to its engagement, null for a NULL column.

FindMillingForceLuggages(int, int)

Finds the milling force luggages of the steps within the specified index range, returning only the step index and the force column: the row's two other blobs are neither copied out nor decoded (SQLite still walks the row's overflow pages to reach the column).

public Dictionary<int, MillingForceLuggage> FindMillingForceLuggages(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, MillingForceLuggage>

The step index to its milling force luggage, null for a NULL column.

Remarks

A step without a row is not in the dictionary; a step whose row holds a NULL force column (physics not active for the step – physics off, no tool, a cutter without flutes, the spindle not turning – or a touching step the force kernel declined, such as one without cutting parameters) maps to null. The two distinctions are kept apart so a caller walking the steps can tell a step that was never written from one that was written without a force.

FindStepBriefs(int, int)

Finds the StepBrief rows within the specified index range, both briefs decoded, by step index ascending – for a scan that reads the store one segment at a time. A step without a row is absent.

public List<StepBrief> FindStepBriefs(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

List<StepBrief>

The rows (StepBrief): the step index, the force brief (null for the null record) and the sequential physics state.

FindStepLuggages(int, int)

Finds the whole step luggage rows within the specified index range, every kind decoded – for a consumer that needs all three (the physics recompute, which reads every kind and writes the force column back through AmendStepPhysics(IReadOnlyList<StepPhysicsAmendment>), and the HardNc NC optimization). A consumer that uses one kind reads that kind alone (FindSubstractions(int, int), FindLayerMillingEngagements(int, int), FindMillingForceLuggages(int, int)).

public List<StepLuggage> FindStepLuggages(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

List<StepLuggage>

The rows the store holds in the range, by step index ascending; a NULL column is a null member.

FindStepVariables(int, int)

Finds every value written on the steps of [begin, end), by step index then by key: a step without a row is absent. Answers empty without a query when no key was ever written. For the inspecting-slot scan and the step-file export, which read every key of a segment of steps at a time: one range search of the primary key under each key (EveryKeyRangeSql).

public Dictionary<int, Dictionary<string, object>> FindStepVariables(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, Dictionary<string, object>>

The values by step index, then by key.

FindStepVariables(string, int, int)

Finds the values written under key on the steps of [begin, end), by step index: a step without a row is absent. A key never written answers empty without a query. For a scan that reads the store one segment at a time (the inspecting-slot scan of a written key, a script's range read).

public Dictionary<int, object> FindStepVariables(string key, int beginIndex, int endIndex)

Parameters

key string

The key.

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, object>

The values by step index, a double or a string each.

FindStoredStepBriefIndices(int, int)

Finds which steps within the specified index range have a StepBrief row: the step indices alone, no blob copied out or decoded – the step-physics recompute's pre-scan asks it whether the briefs of every step of its chain are stored, one chunk at a time, before it touches anything (a store that lost its rows since the play – reset by another instance, cleared – cannot be the chain's input).

public HashSet<int> FindStoredStepBriefIndices(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

HashSet<int>

The step indices of the rows the store holds in the range.

FindStoredStepLuggageKinds(int, int)

Finds which kinds of luggage the rows within the specified index range hold: step index to the flags of the row's non-NULL columns, for the steps that have a row. Reads the columns' NULL-ness only – no blob is copied out or decoded (SQLite answers it from the record header) – so a pre-scan that asks “is the engagement of these steps stored?” costs no decoding.

public Dictionary<int, StepLuggageKinds> FindStoredStepLuggageKinds(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, StepLuggageKinds>

Step index to the stored kinds; a step without a row is absent, a row with every column NULL maps to None.

FindSubstractions(int, int)

Finds the substractions of the rows within the range; see FindMillingForceLuggages(int, int).

public Dictionary<int, Substraction> FindSubstractions(int beginIndex, int endIndex)

Parameters

beginIndex int

The beginning index (inclusive).

endIndex int

The ending index (exclusive).

Returns

Dictionary<int, Substraction>

The step index to its substraction, null for a NULL column.

InsertStepConsignments(IReadOnlyList<StepConsignment>)

Inserts a batch of step consignments (StepConsignment): every consignment's StepLuggage row, then, for every consignment with a step (HasStep), its StepBrief row – the force brief's tri-state record (a null brief is the 2-byte null record, never an SQL NULL) and the sequential physics state's record – in one transaction for the batch: a step's two rows are committed together or not at all. INSERT OR REPLACE on both tables, as InsertStepLuggages(IEnumerable<StepLuggage>). A luggage-only consignment (its step's sequential stage faulted before publishing it) gets its luggage row alone. The blobs go through the one buffer of the batch, as the luggage's. The luggage row's two encoded columns are made on this thread from the consignment's bytes: the substraction's contours deflated, the force kind cut to its header when the consignment's brief is the no-cut record and the array is zeros. The values written onto a step while its consignment was live (WrittenValues, the chain's C4c) are rows of the StepVariable table in the same transaction, taken off the consignment once (TakeForWrite()).

public void InsertStepConsignments(IReadOnlyList<StepConsignment> items)

Parameters

items IReadOnlyList<StepConsignment>

The consignments of the batch.

InsertStepLuggages(IEnumerable<StepLuggage>)

Inserts the step luggages into the database, whole rows, one transaction for the batch (INSERT OR REPLACE: a row written again is replaced whole, so a writer that leaves a kind out NULLs that column).

public void InsertStepLuggages(IEnumerable<StepLuggage> items)

Parameters

items IEnumerable<StepLuggage>

The items to insert.

Remarks

The batch serialises every blob through one buffer and copies it out at its exact length. A fresh MemoryStream per blob grew from 256 B by doubling and was copied once more at the end, so a 0.6 MiB substraction cost about 2.6 MiB of allocations, most of them on the large object heap. The bytes are each object's WriteBin output, the substraction's in the column's deflated form; the force column keeps the whole array here – a luggage carries no force brief, and the header-only form needs the brief's no-cut tag beside the zeros (InsertStepConsignments(IReadOnlyList<StepConsignment>), AmendStepPhysics(IReadOnlyList<StepPhysicsAmendment>)).

IsStepVariableKeyWritten(string)

Whether a value was ever written under key since the file was recreated (the key has a row of the key table), read off the mirrored key table: no query.

public bool IsStepVariableKeyWritten(string key)

Parameters

key string

The key.

Returns

bool

Reset()

Resets the storage by deleting and recreating the database file.

public void Reset()

Remarks

Delete-and-recreate instead of DELETE FROM: the per-run cache grows to many GB of step blobs and SQLite never returns freed pages to the file system (VACUUM would rewrite the whole file), so recreating the file is both the fastest wipe and the only practical space reclaim. Callers drain the luggage writer first (ResetStateAndClStrip → WaitAll) and the file is per-instance, so no other writer can be mid-batch. Falls back to the in-place table wipe when the file is still held open (e.g. a concurrent reader mid-query keeps the delete off on Windows). Two resets of one file never interleave: the delete, the fallback wipe and the table creation of one run before the other's.

TryGetLayerMillingEngagement(int, out LayerMillingEngagement)

Reads the engagement of one step's row; see TryGetMillingForceLuggage(int, out MillingForceLuggage).

public bool TryGetLayerMillingEngagement(int stepIndex, out LayerMillingEngagement engagement)

Parameters

stepIndex int

The step index.

engagement LayerMillingEngagement

The stored engagement, null for a NULL column.

Returns

bool

true when the store holds the step's row.

TryGetMillingForceBrief(int, out MillingForceBrief)

Reads the force brief of one published step's StepBrief row: the tri-state record read back (ReadBin(BinaryReader)), null for the null record.

public bool TryGetMillingForceBrief(int stepIndex, out MillingForceBrief brief)

Parameters

stepIndex int

The step index.

brief MillingForceBrief

The brief, null when the physics did not run on the step.

Returns

bool

true when the store holds the step's briefs' row.

TryGetMillingForceLuggage(int, out MillingForceLuggage)

Reads the force luggage of one step's row, selecting that column alone. A single row of one column reads in tens of microseconds (connection from the pool, the per-connection PRAGMAs, the select, the decode), so a step picked in the GUI reads straight from the store.

public bool TryGetMillingForceLuggage(int stepIndex, out MillingForceLuggage luggage)

Parameters

stepIndex int

The step index.

luggage MillingForceLuggage

The stored force luggage, null for a NULL column.

Returns

bool

true when the store holds the step's row.

TryGetSeqPhysicsState(int, out SeqPhysicsState)

Reads the sequential physics state of one published step's StepBrief row.

public bool TryGetSeqPhysicsState(int stepIndex, out SeqPhysicsState brief)

Parameters

stepIndex int

The step index.

brief SeqPhysicsState

The sequential physics state; every published step has one.

Returns

bool

true when the store holds the step's briefs' row.

TryGetStepVariable(int, string, out object)

Reads the value written onto one step under a key: a double (a NaN read back from its raw bytes) or a string (StepVariableValue). A key never written answers no row without a query.

public bool TryGetStepVariable(int stepIndex, string key, out object value)

Parameters

stepIndex int

The step index.

key string

The key.

value object

The value; null without a row.

Returns

bool

true when the store holds the step's row under the key.

TryGetSubstraction(int, out Substraction)

Reads the substraction of one step's row; see TryGetMillingForceLuggage(int, out MillingForceLuggage).

public bool TryGetSubstraction(int stepIndex, out Substraction substraction)

Parameters

stepIndex int

The step index.

substraction Substraction

The stored substraction, null for a NULL column.

Returns

bool

true when the store holds the step's row.

WriteStepVariable(int, string, object)

Writes one value onto one step's row under a key; see WriteStepVariables(IReadOnlyList<StepVariableWrite>).

public void WriteStepVariable(int stepIndex, string key, object value)

Parameters

stepIndex int

The step index.

key string

The key.

value object

The value; null deletes the row.

WriteStepVariables(IReadOnlyList<StepVariableWrite>)

Writes a batch of values onto steps under step-variable keys, in one transaction for the batch (INSERT OR REPLACE; a null value deletes the row; a key gets its key row on its first value, and a delete under a key never written creates none). A value of a type a step cannot hold (StepVariableValue) is passed over: the host reports it.

public void WriteStepVariables(IReadOnlyList<StepVariableWrite> writes)

Parameters

writes IReadOnlyList<StepVariableWrite>

The writes of the batch.