Technical Architecture, Mathematical Foundations, Embedded-AI Design, API Use, Operational Patterns, and Hardening Record
System: Staqtapp-1Xq
Distribution version: 2.2.5
Hardening designation: 2.2.5H
Maturity: Hardened Beta
Document scope: Architecture and programmer paper for the GitHub Gist / repository documentation surface
Staqtapp executes. 1Xq reasons. Improvement proposes. Reality confirms.
Staqtapp-1Xq is an intelligent storage engine for Python that combines an authoritative typed-storage execution plane with a bounded, deterministic, evidence-governed reasoning subsystem. The design does not attach an unrestricted model to a database and does not permit an AI component to mutate storage merely because it produced a high-scoring recommendation. Instead, it treats intelligence as a separately governed subsystem whose outputs must remain reproducible, classified by provenance, constrained by budgets, fenced by generation and process identity, and verified against real operational evidence before promotion can be considered.
The system uses integer-only Q16 projections, deterministic signatures, bounded arrays and queues, fixed ordering rules, immutable records, event-sourced governed state, authenticated journals, process fencing, scoped activation directives, paired pressure trials, counterfactual evaluation, semantic geometry, phase and harmonic analysis, deterministic clustering, computational memory, property lineage, and explicit evidence classes. These mechanisms form a governed feedback loop:
- Staqtapp executes real storage operations.
- 1Xq observes bounded records derived from those operations.
- Mathematical subsystems project the records into reproducible finite representations.
- Reasoning subsystems classify patterns and produce advisory candidates.
- Governance admits, scopes, expires, and fences any candidate activation.
- Real operation receipts and sealed evidence determine whether the candidate was beneficial.
- The storage plane—not the reasoning subsystem—remains the final authority.
This paper describes the architecture, mathematics, API use, embedded-AI design, safety boundaries, practical programming patterns, operational advice, failure semantics, and the evidence supplied by the 2.2.5H hardening campaign. It intentionally describes a hardened Beta, not a production-stability claim or a public-release announcement.
- Product Identity and Design Thesis
- System Architecture
- Authority Model
- Storage Execution Plane
- Mathematical Foundations
- Embedded-AI Subsystem
- Evidence, Provenance, and Promotion
- Governed Activation
- Public API and Programming Model
- Practical Code Patterns
- Tips, Tricks, and Performance Guidance
- Failure Semantics and Recovery
- Concurrency, Scheduling, and Process Fencing
- Operations Console and Observability
- Security and Hardening Record
- Known Boundaries and Non-Claims
- Engineering Interpretation
- API Quick Index
- Glossary
Staqtapp-1Xq is its own intelligent-storage-engine product. It is not Staqtapp-1X, not Staqtapp-TDS, and does not inherit compatibility, migration, protocol, or telemetry claims from those products.
Its defining architectural decision is the separation of execution authority from reasoning capability.
A conventional “AI database” design may place a model beside a storage API and allow model output to directly choose indexes, rewrite data, alter retention, or route requests. That creates a difficult circular trust problem: the mechanism evaluating the system may also be able to change the system before its claims are verified.
Staqtapp-1Xq rejects that arrangement. The engine distinguishes four statements:
- Execution happened. This is established by the authoritative storage plane and its receipts.
- A mathematical observation was produced. This is a deterministic projection of supplied source records.
- A reasoning subsystem proposed a change. This is advisory output.
- The change improved reality. This requires verified real-operation evidence.
No one statement substitutes for another.
Typed reads, writes, persistence, transactions, integrity checks, revisions, and recovery are controlled by Staqtapp. Embedded intelligence does not acquire mutation authority merely by being local or deterministic.
Every major subsystem uses explicit record limits, candidate limits, dimension limits, stage limits, or operation budgets. Boundedness is not only a denial-of-service defense; it makes runtime cost and replay behavior more understandable.
Ordering, tie-breaking, signatures, integer arithmetic, canonical encoding, and immutable records are used so that the same admitted inputs produce the same result. The term “learning” therefore means bounded pattern retention and candidate evolution, not unconstrained online parameter mutation.
A metric without provenance is not evidence. Records are classified as REALITY, SHADOW, SYNTHETIC, or DEMO. Derived fingerprints and stable signatures are useful for grouping and replay, but are not promoted to proof.
A candidate may be mathematically attractive and still remain in shadow. It must survive admission rules, scoped activation, expiration, fencing, receipt binding, real validation, and conservative decision semantics.
Ordinary storage failures use a false-valued CallFailure result so an application can continue its control loop. Governance and integrity boundaries may still raise typed exceptions where continued execution would violate authority or trust guarantees.
The system can be understood as two planes connected by controlled evidence channels.
flowchart TB
APP[Application] --> API[Staqtapp Public API]
API --> STORAGE[Authoritative Storage Execution Plane]
STORAGE --> PERSIST[Typed Persistence / Revisions / Integrity]
STORAGE --> RECEIPTS[Operation Receipts]
STORAGE --> OBS[Bounded Source Observations]
OBS --> MATH[Discrete Mathematical Projection]
MATH --> MEM[Computational Memory]
MATH --> GEO[Geometry / Phase / Harmonic / Cluster Analysis]
MEM --> REASON[Governed Reasoning Kernel]
GEO --> REASON
REASON --> PROPOSAL[Advisory Improvement Proposal]
PROPOSAL --> GOV[Admission / Scope / Budget / Expiry]
GOV --> ROUTER[Deterministic Activation Router]
ROUTER --> GATEWAY[Scoped Storage Execution Gateway]
GATEWAY --> STORAGE
RECEIPTS --> VERIFY[Evidence Verifier]
VERIFY --> REALITY[Sealed REALITY Evidence]
REALITY --> EVOLVE[Property Evolution Decision]
EVOLVE -. governed candidate only .-> GOV
JOURNAL[Authenticated Journals] --- GOV
FENCE[Process Fencing Tokens] --- ROUTER
STATE[Event-Sourced Governed State] --- GOV
CONSOLE[Operations Console] -. observes .-> STATE
The storage plane provides:
- VFS creation and selection;
- typed value encoding and decoding;
- immediate or explicitly batched writes;
- atomic mutation-only transactions;
- bounded range and payload reads;
- read-index rebuild and inspection;
- revisions and rollback;
- integrity verification and map rebuilding;
- compaction, optimization, repair, and recovery;
- diagnostic event summaries.
It is the only layer that turns an operation request into durable state.
This plane accepts finite source bundles representing actual subsystem observations. Its public aggregator consumes five source families:
| Source family | Example signals | Resulting projection |
|---|---|---|
| Interval geometry | density, fragmentation, merges, splits, occupancy, depth, locality | geometry bins and aggregate measures |
| Computational memory | layer, age, bytes compacted, reuse, promotion, eviction | layer distribution and retention measures |
| Pressure trials | latency, memory, recovery score, outcome | ordered pressure curve and percentiles |
| Evolution lineage | generation, parent, disposition, rollback availability, exposure | lineage marks and rollout measures |
| Intelligence ledger | cycle identity, parents, category, evidence class | relationship and class distribution |
The output is a frozen MathematicalAggregation containing bins, measures, provenance metadata, truncation state, record counts, and a deterministic signature.
The intelligence plane contains multiple specialized subsystems rather than one opaque model:
- semantic geometry;
- phase-space and phase-coupling analysis;
- harmonic and predictive-harmonic analysis;
- curvature and trajectory analysis;
- deterministic cluster formation and cluster search recommendation;
- computational memory and classification retention;
- intent validation and capability evaluation;
- a bounded reasoning kernel;
- meta-context finding and proposal generation;
- counterfactual replay;
- property transformation and lineage;
- pressure evaluation;
- governed activation and real validation;
- intelligence ledger and evidence provenance.
The components produce structured records with explicit reason codes and signatures. They do not silently execute arbitrary code.
Authority in 1Xq is deliberately non-transitive.
A deterministic result is not automatically authoritative. A signed record is not automatically true. A stable signature is not evidence. A synthetic benchmark is not a real workload. A proposal is not an activation. An activation is not proof of improvement.
Observation
↓
Deterministic projection
↓
Classification / finding
↓
Advisory proposal
↓
Admitted candidate generation
↓
Scoped, expiring, fenced directive
↓
Real storage operation
↓
Operation receipt
↓
Verified REALITY evidence
↓
Conservative promotion consideration
Each transition requires a stronger contract.
The mathematical API exposes:
class MathematicalDataClass(str, Enum):
REALITY = "REALITY"
SHADOW = "SHADOW"
SYNTHETIC = "SYNTHETIC"
DEMO = "DEMO"These labels prevent a common failure mode: combining test, simulated, shadow, and real observations into a single undifferentiated score.
A REALITY label alone is not enough for evidence qualification. In the public mathematical source bundle, evidence_qualified=True is valid only when:
- the data class is
REALITY; and - at least one evidence identifier is present.
This is an important distinction between real-origin data and verified evidence.
Stable signatures provide deterministic grouping, replay identity, ordering support, and tamper-sensitive linkage when used inside authenticated structures. They do not independently establish:
- correctness;
- causal attribution;
- receipt validity;
- evidence authority;
- promotion eligibility.
Treat signatures as identity and integrity primitives, not as a substitute for provenance.
A minimal storage session is:
from pathlib import Path
import staqtapp
staqtapp.configure(
storage_dir=Path("./data"),
write_batching=False,
)
created = staqtapp.makevfs("accounts", "Production", "Primary")
if isinstance(created, staqtapp.CallFailure):
raise RuntimeError(created.message)The package root is a broad compatibility facade. New governed integrations should prefer explicit imports from staqtapp.public_api for governance and mathematical contracts.
Use set_value() and get_value() for normal typed storage:
record = {
"name": "Rob",
"active": True,
"balance": 500,
"tags": ["beta", "governed"],
}
write_result = staqtapp.set_value("user_1001", record)
if isinstance(write_result, staqtapp.CallFailure):
raise RuntimeError(write_result.message)
loaded = staqtapp.get_value("user_1001")
if isinstance(loaded, staqtapp.CallFailure):
raise KeyError(loaded.message)
assert loaded["balance"] == 500Variable names follow [A-Za-z_][A-Za-z0-9_]*. Validate names before generating them from external identifiers.
Use validate_value() or encode_value() when an application needs to check representability before entering a transaction or write queue.
candidate = {"score": 123, "accepted": False}
valid = staqtapp.validate_value(candidate)
if isinstance(valid, staqtapp.CallFailure):
raise ValueError(valid.message)This is useful at API boundaries because it separates serialization errors from persistence errors.
run_transaction() is single-VFS, mutation-only, read-your-own-writes, and all-or-nothing.
result = staqtapp.run_transaction([
("set_value", ("balance", 400)),
("set_value", ("status", "paid")),
("removevar", ("temporary",)),
])
if not result:
print(result.failure.error_type, result.failure.message)Use a transaction when multiple names collectively represent one logical state transition. Do not emulate atomicity by issuing independent set_value() calls and hoping they all succeed.
Immediate durability is the default. Adaptive write batching introduces a deliberate distinction between accepted into the queue and durably flushed.
staqtapp.configure(
write_batching=True,
batch_max_operations=100,
batch_max_wait_ms=5,
)
receipt = staqtapp.set_value("event_1", {"type": "login"})
flush = staqtapp.flush_writes()
if not flush:
raise RuntimeError("queued writes did not cross the durability barrier")A returned queued receipt should not be described to upstream code as durable until the flush barrier succeeds.
Avoid loading a large value when the caller only needs a bounded region.
staqtapp.set_value("blob", "x" * 8192)
staqtapp.rebuild_read_index()
chunk = staqtapp.read_payload_range("blob", start=4096, length=1024)For streaming patterns, inspect iter_value(), read_range(), and read_payload_range() rather than repeatedly decoding the complete value.
report = staqtapp.verify_integrity(deep=True)
if isinstance(report, staqtapp.CallFailure):
raise RuntimeError(report.message)
recent = staqtapp.list_revisions(limit=10)
staqtapp.compact_vfs(keep_revisions=32)The operational rule is: verify before repair. Repair should use a declared valid source and should not invent an unrecorded state.
The embedded intelligence is built primarily from finite discrete mathematics and fixed-point integer arithmetic. This is not an aesthetic choice. It reduces floating-point platform drift, makes replay easier, and supports clear bounds.
Many normalized values use an unsigned 16-bit range:
[ Q = {0,1,2,\ldots,65535} ]
A conceptual real value (x \in [0,1]) is represented approximately as:
[ q(x) = \operatorname{round}(65535x) ]
and converted back conceptually by:
[ \hat{x} = \frac{q}{65535} ]
In implementation, arithmetic remains integer-only. For example, a weighted contribution can be computed as:
[ c = \left\lfloor\frac{s \cdot w}{65535}\right\rfloor ]
where both score (s) and weight (w) are Q16 integers.
- finite and validation-friendly domain;
- deterministic integer arithmetic;
- compact representation;
- adequate resolution for ranking and normalized telemetry;
- no NaN, infinity, or architecture-specific floating-point corner cases;
- simple saturation at 0 and 65,535.
Q16 values are not probabilities unless a specific API contract says they are. The same range is also used for density, proximity, confidence, recovery, exposure, weights, and normalized scores.
Many internal analyses clamp a value to the Q16 domain:
def saturate_q16(value: int) -> int:
return min(65_535, max(0, int(value)))Saturation avoids wraparound. It also makes the loss of information explicit: once a result reaches the maximum, downstream consumers know it is at the representational ceiling.
The public mathematical aggregator uses an integer nearest-rank-like index:
[ i = \left\lfloor\frac{(n-1)p + 50}{100}\right\rfloor ]
for percentile (p), after sorting the finite input sequence. The selected value is ordered[i].
This gives deterministic p50, p95, or p99 extraction without interpolation or floating-point rounding rules.
A source sequence of length (n) can be projected into (b) bins using:
[ i_k = \left\lfloor\frac{k(n-1)}{\max(1,b-1)}\right\rfloor, \quad k \in {0,\ldots,b-1} ]
The selected sequence values may then be normalized against a peak and saturated into Q16. The important property is that the same ordered input and bin count produce the same sampled projection.
The semantic-geometry subsystem uses a bounded weighted Manhattan metric. For feature vectors (x,y \in Q^d) with weights (w_i \in Q):
[ D(x,y) = \operatorname{sat}{Q}\left( \sum{i=1}^{d} \left\lfloor\frac{w_i |x_i-y_i|}{65535}\right\rfloor \right) ]
This metric is attractive for governed analysis because it is:
- non-negative;
- symmetric;
- zero for identical coordinates;
- monotonic with coordinate separation;
- compatible with triangle-inequality reasoning before saturation;
- integer-only;
- explainable per dimension.
Neighborhood membership is a radius test:
[ y \in N_r(x) \iff D(x,y) \le r ]
A proximity measure can be expressed as a saturated inverse distance within the chosen scale. Density is derived from finite neighbor counts and/or evidence weights. Increasing the radius cannot remove an existing neighbor.
Deterministic cluster formation avoids nondeterministic initialization. Families and members are sorted by stable identities and signatures; ties are broken by explicit lexical or numeric order.
A cluster result therefore represents:
- a finite member set;
- a family or cluster signature;
- bounded outlier information;
- reason bits explaining inclusion or exclusion;
- stable ordering.
The goal is not to claim optimal clustering in the statistical sense. The goal is reproducible structural grouping suitable for governed recommendation.
Phase analysis models subsystem state as finite coordinates and transitions. A phase point may encode normalized pressure, occupancy, recovery, locality, confidence, or other bounded measures. A trajectory is an ordered sequence:
[ \tau = (x_0, x_1, \ldots, x_T) ]
A transition can be represented as:
[ \Delta_t = x_t - x_{t-1} ]
with signed or direction-coded components. Phase coupling then examines whether two bounded sequences move together under explicit lag and support limits.
This type of mathematics is useful for questions such as:
- Does fragmentation rise when write pressure rises?
- Does recovery strength improve after a property generation changes?
- Are two signals merely correlated because they share a derived fingerprint?
- Is a pattern repeated across enough independent cycles to be admitted?
The system treats correlation as an advisory contribution, not causal proof.
Harmonic subsystems look for repeated finite motifs or periodic structure in bounded sequences. They are suited to identifying recurring operational regimes—for example, cyclic pressure or periodic retention behavior.
Predictive harmonic output remains a forecast-like advisory record. It must not be interpreted as guaranteed future behavior. A useful mental model is:
[ \text{prediction} = f(\text{bounded history}, \text{period candidates}, \text{confidence rules}) ]
where f is deterministic and does not mutate storage.
For three ordered scalar or vector points, a discrete second difference captures change in direction:
[ \kappa_t = x_{t+1} - 2x_t + x_{t-1} ]
A positive, negative, or near-zero curvature classification can identify acceleration, reversal, or stability in a metric. In 1Xq, curvature and trajectory records are bounded observations, not continuous differential-geometric claims.
Structured records are canonicalized and fed into deterministic signature functions. Canonical encoding prevents semantically identical mappings with different key order from producing unrelated identities.
A stable signature commonly binds:
- schema or domain version;
- record identity;
- ordered fields;
- parent signatures;
- reason codes;
- generation and sequence;
- provenance class.
Versioned domains are essential. The same digest algorithm used for two semantic purposes should not imply that the signed objects are interchangeable.
The phrase “embedded AI” in Staqtapp-1Xq refers to a collection of bounded reasoning and adaptation mechanisms integrated with the storage lifecycle. It does not mean that a general-purpose language model has unrestricted access to the storage engine.
The public mathematical hook accepts typed observations and creates a deterministic projection:
from staqtapp.public_api import (
BoundedMathematicalAggregator,
ComputationalMemoryObservation,
IntervalGeometryObservation,
MathematicalDataClass,
MathematicalSourceBundle,
PressureTrialObservation,
)
source = MathematicalSourceBundle(
interval_geometry=(
IntervalGeometryObservation(
interval_density_q16=32_000,
fragmentation_q16=4_100,
merge_count=8,
split_count=2,
occupancy_q16=48_000,
depth=4,
locality_q16=52_000,
),
),
computational_memory=(
ComputationalMemoryObservation(
layer=1,
age_cycles=6,
compacted_bytes=4096,
reuse_count=9,
promoted=False,
evicted=False,
),
),
pressure_trials=(
PressureTrialObservation(
trial_ordinal=1,
latency_ns=240_000,
memory_bytes=8192,
recovery_q16=61_000,
stage_outcome="accepted",
),
),
data_class=MathematicalDataClass.REALITY,
source_ids=("runtime-sample:17",),
evidence_qualified=False,
)
projection = BoundedMathematicalAggregator(
max_records_per_source=256
).aggregate(source, bins=8)The returned aggregation includes:
geometry_bins;memory_layers;pressure_curve;evolution_marks;ledger_distribution;- structured
measures; - source and evidence identifiers;
- evidence qualification;
- truncation state;
- per-source record counts;
- stable signature.
Computational memory stores bounded classification records, not arbitrary unbounded conversation history. Its design concerns include:
- memory class;
- layer;
- age;
- reuse count;
- compaction;
- promotion;
- eviction;
- finite registry limits;
- mapped read views where appropriate.
This permits the reasoning subsystem to retain useful classifications while enforcing explicit resource ceilings and lifecycle policy.
The operation-intent layer defines what is being requested before alternatives are ranked. An intent carries:
- operation type;
- payload class;
- resource bounds;
- constraints;
- evidence requests;
- stable intent signature.
The reasoning kernel first validates the intent and evaluates capabilities. Only then does it consider alternatives.
This ordering matters. Ranking an action that was never authorized or cannot be performed is wasted work and a governance hazard.
The unified reasoning kernel is deterministic and non-executing. Conceptually:
validate intent
↓
evaluate capability
↓
enforce candidate bound
↓
reject candidates missing capabilities
↓
add bounded correlation contribution when admitted
↓
sort by score, then stable identity
↓
return plan requiring Staqtapp authorization
A representative score is:
[ S(a) = \min\left(65535, B(a) + \left\lfloor\frac{C(a)W_c}{65535}\right\rfloor\right) ]
where:
- (B(a)) is the alternative’s base score;
- (C(a)) is admitted correlation strength;
- (W_c) is the configured correlation weight.
The plan explicitly records execution_authority="staqtapp_authorization_required".
The meta-context kernel examines completed reasoning-cycle evidence and emits bounded findings and improvement proposals. It uses stages such as:
observe → classify → interpret → propose → simulate → shadow → validate → promote
A proposal contains:
- subject and property boundary;
- candidate generation;
- expected gain;
- stability;
- source finding identities;
- a finite transformation;
- stable proposal signature;
- explicit advisory authority.
The proposal contract denies active-state mutation, history rewriting, and self-granted execution authority.
Counterfactual scenarios compare baseline and candidate outcomes using synthetic or replayed conditions. Their purpose is to reject obviously harmful candidates before real exposure.
Counterfactual output is not REALITY evidence. Even a consistently superior replay result cannot alone authorize promotion.
The property transformer creates typed candidate generations within declared field and candidate limits. Evolution records preserve:
- parent generation;
- candidate generation;
- transformation identity;
- activation scopes;
- real validation outcomes;
- rollback availability;
- disposition and reasons.
This turns “learning” into auditable lineage rather than in-place mutation of hidden parameters.
The pressure engine evaluates candidates under staged workloads. The hardened design includes paired AB/BA trials to reduce order bias:
trial 0: candidate → baseline
trial 1: baseline → candidate
trial 2: candidate → baseline
trial 3: baseline → candidate
If invocation order itself introduces a benefit or penalty, alternating order exposes the effect. Conservative semantics can return an inconclusive result when variance is excessive rather than forcing a winner.
The intelligence ledger links cycles, evidence, outcomes, generations, and replay requests. It enables questions such as:
- Which evidence contributed to this proposal?
- Which parent cycles preceded this classification?
- Was the outcome authoritative or synthetic?
- Which generation produced the observed result?
- Has this evidence already been consumed?
The ledger is part of traceability, not an execution engine.
The public governance boundary exposes types including:
EvidenceSubmission;EvidenceBinding;EvidenceProvenanceEnvelope;EvidenceVerifier;VerifiedEvidence;EvidenceTrustStore;ConsumedEvidenceStore;TrustedCollector;VerificationStatus.
A robust evidence record should bind the collector, operation identity, activation receipt, payload digest, sequence or timestamp domain, and evidence class.
Durable anti-replay storage prevents the same valid evidence object from being consumed repeatedly to inflate confidence or satisfy multiple promotion gates.
The logical rule is:
[ \operatorname{consume}(e) = \begin{cases} \text{accepted}, & e \notin C \ \text{rejected}, & e \in C \end{cases} ]
where (C) is the durable set of consumed evidence identities.
Evidence should refer to the exact operation receipt produced under a scoped activation. Without receipt binding, a later unrelated success could be incorrectly credited to a candidate.
Authenticated append-only journals support:
- tamper detection;
- offline verification;
- key rotation;
- ordered administrative history;
- process-safe scheduling and governed-state reconstruction.
A journal verifies integrity and origin under the configured trust model. It does not prove that the semantic claim inside an honestly signed record is true; that remains the verifier’s responsibility.
Activation is the narrow bridge between advisory intelligence and real storage behavior.
A directive should be treated as an immutable capability-like record. It is expected to bind:
- candidate and baseline generation;
- selection rule;
- operation or scope identity;
- activation budget;
- issue and expiry boundaries;
- fencing token;
- authority or control information.
ActivationRouter consumes an already-authorized directive and deterministically selects a baseline or candidate generation. It does not reason and does not execute storage.
This separation is essential:
reasoner proposes
≠
router selects under directive
≠
gateway executes scoped operation
ScopedStorageExecutionGateway invokes only the storage operation admitted by the directive and emits a receipt. A gateway should reject:
- wrong operation identity;
- expired directive;
- exhausted budget;
- stale fencing token;
- generation mismatch;
- scope mismatch;
- reused authorization where single-use is required.
Activation creates controlled exposure. Promotion is a later governance decision based on real validated outcomes. A candidate can be activated and then rolled back, rejected, or left inconclusive.
The explicitly supported boundary is staqtapp.public_api. The package root remains a compatibility facade and exports a broader surface.
configureopen_vfsmakevfssetpathverify_vfsmigrate_vfsrecover_vfs
set_valueget_valueinspect_valuevalidate_valueencode_valueiter_valueread_rangeread_payload_range
run_transactionrun_vfs_transactioninvoke_apimap_api_callsinvoke_vfs_apimap_vfs_api_calls
list_revisionsrollback_revisionprune_revisionsrevision_storage_report
read_index_inforebuild_read_index
flush_writespending_writeswrite_batching_info
verify_integrityintegrity_reportrebuild_integrity_mapoptimize_vfscompact_vfsrepair_vfs
recent_eventsdiagnostic_countsjson_backend_info
ActivationControlActivationDirectiveActivationReceiptActivationRouterOperationIdentityScopedStorageExecutionGatewaySelectionReasonSelectionRule
EvidenceBindingEvidenceClassEvidenceFailureEvidenceProvenanceEnvelopeEvidenceSubmissionEvidenceTrustStoreEvidenceVerifierTrustedCollectorVerifiedEvidenceVerificationStatusConsumedEvidenceStore
BoundedMathematicalAggregatorMathematicalAggregationMathematicalSourceBundleMathematicalDataClassMathematicalMeasureIntervalGeometryObservationComputationalMemoryObservationPressureTrialObservationEvolutionLineageObservationLedgerCycleObservation
GovernedStateRepositoryGovernedStateEventReconstructedGovernedStateLifecycleStateGovernedRuntimeSchedulerGovernedScheduledJobGovernedSchedulerSnapshotGovernedSchedulingPolicyGovernedResourceSnapshot
AuthenticatedJournalJournalKeyringJournalVerificationReportFencingLeaseFencingLeaseManager
RecordSchemaSchemaRegistryVersionedRecordMigrationResult
Ordinary storage operations may return CallFailure, which is false-valued.
result = staqtapp.get_value("missing")
if not result:
print(result.error_type, result.message)
# The host loop remains alive.
staqtapp.set_value("next_operation", "ready")Do not confuse a valid stored false-like value with a failure. Where ambiguity matters, use isinstance(result, staqtapp.CallFailure).
import staqtapp
from staqtapp.public_api import (
BoundedMathematicalAggregator,
IntervalGeometryObservation,
MathematicalDataClass,
MathematicalSourceBundle,
)
source = MathematicalSourceBundle(
interval_geometry=(
IntervalGeometryObservation(
interval_density_q16=32_000,
fragmentation_q16=4_100,
merge_count=8,
split_count=2,
occupancy_q16=48_000,
depth=4,
locality_q16=52_000,
),
),
data_class=MathematicalDataClass.REALITY,
source_ids=("runtime-sample:17",),
evidence_qualified=False,
)
projection = BoundedMathematicalAggregator(
max_records_per_source=256
).aggregate(source, bins=8)
record = {
"signature": projection.signature,
"data_class": projection.data_class.value,
"evidence_qualified": projection.evidence_qualified,
"truncated": projection.truncated,
"record_counts": projection.record_counts,
"measures": {
item.key: {
"value": item.value,
"unit": item.unit,
"source": item.source,
}
for item in projection.measures
},
}
result = staqtapp.run_transaction([
("set_value", ("cycle_projection", record)),
("set_value", ("cycle_status", "observed")),
])
if not result:
raise RuntimeError(result.failure.message)Why this pattern is preferred:
- source provenance travels with the projection;
- truncation is persisted rather than hidden;
- signature supports replay and comparison;
- status and projection commit atomically;
- no claim of promotion is made.
from staqtapp.public_api import MathematicalDataClass, MathematicalSourceBundle
qualified = MathematicalSourceBundle(
data_class=MathematicalDataClass.REALITY,
source_ids=("runtime:cycle-44",),
evidence_ids=("evidence:sealed-44",),
evidence_qualified=True,
)Never set evidence_qualified=True merely because a record came from a production-like environment. Qualification should follow the evidence-verification path.
receipt = staqtapp.set_value("checkpoint", {"sequence": 81})
if isinstance(receipt, staqtapp.CallFailure):
raise RuntimeError(receipt.message)
flush = staqtapp.flush_writes()
if not flush:
raise RuntimeError("checkpoint accepted but not confirmed durable")
publish_checkpoint_complete()Do not emit an external “committed” event before the configured durability boundary has been crossed.
from typing import Any
import staqtapp
def require_value(name: str) -> Any:
result = staqtapp.get_value(name)
if isinstance(result, staqtapp.CallFailure):
raise LookupError(f"{name}: {result.error_type}: {result.message}")
return resultThis is suitable for code paths where absence or corruption is exceptional. In a long-running service loop, handling CallFailure in-band may be more appropriate.
MAX_OBSERVATIONS = 256
rows = tuple(load_observations())
source_rows = rows[:MAX_OBSERVATIONS]
was_truncated_by_caller = len(rows) > len(source_rows)The aggregator also enforces its own source limits, but bounding at ingestion avoids unnecessary allocation and lets the caller record why data was omitted.
integrity = staqtapp.verify_integrity(deep=True)
if isinstance(integrity, staqtapp.CallFailure):
raise RuntimeError(integrity.message)
# Only after verification and policy review:
optimized = staqtapp.optimize_vfs()
if isinstance(optimized, staqtapp.CallFailure):
raise RuntimeError(optimized.message)Maintenance should be treated as a controlled operation, not a routine response to every latency spike.
For new governance code:
from staqtapp.public_api import BoundedMathematicalAggregatorrather than importing an internal module path. This reduces accidental coupling to implementation helpers.
Bad:
{"p95": 241000}Better:
{
"p95_latency_ns": 241000,
"data_class": "REALITY",
"source_ids": ["pressure:stage-2"],
"evidence_qualified": False,
"truncated": False,
}A number without source, class, and truncation information is easy to misuse.
The aggregator permits a bounded bin range, but 128 bins do not add information to eight observations. Choose a bin count that reflects the sample size and downstream purpose.
A practical rule is:
bins = min(32, max(8, observation_count))while staying within the API’s accepted bounds.
Store latency in nanoseconds, bytes as integers, counts as integers, and normalized values as Q16. Convert to human-readable units in presentation code, not in the evidence record.
A deterministic projection over a truncated source is still deterministic, but it is not equivalent to a projection over the full source. Persist and display the truncation flag.
When batching is enabled, design state names and receipts so application code can distinguish:
queued → flushed → externally acknowledged
Do not collapse these stages.
A transaction is appropriate when partial completion would violate an invariant. Large unrelated update sets can increase contention and failure scope.
When comparing baseline and candidate, alternate execution order. Otherwise warm caches, allocator state, or one-time setup can masquerade as candidate improvement.
A governed system should be allowed to say:
- insufficient evidence;
- excessive variance;
- stale directive;
- scope mismatch;
- no valid alternative;
- replay regression;
- candidate not reached due to bounded early termination.
“Forced winner” logic is dangerous in adaptive storage.
Console-derived hashes and presentation summaries can support navigation and grouping. Unless they pass the evidence path, label them as derived fingerprints rather than proof.
Qt and process telemetry are optional. Headless deployments should install the core package only. Observability should consume snapshots and records without controlling storage locks.
When recording outcomes, include both property generation and intelligence generation. “Latest” is not a stable identity and can invalidate replay.
Use intent validation, capability checks, schema validation, and value validation before scheduling or mapping an operation across many targets.
Rebuild or inspect the read index for workloads that benefit from range access. Do not rebuild reflexively for every read.
Compaction can reduce storage overhead, but a candidate rollout without an adequate rollback horizon weakens governance. Set keep_revisions according to validation duration and operational policy.
The ordinary API favors contained failures:
value = staqtapp.get_value("possibly_missing")
if isinstance(value, staqtapp.CallFailure):
log_storage_failure(value.error_type, value.message)
else:
process(value)
continue_service_loop()This model is useful in agents, schedulers, and interactive systems where one missing value should not terminate the host process.
Governance failures are different. A stale fencing token, invalid evidence signature, unauthorized scope, or integrity failure may require a typed exception because continuing as though authority were valid would violate the system contract.
The rule is not “never raise.” It is:
- expected storage failures are representable in-band;
- authority and trust violations remain explicit and typed;
- broad exception handlers are classified and audited.
Use revision inspection before rollback:
revisions = staqtapp.list_revisions(limit=20)
# Select a revision according to application policy.
result = staqtapp.rollback_revision(revision_id)Do not choose a revision solely by proximity in time. Verify that it predates the unwanted transition and belongs to the correct VFS and generation.
A conservative sequence is:
stop or fence writers
↓
inspect diagnostics
↓
verify integrity
↓
identify declared valid source / revision
↓
perform bounded recovery or repair
↓
verify again
↓
reconstruct governed state
↓
resume under a fresh fencing token
The governed runtime includes durable recovery records and leases. Recovery should itself be journaled so that an operator can distinguish:
- an interrupted recovery;
- a completed recovery;
- a stale worker attempting to resume;
- a new recovery lease;
- a reconstructed scheduler or lifecycle state.
A lease alone may not prevent a paused or partitioned worker from resuming after another worker has acquired ownership. Monotonic fencing tokens solve this by assigning a strictly newer generation to each valid lease.
Let token (t_n) be the current token. A new lease receives:
[ t_{n+1} > t_n ]
A write or activation carrying (t < t_{current}) is rejected even if the stale worker believes its lease is valid.
The governed scheduler records finite job states such as queued, leased, completed, failed, or administratively controlled. Persistent scheduling and event-sourced state allow reconstruction after restart.
Scheduling decisions can use bounded resource snapshots rather than uncontrolled system probing. A snapshot should be treated as time-bound observation, not a perpetual guarantee.
Pause, resume, drain, and maintenance controls should be journaled. “Drain” is not the same as “kill”: it prevents new work while permitting policy-defined in-flight completion.
- acquire a fresh fencing lease after process restart;
- never cache a token indefinitely;
- include the token in the operation or directive binding;
- reject stale tokens at the authoritative gateway;
- keep lease renewal and storage mutation as separate failure domains;
- reconstruct durable state before accepting new work.
The optional PyQt5 console provides pages for:
- Overview;
- Runtime;
- Evolution;
- Pressure;
- Scheduler;
- Intelligence;
- Audit;
- Settings.
It is installed separately:
python -m pip install ".[console]"
staqtapp-consoleThe console is an operational observer. It is not the source of truth for storage state, evidence authority, or promotion.
A sound console adapter should:
- consume immutable or copied snapshots;
- avoid storage hot-path locks;
- tolerate dropped or delayed display samples;
- distinguish stale from current state;
- label synthetic, demo, shadow, reality, and derived data;
- perform no implicit promotion;
- remain optional for headless deployments.
A high score means “high according to this bounded measure,” not “authorized.” A candidate shown in an evolution page may still be shadow-only. A stable signature shown in an audit page is not proof. A scheduler count is a snapshot, not a guarantee that a job remains in that state.
The 2.2.5H package records a broad local hardening campaign. The supplied validation artifacts report:
- canonical NFC JSON and versioned digest/authentication domains;
- authenticated append-only journals, offline verification, and key rotation;
- process-safe leases and monotonic fencing tokens;
- stale-worker rejection;
- sealed evidence provenance and verifier-derived authority;
- receipt binding and durable anti-replay enforcement;
- deterministic scoped activation routing;
- event-sourced governed state;
- durable administrative lifecycle and authenticated scheduling;
- paired AB/BA pressure trials and conservative decision semantics;
- versioned schemas and append-only migration tooling;
- explicit public/internal API boundaries;
- typed governance failures and audited broad exception handling;
- bounded mathematical aggregators;
- explicit
DERIVED_FINGERPRINTlabeling for non-evidence display data; - exclusion of quarantined legacy code from the evaluation wheel.
The package’s reports state:
| Validation area | Supplied result |
|---|---|
| Test suite | 544 passed, 0 skipped, 1 warning |
| Combined line/branch coverage | 86.2% over the declared evaluated core |
| Public API | 133 unique, importable exports |
| Exception audit | 22 evaluated modules, 62 classified broad handlers, 0 unclassified |
| Focused mutation test | 100 killed, 47 survived, 68% killed in canonical.py |
| Static checks | selected Ruff and focused MyPy checks passed |
| Dependency audit | no known vulnerabilities in the local validation environment |
| Secret-pattern scan | no matches in production source or docs |
| Isolated wheel | install, imports, CLI help, bytecode compile, and real VFS round trip passed |
| Console | off-screen Linux interaction and 12-frame visual validation passed |
| API reference PDF | 27 pages, 63 outline entries, no reported layout issues |
These are strong local hardening results, but they should be stated precisely.
The supplied artifacts also disclose important limits:
- external Ubuntu, Windows, and macOS CI remained an unobserved release gate;
- native compositor, multi-DPI, accessibility, Windows, and macOS console validation remained pending;
- MyPy was focused, not full-project type cleanliness;
- no independent SAST or supply-chain attestation was claimed;
- mutation survivors remain a disclosed backlog;
- dependency audit results are point-in-time;
- the package remains Beta.
This paper therefore uses hardened Beta rather than “formally verified,” “production proven,” or “universally secure.”
The most important result of the campaign is not a test count. It is a set of structural guarantees:
- intelligence cannot silently become execution authority;
- evidence classes cannot be casually collapsed;
- stale processes can be fenced;
- activation scope and budget are explicit;
- real outcomes are bound to receipts;
- replays cannot be consumed repeatedly as new evidence;
- nondeterministic ordering is constrained;
- resource use is bounded;
- failures are either contained or surfaced as typed authority violations;
- recovery has a recorded, verifiable path.
Staqtapp-1Xq 2.2.5H should not be described as:
- a production-stable release;
- a formally verified storage engine;
- a distributed consensus database;
- a replacement for external backups;
- an unrestricted autonomous agent;
- a causal inference engine;
- a guarantee that a recommended property will improve performance;
- proof that a synthetic or shadow result will transfer to reality;
- cross-platform validated merely because the wheel is
py3-none-any; - compatible with Staqtapp-1X or Staqtapp-TDS by implication.
It is best described as a hardened Beta intelligent storage engine with a governed, deterministic, evidence-aware embedded reasoning architecture.
Staqtapp-1Xq’s most novel contribution is not any individual mathematical function. Weighted Manhattan distance, fixed-point arithmetic, deterministic sorting, event sourcing, fencing tokens, authenticated journals, and paired trials are all established ideas.
The contribution is their composition into a single authority-preserving loop.
The architecture recognizes that an adaptive storage system has two risks:
- it may make a technically poor decision;
- it may make a decision without legitimate authority or trustworthy evidence.
Many systems focus almost entirely on the first risk. 1Xq treats the second as equally important.
The resulting design has a useful property: every intelligent result can be questioned at the right level.
- Was the source real, shadow, synthetic, or demo?
- Was the source truncated?
- Was the projection deterministic?
- Which reason bits produced the finding?
- Which capability admitted the alternative?
- Which scope allowed activation?
- Which fencing token authorized the process?
- Which operation receipt bound the outcome?
- Which verifier qualified the evidence?
- Was the evidence already consumed?
- Which generation can be rolled back?
That is what makes the system governable.
The design also avoids a false dichotomy between “static deterministic software” and “uncontrolled AI.” It demonstrates a third category: bounded adaptive software in which learning is expressed as finite observations, explicit candidates, immutable lineage, and real-evidence gates.
staqtapp.configure(...)
staqtapp.makevfs(...)
staqtapp.set_value(name, value)
staqtapp.get_value(name)
staqtapp.run_transaction(operations)
staqtapp.flush_writes()
staqtapp.verify_integrity(deep=True)
staqtapp.list_revisions(limit=...)
staqtapp.rollback_revision(...)
staqtapp.compact_vfs(keep_revisions=...)from staqtapp.public_api import (
BoundedMathematicalAggregator,
MathematicalSourceBundle,
MathematicalDataClass,
IntervalGeometryObservation,
ComputationalMemoryObservation,
PressureTrialObservation,
EvolutionLineageObservation,
LedgerCycleObservation,
)from staqtapp.public_api import (
ActivationDirective,
ActivationRouter,
ScopedStorageExecutionGateway,
EvidenceVerifier,
EvidenceTrustStore,
ConsumedEvidenceStore,
GovernedStateRepository,
GovernedRuntimeScheduler,
FencingLeaseManager,
AuthenticatedJournal,
SchemaRegistry,
)Activation — Controlled, scoped exposure of a baseline or candidate generation to real operations.
Authority — The permission to cause or validate a state transition. Reasoning output does not carry storage authority by default.
Bounded — Subject to explicit finite limits on records, candidates, dimensions, stages, memory, or operations.
CallFailure — False-valued ordinary storage failure object that allows a host loop to continue.
Candidate generation — Immutable proposed property generation derived from a parent generation.
Canonicalization — Conversion to a stable representation before digesting, signing, or comparing.
Computational memory — Bounded retained classifications and reuse metadata used by the embedded reasoning subsystem.
Counterfactual — Synthetic or replayed comparison used to estimate what might happen under an alternative; not real evidence.
Derived fingerprint — Deterministic presentation or grouping identity that is explicitly not evidence.
Evidence qualification — Verification process that establishes whether a real-origin record meets the authority and binding requirements for governed use.
Fencing token — Monotonically increasing token used to reject stale workers.
Governed AI — Reasoning constrained by finite inputs, deterministic rules, provenance, scopes, budgets, and external execution authority.
Intelligence ledger — Durable relationship graph linking cycles, evidence, outcomes, and generations.
Non-halting — Ordinary API failure model that returns an inspectable false-valued object rather than terminating the host control loop.
Pressure trial — Bounded baseline/candidate workload comparison, preferably paired in AB/BA order.
Promotion — Governance decision to accept a candidate beyond controlled validation; distinct from proposal and activation.
Q16 — Unsigned fixed-point-like integer domain from 0 to 65,535 used for normalized bounded values.
REALITY — Data class for real operational observations. It is not automatically evidence-qualified.
Receipt binding — Cryptographic or deterministic linkage between an operation result and the activation that caused it.
Shadow — Candidate or observation path that does not control authoritative operations.
Stable signature — Deterministic identity for grouping, ordering, and replay; not proof by itself.
Truncation — Explicit indication that a bounded subsystem processed only part of the available source.
Staqtapp-1Xq is built around a strict proposition: an intelligent storage engine should be able to observe, reason, remember, compare, and propose without quietly becoming its own authority.
The storage plane executes. The mathematical plane projects. The reasoning plane proposes. Governance constrains. Evidence verifies. Reality decides.
The above md file explains the current Staqtapp1 flagship storage engine, not 1X. A beta version hardened by 5.6 Sol Max for 3 hours straight. Includes a more than capable embedded sub-system AI related to very advanced storage related task, for instance materials science or provable CINDER LENR simulation / Immutable Calibration Results.