Structural Information Theory

The Structural Information Theory (Teoría de la Información Estructural, TIE, or SIT in english) offers an alternative approach to understanding data, moving away from measuring statistical uncertainty, entropy (Shannon's theory) towards measuring the inherent structure, geometry, and generative laws of a system. SIT views information deterministically as structure.

Here is a simplified discussion of SIT's core concepts, ensuring the critical mathematical definitions are included:

I. The Formalism: Deconstructing Data into Structure

SIT begins by dissecting a sequence of data into its fundamental components to define structural relationships.

1. The Presence Map ( $M_{Φ}$ )

This is the basic building block of SIT.

For every symbol ( $Φ$ ) in an alphabet, the Presence Map ( $M_{Φ}$ ) is a binary vector (a sequence of 1s and 0s) that acts as the "fingerprint" of that symbol in time. A '1' indicates the symbol appeared at that specific time position.
These maps obey the Fundamental Law of Exclusion—if you add up all the maps for all symbols, you get a vector of all ones, confirming that one and only one symbol occupies every position.

2. The Language of Structure: Transformations ( $τ$ )

Structure is defined by how these Presence Maps relate to one another.

These relationships are described by a finite set of invertible transformations ( $T$ ), which are geometric "rules" applied to the binary vectors.
Basic rules in $T$ include Cyclic Shift ( $S_{β}$ ), Reflection ( $R$ ), and Negation (:). In a physical context, the rules governing the universe could be seen as the transformations within $T$ .

3. Defining Relationships: Structural Adjacency

Two Presence Maps ( $M_{Φ}$ and $M_{Φ^{'}}$ ) are structurally adjacent if one map can be closely generated by applying a transformation ( $T$ ) from the defined space to the other map.

This approximation is measured using the Hamming distance ( $d_{H}$ ), which counts the number of positions that differ. This distance must be below a tolerance threshold ( $δ$ ) to account for "noise" or exceptions.
These adjacencies create the Symbolic Dependency Graph ( $G_{Σ}$ ), which visually maps the file’s internal structure.

II. Core Principles: Finding the Irreducible Information

These principles determine which parts of the data are genuinely new information and which are redundant.

1. The Principle of Structural Partition

The Dependency Graph ( $G_{Σ}$ ) naturally breaks down into $m$ unconnected components (families).

Consequence: The true informational basis of the file is not the original $k$ symbols, but only the $m$ Base Maps, one derived from each independent family. The remaining $k - m$ maps are dependent and therefore considered redundant.
The Conjecture of Redundancy suggests that $m$ is often much smaller than $k$ , potentially bounded by $m \approx k / \log_{2} (N)$ .

2. The Principle of Minimum Structural Description (Causal Information)

This defines the primary measure of SIT: Structural Information ( $I_{S} (F)$ ).

$I_{S} (F)$ is not the file’s statistical randomness (entropy), but the length of the shortest computer program required to generate the file using the SIT structural rules.
For systems generated by simple rules ( $r$ ), this cost is tiny, bounded by $$I_S(F) = O(r \cdot \log N)$$, making it independent of the overall file length ( $N$ ).

III. The Definitive Metric and the Role of Time

SIT provides a specific, computable metric for information and offers a unique perspective on randomness.

1. The Structural Information Metric ( $I_{S} (F)$ )

The total structural information is the sum of costs for describing the irreducible parts, plus the costs for describing how the dependent parts are generated:

I_{S} (F) = \sum_{j = 1}^{m} C_{a l g} (M_{b a s e_{j}}) + \sum_{i = m + 1}^{k} (\log_{2} (| T_{j} |) + C_{a l g} (E_{i}))

This formula quantifies the information as the sum of three elements:

The Algorithmic Complexity ( $C a l g$ ) of the fundamentally independent Base Maps (the truly irreducible parts).
The cost of describing the structural relations (the transformations, $T$ ) used to link the other maps.
The cost of the Exceptions/Errors ( $E$ ) where the rules fail.

2. Structural Entropy ( $H_{S}$ )

Structural Entropy ( $H_{S}$ ) is a global geometric measure, unlike Shannon's local statistical measure.

$H_{S}$ is defined by the group of symmetries ( $G_{F}$ ) that leave the file structure unchanged.
$H_{S} (F) = \log_{2} (Number of distinguishable orbits under G_{F})$
A file with strong symmetries (e.g., a highly repetitive texture) will have a large symmetry group ( $G_{F}$ ), resulting in a drastically lower Structural Entropy ( $H_{S}$ ) compared to its Shannon Entropy.

3. Temporal Randomness

SIT proposes that much of what is perceived as randomness is not due to the choice of the symbol itself, but rather the instant of its appearance. Randomness is often an "illusion causal" (causal illusion), resulting from observing the superposition and desynchronisation of multiple underlying deterministic processes.

Since each Presence Map is a symbol's informational trajectory in time, plotting the intervals between symbol appearances yields its stroboscopic curve.
For structured systems, these curves are contained within an Evolution Envelope, which defines the limits of causal possibilities for that symbol. The size of this envelope measures the symbol's temporal "freedom".

source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5374335
(in Spanish)

#theory #information #statistics #structure #entropy #Shannon #geometry