Block code
Fundamentals
Definition
A block code over an alphabet is formally defined as a subset , consisting of codewords each of fixed length , where the encoding process maps messages composed of symbols from to corresponding -symbol codewords in .[5] This structure ensures that the code operates on discrete blocks of data, distinguishing it from codes that process variable-length or streaming inputs.[1] Encoding can be systematic or nonsystematic: in systematic encoding, the original message symbols appear explicitly as a contiguous portion of the -symbol codeword, with the remaining symbols serving as redundancy; in nonsystematic encoding, the message symbols are transformed and intermixed with redundancy without direct embedding.[6] Block codes play a crucial role in discrete memoryless channels by introducing controlled redundancy into the transmitted signal, enabling the detection and correction of errors introduced by noise while preserving reliable communication up to the channel's capacity.[7] The foundational concept of block codes originates from Claude Shannon's 1948 paper "A Mathematical Theory of Communication," which established the theoretical framework for error-correcting codes as a means to achieve reliable transmission over noisy channels, positioning block codes as a core model in information theory.[7]Parameters
The parameters of a block code quantify its structural properties and error-handling capabilities, providing essential metrics for evaluating efficiency and reliability in communication systems. The block length denotes the fixed total number of symbols comprising each codeword, which establishes the overall size of the encoded block and directly influences the level of redundancy incorporated to combat transmission errors. In binary block codes, where the alphabet , the symbols are bits, and represents the total bit length of the codeword.[8] The message length specifies the number of information-carrying symbols within each codeword, determining the code's capacity to represent distinct messages; for an alphabet of size , this yields possible messages.[9] By design, holds for all block codes, as the additional symbols serve as redundancy to enable error detection and correction without expanding the message space. The rate measures the code's efficiency as the fraction of symbols dedicated to information rather than redundancy, with higher rates approaching the theoretical channel capacity for reliable communication as established in foundational information theory.[7] For instance, rates close to but below the capacity ensure arbitrarily low error probabilities for sufficiently large .[7] The minimum distance is the smallest Hamming distance—the number of symbol positions differing—between any two distinct codewords, serving as the primary determinant of the code's robustness against errors.[10] A block code with minimum distance can detect up to errors, as any such alteration cannot transform one valid codeword into another.[11] Furthermore, it can correct up to errors, since spheres of radius around codewords remain disjoint, allowing unique decoding within those regions.[10]Notation and Conventions
Alphabet and Fields
In block codes, the alphabet is a finite set of symbols from which codewords are constructed, typically denoted by its cardinality , where . Common examples include the binary alphabet with , used in many practical systems for its simplicity, and q-ary alphabets where symbols are elements of a larger set, such as .[12] The choice of forms the foundational symbol set for encoding messages into fixed-length blocks of length , influencing the overall code structure and error-handling capabilities.[13] For algebraic constructions, particularly linear block codes, the alphabet is often a finite field, known as a Galois field , where for a prime and positive integer . These fields provide the necessary arithmetic operations—addition and multiplication—that enable vector space structures over , facilitating efficient encoding and decoding. Binary codes specifically employ , where addition is modulo-2 (XOR) and multiplication is AND, simplifying hardware implementations. Reed-Solomon codes, a prominent class of algebraic codes, require to represent symbols as polynomials of degree less than , allowing correction of multiple symbol errors in applications like data storage.[12][13] Nonlinear block codes may use non-field alphabets, such as sets of integers without field properties, to achieve specific performance in non-standard metrics. For instance, permutation codes operate over the symmetric group of permutations of , treating codewords as ordered lists where each integer from 1 to appears exactly once, and distance is measured by the number of mismatched positions. These differ from field-based codes by lacking a linear vector space, instead relying on group operations for structure, which can be advantageous in channels with permutation errors like storage devices.[14] Larger alphabet sizes enable higher code rates by packing more information per symbol while maintaining desired minimum distances, approaching bounds like the Singleton bound for certain constructions. However, this comes at the cost of increased computational complexity, as operations in larger (e.g., multiplication in ) require more resources than binary XOR, and decoding algorithms scale with .[15][13]Dimensions and Rate
In coding theory, block codes are standardly denoted using the parameters , where represents the code length (number of symbols per codeword), the dimension (number of independent information symbols), the minimum Hamming distance, and the alphabet size over the finite field . This notation encapsulates the core structural properties of the code in a compact form. For binary codes where , the subscript is typically omitted, yielding the simplified notation, which has become ubiquitous in the literature on binary linear codes. The evolution of this notation traces back to Richard W. Hamming's seminal 1950 paper on error-detecting and error-correcting codes, where he introduced the binary Hamming code using the parameters to denote length and dimension; subsequent developments in the 1950s and 1960s extended it to general -ary settings as algebraic coding theory incorporated finite fields beyond .[16][9][10][16] For linear block codes, the dimension serves as a logarithmic measure of the code's size: the total number of codewords equals , reflecting the -dimensional subspace structure over . This parameterization allows for systematic encoding via a generator matrix, where the rows form a basis for the code. In practice, determines the information throughput, with redundancy introduced through the parity symbols to achieve error control. The choice of directly influences the code's capability to balance reliability and efficiency.[9][16] The rate of a block code, defined as , quantifies the proportion of information symbols in each codeword and thus the code's efficiency; values closer to 1 indicate less overhead from redundancy. For families of codes designed for varying block lengths, the asymptotic rate provides a normalized metric for evaluating performance in high-rate regimes, such as those approaching channel capacity limits. A key interrelation among parameters is previewed by the Singleton bound, which asserts that any block code satisfies , establishing an upper limit on distance relative to dimension and length. This bound highlights inherent trade-offs in code design, though achieving equality (in maximum distance separable codes) remains challenging beyond specific constructions.[17][18][17]Minimum Distance
The Hamming distance between two codewords is the number of positions at which they differ.[10] This metric, introduced by Richard Hamming, serves as the foundational measure of dissimilarity in coding theory.[10] The minimum distance of a block code is defined as the smallest Hamming distance between any two distinct codewords, given by
where denotes the Hamming weight, or number of nonzero symbols in a vector.[11] This parameter quantifies the code's robustness against errors, as it determines the radius for unique nearest-neighbor decoding, ensuring that codewords are sufficiently separated to distinguish error patterns within that sphere.[19]
For linear block codes over a field, which form a vector subspace containing the all-zero codeword, the minimum distance simplifies to the minimum weight of any nonzero codeword:
[11] This equivalence holds because the difference of two codewords is itself a codeword under vector addition.[19] The weight distribution provides further insight into the code's structure, captured by the weight enumerator coefficients , where is the number of codewords of weight , with and .[19]
Computing the minimum distance for linear codes involves analyzing the generator matrix (a matrix whose rows span the code) to find the lowest-weight linear combination of its rows, or equivalently, the parity-check matrix (an matrix satisfying for codewords ). Specifically, is the smallest integer such that some columns of are linearly dependent over the field.[11][19]
Constructions and Examples
Linear Block Codes
Linear block codes form a fundamental subclass of block codes in coding theory, characterized by their algebraic structure as vector subspaces over finite fields. A linear block code $ C $ of length $ n $ over the finite field $ \mathrm{GF}(q) $ is defined as a $ k $-dimensional subspace of the vector space $ \mathrm{GF}(q)^n $, meaning it is closed under addition and scalar multiplication, with $ |C| = q^k $ codewords.[20] This linearity enables efficient encoding and decoding via matrix operations, distinguishing them from nonlinear codes.[2] The code can be generated using a generator matrix $ \mathbf{G} $, which is a $ k \times n $ matrix whose rows form a basis for $ C $. Any codeword $ \mathbf{c} \in C $ is obtained as $ \mathbf{c} = \mathbf{m} \mathbf{G} $, where $ \mathbf{m} $ is a $ k $-tuple message vector in $ \mathrm{GF}(q)^k $. Often, $ \mathbf{G} $ is put into systematic form $ \mathbf{G} = [\mathbf{I}_k \mid \mathbf{P}] $, where $ \mathbf{I}_k $ is the $ k \times k $ identity matrix and $ \mathbf{P} $ is a $ k \times (n-k) $ parity submatrix; this form directly appends $ k $ message symbols followed by $ n-k $ parity symbols to form the codeword.[21][2] Dually, the code is defined by a parity-check matrix $ \mathbf{H} $, an $ (n-k) \times n $ matrix whose rows span the dual space $ C^\perp $, such that $ \mathbf{c} \in C $ if and only if $ \mathbf{H} \mathbf{c}^T = \mathbf{0} $. For a systematic generator $ \mathbf{G} = [\mathbf{I}k \mid \mathbf{P}] $, the corresponding $ \mathbf{H} $ takes the form $ \mathbf{H} = [-\mathbf{P}^T \mid \mathbf{I}{n-k}] $ (or equivalently over $ \mathrm{GF}(2) $, $ \mathbf{H} = [\mathbf{P}^T \mid \mathbf{I}_{n-k}] $). This matrix facilitates error detection, as a received vector $ \mathbf{r} $ is a codeword precisely when the syndrome $ \mathbf{H} \mathbf{r}^T = \mathbf{0} $.[20][2] Encoding a message $ \mathbf{m} $ into a codeword $ \mathbf{c} $ is performed by the linear map $ \mathbf{c} = \mathbf{m} \mathbf{G} $, which requires $ O(nk) $ operations over $ \mathrm{GF}(q) $ and preserves the subspace structure. In systematic form, the first $ k $ positions of $ \mathbf{c} $ are simply the message symbols, with the remaining positions computed as linear combinations specified by $ \mathbf{P} $.[21] Prominent examples include the binary Hamming code, a perfect single-error-correcting linear code whose parity-check matrix has columns that are all nonzero vectors in . In its systematic form, parity checks are placed on positions 1, 2, and 4. The extended Hamming code adds an overall parity bit, yielding a code capable of single-error correction and double-error detection; its generator matrix appends a column of all ones to the original .[20][22] Reed–Solomon codes are another important class of linear block codes, defined over with length , dimension , and minimum distance . They are cyclic and particularly effective for correcting burst errors, widely used in digital storage such as CDs, DVDs, and QR codes. BCH codes generalize Hamming codes to multiple-error correction, constructed as cyclic codes over capable of correcting up to errors with designed distance . Binary BCH codes of length include the Hamming code as the single-error case and are used in applications like satellite communications. Linear codes can be modified to derive new codes via standard operations that preserve linearity. Shortening reduces length by fixing certain message symbols to zero and puncturing the corresponding code positions, transforming an $ (n,k,d)_q $ code into an $ (n-1,k-1,d)_q $ code with at least the original minimum distance. Puncturing shortens the code by deleting specific coordinate positions from all codewords, yielding an $ (n-1,k,d-1)_q $ code (or better), which sacrifices some distance for reduced length. Extending increases length by appending a new parity-check symbol (e.g., an overall parity bit in binary cases), producing an $ (n+1,k,d+1)_q $ or $ (n+1,k,d)_q $ code, as in the extended Hamming example, to enhance error-detection capability. These operations allow construction of families of codes from smaller base codes.[23][24][25]Nonlinear Block Codes
Nonlinear block codes are defined as arbitrary subsets , where is a finite alphabet of size , such that the minimum Hamming distance between any two distinct codewords is at least . This general formulation encompasses all block codes, including linear ones as a special case where forms a vector subspace over the field , but nonlinear codes lack this linearity and thus do not require the sum of any two codewords to be in .[26][27] A key advantage of nonlinear codes is their potential to achieve larger cardinalities for fixed length and minimum distance compared to linear codes, enabling better rates in certain parameter regimes. The Plotkin construction exemplifies this by combining two codes of length to form a nonlinear code of length , often yielding improved distance properties without the constraints of linearity. For binary alphabets, nonlinear constructions frequently attain the optimal value , the maximum possible , particularly for small , as evidenced by comprehensive tables of best-known codes that include nonlinear examples surpassing linear bounds.[28][29] Prominent constructions include the Nordstrom-Robinson code, a binary nonlinear code with parameters , which achieves and exceeds the size of the best linear binary code for these parameters—a Reed-Muller code with only 32 codewords and distance exceeding 6, but no linear code reaches 256 codewords at distance 6.[30] In comparison to linear codes, nonlinear codes share parameters , , and effective rate but allow to be any integer rather than a power of , facilitating optimizations for specific applications where algebraic simplicity is secondary to maximal code size. Tables of for binary codes up to moderate demonstrate that nonlinear codes are optimal in numerous cases, underscoring their role in pushing theoretical limits.[31]Error Handling Properties
Detection Capabilities
Block codes enable error detection by leveraging the minimum distance between codewords, ensuring that certain error patterns cannot transform one valid codeword into another. Specifically, a block code can detect up to errors, as any error pattern of weight at most will result in a received word that is not a codeword, since the spheres of radius around distinct codewords do not overlap.[12] This detection capability holds for both linear and nonlinear block codes, relying solely on the code's distance property without attempting to identify the exact error locations. For linear block codes, error detection is efficiently performed using syndrome decoding with the parity-check matrix . Upon receiving a vector , where is the transmitted codeword and is the error vector, the syndrome is computed. If , the received vector is deemed a valid codeword (no error detected); otherwise, a nonzero syndrome indicates the presence of errors, allowing detection without correction.[2] This method is particularly effective for detecting low-weight errors, as syndromes distinguish erroneous receptions from codewords. A simple example of a detection-oriented block code is the parity-check code, which has and appends a single parity bit to ensure an even (or odd) number of 1s in the codeword. Such codes reliably detect any single error, as it would alter the parity and produce a nonzero syndrome, though they fail to detect even numbers of errors.[32] More advanced cyclic block codes, like the cyclic redundancy check (CRC), enhance detection for burst errors—consecutive bit flips common in transmission channels—by using a generator polynomial to append check bits that detect all bursts of length up to the degree of the polynomial, making CRC widely used in data links and storage systems.[33] The performance of block codes in detection is often quantified by the probability of undetected error, which occurs when an error pattern coincides with a nonzero codeword, mapping the received vector to another valid codeword. For a linear block code over an alphabet of size on a memoryless channel with random errors, this probability approximates for low error rates, reflecting the fraction of nonzero syndromes that correspond to undetectable errors.[34] This bound underscores the trade-off: higher redundancy ( larger) improves detection reliability by reducing the likelihood of undetectable errors.Correction Capabilities
Block codes enable the correction of errors in transmitted data by leveraging the minimum distance between codewords. Specifically, a block code can correct up to errors per codeword, as this ensures that the spheres of radius centered at each codeword in the Hamming space are disjoint, preventing overlap that could lead to ambiguous decoding.[10] The standard approach to error correction is maximum likelihood decoding, which selects the codeword closest to the received vector in Hamming distance, assuming errors occur independently with equal probability. This method guarantees correct decoding for up to errors, as any received vector within the decoding sphere of a codeword will be assigned to that codeword. For linear block codes, decoding often relies on syndrome computation, where the syndrome of the received vector is calculated as $ \mathbf{s} = \mathbf{H} \mathbf{r} $ (with the parity-check matrix and the received vector); if , no correction is needed, otherwise, the syndrome identifies the error pattern.[10] For small codes like the Hamming code, precomputed syndrome lookup tables map each nonzero syndrome directly to the corresponding single-error pattern, enabling efficient correction. Algebraic decoding methods extend this capability for more complex linear codes. For example, Reed-Solomon codes, which are nonbinary linear block codes, use the Berlekamp-Massey algorithm to solve for the error-locator polynomial from the syndrome sequence, allowing correction of up to errors in a bounded algebraic manner.[35] Similarly, BCH codes, a class of cyclic binary codes designed to correct multiple random errors, employ related algebraic techniques to achieve their correction capability, with parameters chosen such that the designed distance ensures up to errors are correctable.[36] Beyond errors, decoding may fail entirely (if the received vector falls outside all decoding spheres) or result in miscorrection (assigning to an incorrect codeword), potentially introducing more errors than present, which underscores the importance of the minimum distance in limiting reliable correction.Theoretical Bounds
Upper Bounds
The Singleton bound establishes a fundamental limit on the size of a block code, stating that for a code of length over an alphabet of size with minimum distance , the number of codewords satisfies . Equivalently, in terms of the dimension , the bound implies . This result is derived by projecting the code onto any coordinates; to preserve the minimum distance, this projection must be injective, limiting the code size to at most . Codes achieving equality in the Singleton bound are known as maximum distance separable (MDS) codes, a class exemplified by Reed-Solomon codes. The Hamming bound, also called the sphere-packing bound, provides another key upper bound on , given by
where is the largest integer such that the code can correct errors. This bound follows from the observation that the spheres of radius centered at each codeword are disjoint and contained within the entire space of possible words, leading to the volume constraint. Codes achieving equality are termed perfect, meaning their spheres exactly partition the space; notable examples include the Hamming codes over finite fields.[37]
When the minimum distance exceeds , the Plotkin bound yields a tighter restriction: . This construction relies on a double-counting argument over pairwise distances in the code, showing that large forces the codewords to be sufficiently separated, capping the size in high-distance regimes. The bound is especially relevant for codes with rates approaching the capacity limit in noisy channels with high relative distance . The original formulation was for binary codes, but it generalizes directly to -ary alphabets via similar averaging techniques.[38]
The Elias-Bassalygo bound refines the Plotkin bound for -ary codes by optimizing the underlying distance averaging, resulting in a stricter upper limit on for parameters where the simple Plotkin is loose. Specifically, it improves the constant factors in the denominator, providing better performance for non-binary alphabets in the regime . This tightening combines Elias's early random coding arguments with Bassalygo's combinatorial refinements, influencing subsequent asymptotic analyses of code families.
Lower Bounds
Lower bounds in coding theory provide guarantees on the minimum size of a block code achieving given parameters, ensuring the existence of codes with desirable rate and error-correcting capabilities. These bounds are typically derived using probabilistic or constructive arguments that demonstrate the non-emptiness of certain code spaces.[39] The Gilbert-Varshamov (GV) bound is a fundamental lower bound on the maximum number of codewords in a -ary code of length and minimum distance . It states that there exists a code with . This bound arises from a greedy construction: start with an empty code and iteratively add a codeword that is at distance at least from all existing codewords, which is always possible until the codewords' Hamming balls of radius cover the entire space. The total volume of these balls provides the denominator, ensuring the bound holds. Independently discovered by Edgar Gilbert in 1952 and Rom Varshamov in 1957, the GV bound uses a random coding argument for the linear case, where a random linear code over achieves the stated size with high probability.[40][39][41] The GV bound plays a central role in asymptotic coding theory, where the normalized rate satisfies for relative distance , with the -ary entropy function; this limit is often superior to the Hamming bound (an upper bound) for large in regimes where the Hamming bound is loose.[39]Geometric and Advanced Perspectives
Sphere Packing
In coding theory, the Hamming space refers to the set of all q-ary sequences of length n, denoted , equipped with the Hamming metric, where the distance between two vectors is the number of coordinate positions in which they differ.[42] A sphere of radius centered at a vector consists of all vectors within Hamming distance at most from . The volume (cardinality) of such a sphere is given by
which counts the number of possible error patterns of weight at most .[43]
Geometrically, a block code with minimum distance corresponds to a packing of disjoint spheres of radius centered at the codewords, where these spheres collectively cover the error-correctable portion of the Hamming space. Since the spheres do not overlap and are contained within the total space of volume , the size of the code satisfies , yielding the sphere-packing bound (also known as the Hamming bound) on the maximum number of codewords.[43]
A perfect packing arises when the spheres tile the entire Hamming space without gaps or overlaps, so ; codes achieving this equality are termed perfect codes.[44] Notable examples include the Hamming codes and Golay codes, which saturate the sphere-packing bound.[44] The binary Golay code, a linear code capable of correcting up to errors, is perfect, as its 4096 spheres of radius 3 exactly partition the vectors in the space.
q-ary generalizations of Hamming codes, constructed using parity-check matrices whose columns represent all nonzero points in the projective geometry PG(m-1, q), are perfect 1-error-correcting codes with parameters for any prime power q and integer m ≥ 2.[42]