madelaine/publicMasters
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

spell checked course on codes

Browse Source
This commit is contained in:
Florent Madelaine 2025-01-27 10:32:07 +01:00
parent ee9a6fc488
commit 894ea1502f
1 changed files with 6 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
2-InformationTheory/1InformationTheory.md
View File

@ -1,9 +1,9 @@
# Information Theory
## Coding theory
When we store or transmit data, no system is perfect and some bits of information are incorrectly stred/retrieved or transmitted.
When we store or transmit data, no system is perfect and some bits of information are incorrectly stored/retrieved or transmitted.
The purpose of this field is to come up with coding and decoding methods that allows to detect and correct errors with a high probablilty.
The purpose of this field is to come up with coding and decoding methods that allows to detect and correct errors with a high probability.
We shall provide an introduction with simple codes.
@ -18,15 +18,15 @@ This is essentially a practical version of Shannon's noisy-channel coding theore
[Details here](https://en.wikipedia.org/wiki/Binary_symmetric_channel)
So in a nutshell, we can tranform a binary symmetric channel with this repetition trick into a binary symmetric chanel with an arbitrary low error rate.
So in a nutshell, we can transform a binary symmetric channel with this repetition trick into a binary symmetric channel with an arbitrary low error rate.
It is not very practical because, it is achieved at a very high cost in terms of transmited of information compared with the actual information we wish to send.We shall therefore look for cheaper alternatives.
It is not very practical because, it is achieved at a very high cost in terms of transmitted of information compared with the actual information we wish to send.We shall therefore look for cheaper alternatives.
## Detection
If we are ready to forget about correction and concentrate on detection there is a very simple trick.
We transmit some bits of information $b_1\ldots b_n$ with *one additionnal bit*
We transmit some bits of information $b_1\ldots b_n$ with *one additional bit*
$c$ that is computed via a very simple method from these bits, that is a certain function $f$ of $n$ arguments such that $f(b_1\ldots b_n)=c$.
At reception of some word $b'_1\ldots b'_nc'$ we check whether $f(b'_1\ldots b'_n)=c'$. If it does we assume that there is no error (we might be wrong here), it it does not we assume that there is an error and ask for retransmission of this message (we are correct here).
@ -44,7 +44,7 @@ Assume that we have 9 bits of information. We write this bits in the form of a 3
With this scheme we can detect 2 errors, but not correct them as there might be up to three codewords that are the nearest to a received message with 2 errors.
We can always correct 1 error by recomputing the redundant data and compare it to the received data.
In particular, if the error is within the information part, the line / column of the bit to correct lies at the intersection of the line and colmun where the two redundant bits differ.
In particular, if the error is within the information part, the line / column of the bit to correct lies at the intersection of the line and column where the two redundant bits differ.
We call this informally the matrix code below.