Directed information

Directed information, $I(X^{n}\to Y^{n})$ , is a measure of information theory and it measures the amount of information that flows from the process $X^{n}$ to $Y^{n}$ , where $X^{n}$ denotes the vector $X_{1},X_{2},...,X_{n}$ and $Y^{n}$ denotes $Y_{1},Y_{2},...,Y_{n}$ . The term "directed information" was coined by James Massey and is defined as

I(X^{n}\to Y^{n})=\sum _{i=1}^{n}I(X^{i};Y_{i}|Y^{i-1})

where $I(X^{i};Y_{i}|Y^{i-1})$ is the conditional mutual information.

Note that if $n=1$ , directed information becomes mutual information $I(X; Y)$ . Directed information has many applications in problems where causality plays an important role such as capacity of channel with feedback,^[1]^[2] capacity of discrete memoryless networks with feedback,^[3] gambling with causal side information,^[4] and compression with causal side information.^[5]

References

↑ Massey, James (1990). "Causality, Feedback And Directed Information" (ISITA).
↑ Permuter, Haim Henry; Weissman, Tsachy; Goldsmith, Andrea J. (February 2009). "Finite State Channels With Time-Invariant Deterministic Feedback". IEEE Transactions on Information Theory. 55 (2): 644–662. doi:10.1109/TIT.2008.2009849.
↑ Kramer, G. (January 2003). "Capacity results for the discrete memoryless network". IEEE Transactions on Information Theory. 49 (1): 4–21. doi:10.1109/TIT.2002.806135.
↑ Permuter, Haim H.; Kim, Young-Han; Weissman, Tsachy (June 2011). "Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing". IEEE Transactions on Information Theory. 57 (6): 3248–3259. doi:10.1109/TIT.2011.2136270.
↑ Simeone, Osvaldo; Permuter, Haim Henri (June 2013). "Source Coding When the Side Information May Be Delayed". IEEE Transactions on Information Theory. 59 (6): 3607–3618. doi:10.1109/TIT.2013.2248192.

This article is issued from Wikipedia - version of the 11/20/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.