A Preferred Definition of Conditional Rényi Entropy

Abstract

The Rényi entropy is a generalization of Shannon entropy to a one-parameter family of entropies. Tsallis entropy too is a generalization of Shannon entropy. The measure for Tsallis entropy is non-logarithmic. After the introduction of Shannon entropy , the conditional Shannon entropy was derived and its properties became known. Also, for Tsallis entropy, the conditional entropy was introduced and its properties were shown. But no specific definition has been given for the conditional Rényi entropy. Several authors have used some definitions of the conditional Rényi entropy, to find their properties and relations among them, but there is no general agreement on any specific definition In this paper, we focus on the definitions of the conditional Rényi entropy, and select one of them on the basis of a relation between Rényi and Tsallis entropies, and show that the chain rule holds generally for the case of conditional Rényi entropy. Then, using this definition, we show some of the properties of conditional Rényi entropy. Finally, we show the relations among Rényi, Shannon and Tsallis entropies.