"BITS" OF INFORMATION

If a dog bites man, it is not a news. If a man bites dog, it is a news. Anything surprise is the news and an information. If an event certainly is not going to occur then its probability of happening is 0 and it is not an information. If an event is certainly going to happen then its probability is 1 and it is also not an information.

An event which is having probability between 0 and 1 carries some information. For example: In tossing of a coin getting a head or tail is having a 50% chance or 0.5 probability. Out come of the toss is an information. In the case of rolling of dice, each outcome has 1/6 chance or 0.167 probability. Certainly the outcome of a dice is the surprise and an information.

Shannon gave a formula to calculate information content in an event based on probability.

I want to calculate the information content of coin toss and I also want to represent it in bits-binary digit format. Here probability p = 0.5 and to represent in binary, the logarithm's base must be 2.

From figure 2 we get, information content is 1 bit. That is , outcome tail can be indicated by a bit 0 and head by a bit 1.

An institution follows the following grading system.
O - outstanding
A- first class
B - second class
C- third class
F- fail

How much information is there in the grades O,A,B,C,F.
Each appearance of the grade has 1/5 chance or 0.2 probability.
From figure 2, information content in the grades is 2.3 bits. That is , slightly over 2 bits is required to encode the grades.

O- 0
A-1
B-10
C-11
F-100

Suppose we have a string like this,
"AAAACCBBBADBBAA"
Info. content in the string can be calculated using slightly modified Shannon's formula and we also get maximum number of bits required to encode it efficiently.

Initially the computer system was based on 8 bits code then we moved to 16 bits, 32 bits, and even 64 bits. More bits means more info, more pixels, more colours and more clarity.

Shannon's formula is basic to today's digital communication. It has led to efficient error-detecting and error-correcting codes. It is used in CD's, DVD's, data compression and extracting meaning from DNA sequences.

Science update: Stephen Hawking appears as hologram in Hong Kong and discusses present situation of the world.

Comments

THE EARTH, A SUPER ORGANISM

JOIN MY COURSE: "Become a programmer in a day with python" A man called 'love lock' (what a name) proposed a theory called Gaia theory, named after Greek Goddess. It says, "Earth is a self-regulating organism like a human being. The organic life in it interacts with in-organic matter and maintains atmosphere, temperature and environment". Hence the earth is still suitable for the life to thrive. Imagine, in a particular place, there are lot of flowers. Some flowers are white and some are darkly coloured. We know, white reflects light and heat while dark absorbs the same. White flowers can thrive in hot climate. But dark flowers requires cold climate. The absorption and reflection balances and the environment reaches average, warm temperature at which both the flowers can co-exist. This is the essence of "Gaia" theory. On our earth, ...

THE PARABOLA

A jet of water shooting from a hose pipe will follow a parabolic path. What is the so special about parabola. Y= x^2 Draw a graph for the above equation. It will result in a parabola. This parabola is also called unit parabola. Any equation involving square will yield a parabola. Example: Y = 2x^2 +3x+3 (also called quadratic equation) X= 2 and -2, both satisfies the equation 4 = X^2. Parabolic equations always have two solutions. Any motion taking place freely under gravity follows parabolic path. Examples: An object dropped from a moving train, A bomb dropped from flying plane, A ball kicked upwards. If a beam of light rays fall on the parabolic shaped mirror, they will be reflected and brought to focus on a point. This fact is made use of in Dish Antenna, Telescope mirrors, etc. ...

DISORDER IS THE "ORDER OF THE DAY"

Imagine a balloon full of air. The air molecules are moving randomly inside the balloon. Let us pierce the balloon with a pin. The air rushes out. Why should not the air molecules stay inside the balloon safely and ignore the little hole? That is not the way the world works. The molecules always "want to occupy as many states as possible". Hence the air goes out in the open to occupy more volume. The things always goes into disorder (entropy) and the disorder increases with time. The above statement is what we call "second law of thermodynamics". Consider a cup of coffee on the table. Suppose the heat from entire room flows to your cup of coffee, the coffee will boil and the rest of the room will freeze. Freezing means bringing things to order and arrangement. It violates the second law. Hence it will never happen. Hence heat must flow from high ...

sciencecapsules

Search This Blog