Register to reply

Why frequency analysis works?

by Avichal
Tags: analysis, frequency
Share this thread:
Avichal
#1
Jul9-13, 08:13 AM
P: 283
English has the letter 'e' with most frequency. Other language also have some of their alphabets appearing more frequently in text hat others. Why is this? Why don't all letters appear with equal frequency?
Do humans speak vowels more comfortably? What exactly is the reason?
Phys.Org News Partner Social sciences news on Phys.org
Physicists create tool to foresee language destruction impact and thus prevent it
Affirmative action elicits bias in pro-equality Caucasians
Election surprises tend to erode trust in government
Bobbywhy
#2
Jul11-13, 10:12 PM
PF Gold
P: 1,881
Tone languages are languages (like Chinese, Thai, Yoruba, and Zulu) in which the pitch or “tone” of words and syllables makes a difference to word meaning. For example, in Chinese huār (with a high level pitch) means ‘flower’ and hur (with a falling pitch) means ‘picture’. In non-tonal languages (like English or Spanish), pitch is only used at the sentence level, for emphasis and overall meanings like questioning. Roughly half the languages in the world are tonal and half are non-tonal, but they’re fairly unevenly distributed: tone languages are the norm in sub-Saharan Africa and are common in Southeast Asia and among Native American languages especially in parts of Central and South America. Non-tone languages are the norm in Europe and Central, South and West Asia, and among the aboriginal languages of Australia.
http://www.lel.ed.ac.uk/~bob/tonegenessummary.html

The World Atlas of Language Structures (WALS) is a large database of structural (phonological, grammatical, lexical) properties of languages gathered from descriptive materials (such as reference grammars) by a team of 55 authors (many of them the leading authorities on the subject).
http://wals.info/

specifically, see this chapter:
Chapter 2: Vowel Quality Inventories
by Ian Maddieson
Some excerpts:

1. Introduction
This chapter discusses the number of vowel contrasts in the inventory of sounds in languages.

2. Establishing the values.
When vowel qualities are counted in this way in the sample of languages surveyed for this chapter, the average number of vowels in a language is just fractionally below 6. The smallest vowel quality inventory recorded is 2 and the largest 14.

3. Geographical distribution
There are strong areal patterns in the distribution of vowel quality inventories. Not surprisingly, languages with average inventory sizes are the most widely scattered. In just a few areas, southern Africa being one, they occur almost to the exclusion of the other two types.

http://wals.info/chapter/2
SteamKing
#3
Jul12-13, 03:29 AM
Emeritus
Sci Advisor
HW Helper
Thanks
PF Gold
P: 6,286
Quote Quote by Avichal View Post
English has the letter 'e' with most frequency. Other language also have some of their alphabets appearing more frequently in text hat others. Why is this? Why don't all letters appear with equal frequency?
Do humans speak vowels more comfortably? What exactly is the reason?
Why should letters appear with equal frequency? Do the sounds of a language occur with equal frequency?

Vowels are used to sound out the consonants, at least in Indo-European languages. English has 5 vowels and 21 consonants. Other languages will have a slightly different mix.

Frequency analysis is one tool which can be used to attack ciphered messages. Other tools are needed along with FA to produce a complete decipherment.

Avichal
#4
Jul13-13, 04:18 AM
P: 283
Why frequency analysis works?

Quote Quote by SteamKing View Post
Why should letters appear with equal frequency? Do the sounds of a language occur with equal frequency?

Vowels are used to sound out the consonants, at least in Indo-European languages. English has 5 vowels and 21 consonants. Other languages will have a slightly different mix.

Frequency analysis is one tool which can be used to attack ciphered messages. Other tools are needed along with FA to produce a complete decipherment.
Yes, I know it's used to attack ciphered messages. Actually this question arose from that very context. I was curious to know why some letters had more frequency?
The question perhaps requires the knowledge of how human speech works. Can you pleas explain why this work
SteamKing
#5
Jul13-13, 08:36 AM
Emeritus
Sci Advisor
HW Helper
Thanks
PF Gold
P: 6,286
All I can say about FA development comes from this article:

http://en.wikipedia.org/wiki/Frequency_analysis

See the section on History and Usage.

If you are really interested in cryptography and ciphers, I recommend the book by Kahn (in the References portion of the same article.)

However, it does stand to reason that the occurrence of letters in written text, like a lot of things, would have some statistical distribution, given enough samples of text written in the same language. Some clever person recognized this in the mists of time, before statistical analysis was ever thought of.
Avichal
#6
Jul16-13, 02:45 AM
P: 283
Quote Quote by SteamKing View Post
All I can say about FA development comes from this article:

http://en.wikipedia.org/wiki/Frequency_analysis

See the section on History and Usage.

If you are really interested in cryptography and ciphers, I recommend the book by Kahn (in the References portion of the same article.)

However, it does stand to reason that the occurrence of letters in written text, like a lot of things, would have some statistical distribution, given enough samples of text written in the same language. Some clever person recognized this in the mists of time, before statistical analysis was ever thought of.
Yes, there has to be some statistical distribution but I find the sound of 'e', 'a' the most common across many languages. I am trying to find a reason behind this


Register to reply

Related Discussions
Advantage of frequency domain analysis? Electrical Engineering 12
How Frequency Division Multiplexing works General Physics 3
Noise Frequency Analysis General Physics 6
Phonon frequency analysis Engineering, Comp Sci, & Technology Homework 0
Power analysis of low frequency component Electrical Engineering 0