Help with Kolmogorov Complexity proof

In summary, the conversation discusses the concept of Kolmogorov complexity and its incomputability. The core concept of Kolmogorov complexity is that it measures the length of a program that generates a string, with shorter programs indicating simpler patterns. The conversation then introduces a proof by contradiction, using a pseudo Python code, to show the incomputability of Kolmogorov complexity. However, the speaker presents their own code that they believe can compute Kolmogorov complexity, by generating and running all possible programs. They realize their mistake and conclude that the halting problem makes it impossible to determine the shortest program for a given string.
  • #1
Fooality
196
42
Hey, has anyone here done Kolmogorov complexity? I hadn't, and I was just reading the Wikipedia page to start learning it. The core concept makes sense: The Kolmogorov complexity for a a string s, K(s) is equivalent to the length of a program that generates that string (It can be any math object, not just strings.) So a string that's long but with a simple pattern behind it will have a short program and small K, where a random string will have larger K.

Then I got to this proof, which claims to show the incomputability of K. Its a proof by contradiction, here given in pseudo Python, and available on the Wikipedia page in pseudo Pascal.
Python:
function GenerateComplexString():
     for i in range(infinity):
         for each string s of length exactly i:
            if KolmogorovComplexity(s) >= 8000000000
                return s
This code enumerates through the set of all strings as s, until it finds the first one with K(s) > than some high number (in this case 8000000000) and outputs it. Since this program is shorter than 8000000000, its a contradiction, and they used it to say that K(s), (here written as KolmogorovComplexity(s)) must not be computable.

The problem is, using a similar dirty trick, I can construct a KolmogorovComplexity function that clearly is computable. The idea behind it is to generate and run all possible programs and return the first one that produces the string. It looks like this:
Python:
function Execute(code, s):
    string output = run(code)
    if output = s:
      addToPotentialAnswers(len(code), code)

function KolmogorovComplexity(s):
     for i in range(infinity):
         LaunchOnNewVirtualProcessor(Execute, (ConvertToBinary(i), s))

The LaunchOnNewVirtualProcessor runs Execute on a "virtual processor", and exploits the fact that any Turing machine can simulate any other number of Turing machines, so if the given code doesn't halt, it slows it down but the whole process keeps going. (Its not assumed all every number representing a program generates a number or halts) So long as the strings are finite, as they are in their function, and the programs that output strings halt (which is totally reasonable), than this works, and everything becomes incredibly confusing.

So what am I getting wrong, there's got to be something, because their proof seems totally sound, the more I go over it in my mind.

edit: Changed my code, can't assume that the process which ends first has the shortest code.
 
Last edited:
Technology news on Phys.org
  • #2
Okay, I think I might see it now. My program generates a list of potential answers, but you can never quite say when the its done, when the potential answer is the best answer. So though my program will generate at some point the answer, you can never quite say when that is. That must be it.

Edit: Yeah, that's totally it. I see it now. Mods can delete if they want. My error was in thinking there was some sufficiently huge amount of time within which a program with the right answer would halt, but not true. The best program might be some small thing that simulates the entire universe to find the best answer. A really interesting illustration of how deep the Halting problem is.
 
Last edited:
  • Like
Likes jim mcnamara

1. What is Kolmogorov Complexity?

Kolmogorov Complexity is a measure of the complexity or amount of information contained in a string of data. It is based on the idea that the complexity of a string can be measured by the length of the shortest computer program that can produce that string.

2. Why is Kolmogorov Complexity important?

Kolmogorov Complexity is important because it allows us to quantify the amount of information in a data set and distinguish between random and meaningful data. It is used in various fields such as computer science, algorithmic information theory, and artificial intelligence.

3. How is Kolmogorov Complexity calculated?

Kolmogorov Complexity is calculated by finding the length of the shortest program that can produce a given string of data. This program must be able to run on a universal computer, which is a hypothetical computer with unlimited memory and processing power.

4. What is the relationship between Kolmogorov Complexity and randomness?

Kolmogorov Complexity and randomness are closely related. Random data has a high Kolmogorov Complexity, as it cannot be compressed or described by a simple program. On the other hand, meaningful or structured data has a lower Kolmogorov Complexity, as it can be described by a shorter program.

5. How is Kolmogorov Complexity used in proofs?

Kolmogorov Complexity is often used in proofs to show that a given problem is undecidable or unsolvable. This is done by demonstrating that the Kolmogorov Complexity of the problem is greater than the complexity of the shortest program that can solve it, making it impossible to solve in a finite amount of time.

Similar threads

  • Programming and Computer Science
Replies
1
Views
1K
  • Programming and Computer Science
Replies
8
Views
2K
  • Programming and Computer Science
Replies
3
Views
1K
  • Programming and Computer Science
Replies
3
Views
2K
  • Programming and Computer Science
Replies
29
Views
2K
  • Programming and Computer Science
Replies
2
Views
1K
  • Engineering and Comp Sci Homework Help
Replies
3
Views
950
  • Programming and Computer Science
Replies
2
Views
779
  • Programming and Computer Science
Replies
1
Views
754
  • Programming and Computer Science
Replies
29
Views
3K
Back
Top