Help with Kolmogorov Complexity proof

AI Thread Summary
The discussion revolves around Kolmogorov complexity, specifically its incomputability. The core concept is that the Kolmogorov complexity K(s) of a string s is defined as the length of the shortest program that generates that string. A proof by contradiction suggests that K(s) is incomputable by demonstrating a program that searches for strings with high complexity. However, a participant proposes an alternative approach to compute K(s) by generating and executing all possible programs to find a match for the string. Initially, they believe this method could compute K(s), but upon further reflection, they realize the flaw lies in assuming that the program that halts first will yield the shortest code. They conclude that while their approach can generate potential answers, it cannot definitively determine when the best answer is found, highlighting the complexities of the Halting problem and the nature of Kolmogorov complexity.
Fooality
Messages
197
Reaction score
45
Hey, has anyone here done Kolmogorov complexity? I hadn't, and I was just reading the Wikipedia page to start learning it. The core concept makes sense: The Kolmogorov complexity for a a string s, K(s) is equivalent to the length of a program that generates that string (It can be any math object, not just strings.) So a string that's long but with a simple pattern behind it will have a short program and small K, where a random string will have larger K.

Then I got to this proof, which claims to show the incomputability of K. Its a proof by contradiction, here given in pseudo Python, and available on the Wikipedia page in pseudo Pascal.
Python:
function GenerateComplexString():
     for i in range(infinity):
         for each string s of length exactly i:
            if KolmogorovComplexity(s) >= 8000000000
                return s
This code enumerates through the set of all strings as s, until it finds the first one with K(s) > than some high number (in this case 8000000000) and outputs it. Since this program is shorter than 8000000000, its a contradiction, and they used it to say that K(s), (here written as KolmogorovComplexity(s)) must not be computable.

The problem is, using a similar dirty trick, I can construct a KolmogorovComplexity function that clearly is computable. The idea behind it is to generate and run all possible programs and return the first one that produces the string. It looks like this:
Python:
function Execute(code, s):
    string output = run(code)
    if output = s:
      addToPotentialAnswers(len(code), code)

function KolmogorovComplexity(s):
     for i in range(infinity):
         LaunchOnNewVirtualProcessor(Execute, (ConvertToBinary(i), s))

The LaunchOnNewVirtualProcessor runs Execute on a "virtual processor", and exploits the fact that any Turing machine can simulate any other number of Turing machines, so if the given code doesn't halt, it slows it down but the whole process keeps going. (Its not assumed all every number representing a program generates a number or halts) So long as the strings are finite, as they are in their function, and the programs that output strings halt (which is totally reasonable), than this works, and everything becomes incredibly confusing.

So what am I getting wrong, there's got to be something, because their proof seems totally sound, the more I go over it in my mind.

edit: Changed my code, can't assume that the process which ends first has the shortest code.
 
Last edited:
Technology news on Phys.org
Okay, I think I might see it now. My program generates a list of potential answers, but you can never quite say when the its done, when the potential answer is the best answer. So though my program will generate at some point the answer, you can never quite say when that is. That must be it.

Edit: Yeah, that's totally it. I see it now. Mods can delete if they want. My error was in thinking there was some sufficiently huge amount of time within which a program with the right answer would halt, but not true. The best program might be some small thing that simulates the entire universe to find the best answer. A really interesting illustration of how deep the Halting problem is.
 
Last edited:
  • Like
Likes jim mcnamara
Dear Peeps I have posted a few questions about programing on this sectio of the PF forum. I want to ask you veterans how you folks learn program in assembly and about computer architecture for the x86 family. In addition to finish learning C, I am also reading the book From bits to Gates to C and Beyond. In the book, it uses the mini LC3 assembly language. I also have books on assembly programming and computer architecture. The few famous ones i have are Computer Organization and...
I have a quick questions. I am going through a book on C programming on my own. Afterwards, I plan to go through something call data structures and algorithms on my own also in C. I also need to learn C++, Matlab and for personal interest Haskell. For the two topic of data structures and algorithms, I understand there are standard ones across all programming languages. After learning it through C, what would be the biggest issue when trying to implement the same data...
Back
Top