Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Big-O and recurrence equations.

  1. Feb 10, 2007 #1
    Can anyone point me to resources showing me how to prove recurance equations like:

    [itex]I(n) \le c[/itex] if n = 0
    [itex]d+I(n-1)[/itex] if n >= 1

    i.e. [tex]I(n) \le c + dn[/tex]

    Also for proving things like: [itex]30000n^2 + n \log n[/itex] is [itex] O(n^3)[/itex] using the basic definition of big-O notation ([itex]f(n) \le k \cdot g(n)[/itex] for some k and [itex]n \le n_0[/itex])

    I never took algebra so I never learned this stuff while CS majors did :( It'd be helpful if I knew what to search for.
    Last edited: Feb 10, 2007
  2. jcsd
  3. Feb 12, 2007 #2

    I'm not sure I entirely understand your example but generally recurrence relations can be proven using mathematical induction. Studying proof by mathematical induction will help. The same of method of proof that allows us to prove things like:

    [tex]\sum_{i=1}^{n} i = \frac {n(n + 1)} {2}[/tex]

    Also allows us to prove recurrence relations (that is one). There are some variations but at the risk of oversimplifying, the general idea is to establish the truth of a proposition by showing that it follows from smaller instances of the same proposition, as long as the truth of the smallest instance can be explicitly established. Proofs by induction normally involve establishing the truth of a base case and then the application of an inductive hypothesis (inductive step) which we assume to be true to show the truth of the proposition, no matter what size, follows from the truth of all the previous propositions.

    Regarding big-O, the whole purpose for it's existence is to provide a "fuzzy ruler" with which to measure the efficiency of algorithms. With that in mind, we normally want leave out tricky things like constants and such. For easy examples like the one you listed it is sufficient to note that:

    [itex]30000n^2 + n \log n \epsilon O(n^3)[/itex]

    By noting that:

    [itex]30000n^2 + n \log n < n^2 + n^2 = 2n^2 < 2n^3[/itex]

    Thus, [itex]30000n^2 + n \log n \epsilon O(n^3)[/itex] for very large n.

    For harder functions it helps to study the asymptotic behavior of the functions in question. That is to say, to see if [itex]f(n) \epsilon O(g)[/itex] we look at the limit:

    [itex]\lim_{n\rightarrow\infty}\frac {f(n)} {g(n)}[/itex]

    From there we have a couple possibilities. If the limit exists and is finite we know that [itex]f(n) \epsilon O(g)[/itex]. If the limit is infinity then [itex]f(n) \neg\epsilon O(g)[/itex]. Additionally if the limit is finite and non zero we can say that [itex]O(f) = O(g)[/itex]
    Last edited: Feb 12, 2007
  4. Feb 13, 2007 #3


    User Avatar
    Science Advisor

    Are you guys sure about that? I could have sworn it was [tex]O(n^2)[/tex]
  5. Feb 13, 2007 #4
    I've used induction to prove convergence in sequences or formulas like you posted but I've never used it in the context of CS.

    So parabrham you are saying basically [itex]n log n \le n^2[/itex], adding the two [itex]n^2[/itex]'s (drop 30000) and saying theyre less than [itex]x^3[/itex]?? I thought we always went with the best possible order...

    I would have said O(n^2) as well but they also say prove [itex]30000n^2[/itex] is [itex]O(n^3 \log n)[/itex]?!? I don't understand how [itex]30000n^2[/itex] isn't simply [itex]O(n^2)[/itex].
    Last edited: Feb 13, 2007
  6. Feb 13, 2007 #5


    User Avatar
    Science Advisor
    Homework Helper

  7. Feb 14, 2007 #6


    User Avatar
    Science Advisor

    Yes that's discussing recurrence relations. But the [itex]30000n^2 + n \log n[/itex] was not given as part of a recurrence relation but just a simple function of n.

    [tex]30000n^2 + n \log n < 30001 n^2[/tex] and is therefore [tex]O(n^2)[/tex]
    Last edited: Feb 14, 2007
  8. Feb 19, 2007 #7
    You guys are correct. Sorry I was away for a bit.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook