# Funzioni di ordinamento per crescita asintotica

35

Supponiamo che io abbia un elenco di funzioni, ad esempio

$\phantom{\rule{2em}{0ex}}{n}^{\mathrm{log}\mathrm{log}\left(n\right)},{2}^{n},n!,{n}^{3},n\mathrm{ln}n,\dots$$\qquad n^{\log \log(n)}, 2^n, n!, n^3, n \ln n, \dots$

Come posso ordinarli asintoticamente, cioè dopo la relazione definita da

$\phantom{\rule{2em}{0ex}}f{\le }_{O}g\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in O\left(g\right)$$\qquad f \leq_O g \iff f \in O(g)$ ,

supponendo che siano effettivamente comparabili a coppie (vedi anche qui )? L'uso della definizione di $O$$O$ sembra imbarazzante ed è spesso difficile provare l'esistenza di costanti adatte $c$$c$ e ${n}_{0}$$n_0$ .

Si tratta di misure di complessità, quindi siamo interessati al comportamento asintotico come $n\to +\mathrm{\infty }$$n \to +\infty$ e supponiamo che tutte le funzioni assumano solo valori non negativi ( $\mathrm{\forall }n,f\left(n\right)\ge 0$$\forall n, f(n) \ge 0$ ).

4
Dal momento che l'OP non è più tornato, sto rimuovendo le cose localizzate e ne faccio una domanda di riferimento.
Raffaello

Risposte:

48

Se si desidera una prova rigorosa, il seguente lemma è spesso utile resp. più utile delle definizioni.

Se $c=\underset{n\to \mathrm{\infty }}{lim}\frac{f\left(n\right)}{g\left(n\right)}$$c = \lim_{n\to\infty} \frac{f(n)}{g(n)}$ esiste, quindi

• $c=0 \qquad \ \,\iff f \in o(g)$ ,
• $c\in \left(0,\mathrm{\infty }\right)\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in \mathrm{\Theta }\left(g\right)$$c \in (0,\infty) \iff f \in \Theta(g)$ and
• $c=\infty \quad \ \ \ \iff f \in \omega(g)$.

With this, you should be able to order most of the functions coming up in algorithm analysis¹. As an exercise, prove it!

Of course you have to be able to calculate the limits accordingly. Some useful tricks to break complicated functions down to basic ones are:

• Express both functions as ${e}^{\dots }$$e^{\dots}$ and compare the exponents; if their ratio tends to $0$$0$ or $\mathrm{\infty }$$\infty$, so does the original quotient.
• More generally: if you have a convex, continuously differentiable and strictly increasing function $h$$h$ so that you can re-write your quotient as

$\phantom{\rule{2em}{0ex}}\frac{f\left(n\right)}{g\left(n\right)}=\frac{h\left({f}^{\ast }\left(n\right)\right)}{h\left({g}^{\ast }\left(n\right)\right)}$$\qquad \displaystyle \frac{f(n)}{g(n)} = \frac{h(f^*(n))}{h(g^*(n))}$,

with ${g}^{\ast }\in \mathrm{\Omega }\left(1\right)$$g^* \in \Omega(1)$ and

$\phantom{\rule{2em}{0ex}}\underset{n\to \mathrm{\infty }}{lim}\frac{{f}^{\ast }\left(n\right)}{{g}^{\ast }\left(n\right)}=\mathrm{\infty }$$\qquad \displaystyle \lim_{n \to \infty} \frac{f^*(n)}{g^*(n)} = \infty$,

then

$\phantom{\rule{2em}{0ex}}\underset{n\to \mathrm{\infty }}{lim}\frac{f\left(n\right)}{g\left(n\right)}=\mathrm{\infty }$$\qquad \displaystyle \lim_{n \to \infty} \frac{f(n)}{g(n)} = \infty$.

See here for a rigorous proof of this rule (in German).

• Consider continuations of your functions over the reals. You can now use L'Hôpital's rule; be mindful of its conditions²!

• Have a look at the discrete equivalent, Stolz–Cesàro.
• When factorials pop up, use Stirling's formula:

$\phantom{\rule{2em}{0ex}}n!\sim \sqrt{2\pi n}{\left(\frac{n}{e}\right)}^{n}$$\qquad \displaystyle n! \sim \sqrt{2 \pi n} \left(\frac{n}{e}\right)^n$

It is also useful to keep a pool of basic relations you prove once and use often, such as:

• i logaritmi diventano più lenti dei polinomi, cioè

$\phantom{\rule{2em}{0ex}}\left(\mathrm{log}n{\right)}^{\alpha }\in o\left({n}^{\beta }\right)$$\qquad\displaystyle (\log n)^\alpha \in o(n^\beta)$$\alpha ,\beta >0$$\alpha, \beta > 0$

• ordine dei polinomi:

$\phantom{\rule{2em}{0ex}}{n}^{\alpha }\in o\left({n}^{\beta }\right)$$\qquad\displaystyle n^\alpha \in o(n^\beta)$ for all $\alpha <\beta$$\alpha < \beta$.

• polynomials grow slower than exponentials:

$\phantom{\rule{2em}{0ex}}{n}^{\alpha }\in o\left({c}^{n}\right)$$\qquad\displaystyle n^\alpha \in o(c^n)$ for all $\alpha$$\alpha$ and $c>1$$c > 1$.

It can happen that above lemma is not applicable because the limit does not exist (e.g. when functions oscillate). In this case, consider the following characterisation of Landau classes using limes superior/inferior:

With ${c}_{s}:=\underset{n\to \mathrm{\infty }}{lim sup}\frac{f\left(n\right)}{g\left(n\right)}$$c_s := \limsup_{n \to \infty} \frac{f(n)}{g(n)}$ we have

• $0\le {c}_{s}<\mathrm{\infty }\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in O\left(g\right)$$0 \leq c_s < \infty \iff f \in O(g)$ and
• ${c}_{s}=0\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in o\left(g\right)$$c_s = 0 \iff f \in o(g)$.

With ${c}_{i}:=\underset{n\to \mathrm{\infty }}{lim inf}\frac{f\left(n\right)}{g\left(n\right)}$$c_i := \liminf_{n \to \infty} \frac{f(n)}{g(n)}$ we have

• $0<{c}_{i}\le \mathrm{\infty }\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in \mathrm{\Omega }\left(g\right)$$0 < c_i \leq \infty \iff f \in \Omega(g)$ and
• ${c}_{i}=\mathrm{\infty }\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in \omega \left(g\right)$$c_i = \infty \iff f \in \omega(g)$.

Furthermore,

• $0<{c}_{i},{c}_{s}<\mathrm{\infty }\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\in \mathrm{\Theta }\left(g\right)\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}g\in \mathrm{\Theta }\left(f\right)$$0 < c_i,c_s < \infty \iff f \in \Theta(g) \iff g \in \Theta(f)$ and
• ${c}_{i}={c}_{s}=1\phantom{\rule{thickmathspace}{0ex}}⟺\phantom{\rule{thickmathspace}{0ex}}f\sim g$$c_i = c_s = 1 \iff f \sim g$.

Check here and here if you are confused by my notation.

¹ Nota bene: My colleague wrote a Mathematica function that does this successfully for many functions, so the lemma really reduces the task to mechanical computation.

@Juho Not publicly, afaik, but it's elementary to write yourself; compute Limit[f[n]/g[n], n -> Infinity] and perform a case distinction.
Raphael

20

Another tip: sometimes applying a monotone function (like log or exp) to the functions makes things clearer.

5
This should be done carefully: $2n\in O\left(n\right)$$2n\in O(n)$, but ${2}^{2n}\notin O\left({2}^{n}\right)$$2^{2n}\notin O(2^n)$.
Shaull

2
Seconded. The "apply monotone function" thing seems to be some kind of folklore which does not work in general. We have been working on sufficient criteria and have been come up with what I posted in the latest revision of my answer.
Raphael

17

Skiena provides a sorted list of the dominance relations between the most common functions in his book, The Algorithm Design Manual:



$n!\gg {c}^{n}\gg {n}^{3}\gg {n}^{2}\gg {n}^{1+ϵ}\gg n\mathrm{lg}n\gg n\gg {n}^{1/2}$
$$≫lg2n≫lgn≫lgnlglgn≫lglgn≫α(n)≫1≫lg2⁡n≫lg⁡n≫lg⁡nlg⁡lg⁡n≫lg⁡lg⁡n≫α(n)≫1 \gg \lg^2n \gg \lg n \gg \frac{\lg n}{\lg\lg n} \gg \lg\lg n \gg \alpha(n) \gg 1$$

Here $$α(n)α(n)\alpha(n)$$ denotes the inverse Ackermann function.

That's an oddly specific list. Many of the relations (whatever $\gg$$\gg$ means exactly) can be summarised to a handful of more general lemmata.
Raphael

It's his notation for a dominance relation.
Robert S. Barnes

11

Tip: draw graphs of these functions using something like Wolfram Alpha to get a feeling for how they grow. Note that this is not very precise, but if you try it for sufficiently large numbers, you should see the comparative patterns of growth. This of course is no substitute for a proof.

E.g., try: plot log(log(n)) from 1 to 10000 to see an individual graph or plot log(log(n)) and plot log(n) from 1 to 10000 to see a comparison.

9
Should we really recommend vodoo?
Raphael

+1 for suggesting to draw graphs of the functions, although the linked graphs are rather confusing unless you know what they mean.
Tsuyoshi Ito

1
Take a graph as a hint what you might want to prove. That hint may be wrong of course.
gnasher729

8

I suggest proceeding according to the definitions of various notations. Start with some arbitrary pair of expressions, and determine the order of those, as outlined below. Then, for each additional element, find its position in the sorted list using binary search and comparing as below. So, for example, let's sort ${n}^{\mathrm{log}\mathrm{log}n}$$n^{\log\log n}$ and ${2}^{n}$$2^n$, the first two functions of n, to get the list started.

We use the property that $n={2}^{\mathrm{log}n}$$n = 2^{\log n}$ to rewrite the first expression as ${n}^{\mathrm{log}\mathrm{log}n}=\left({2}^{\mathrm{log}n}{\right)}^{\mathrm{log}\mathrm{log}n}={2}^{\mathrm{log}n\mathrm{log}\mathrm{log}n}$$n^{\log\log n} = (2^{\log n})^{\log\log n} = 2^{\log n\log\log n}$. We could then proceed to use the definition to show that ${n}^{\mathrm{log}\mathrm{log}n}={2}^{\mathrm{log}n\mathrm{log}\mathrm{log}n}\in o\left({2}^{n}\right)$$n^{\log\log n} = 2^{\log n\log\log n} \in o(2^n)$, since for any constant $c>0$$c > 0$, there is an ${n}_{0}$$n_0$ such that for $n\ge {n}_{0}$$n \geq n_0$, $c\left({n}^{\mathrm{log}\mathrm{log}n}\right)=c\left({2}^{\mathrm{log}n\mathrm{log}\mathrm{log}n}\right)<{2}^{n}$$c(n^{\log\log n}) = c(2^{\log n\log\log n}) < 2^n$.

Next, we try ${3}^{n}$$3^n$. We compare it to ${2}^{n}$$2^n$, the largest element we have placed so far. Since ${3}^{n}=\left({2}^{\mathrm{log}3}{\right)}^{n}={2}^{n\mathrm{log}3}$$3^n = (2^{\log 3})^n = 2^{n\log3}$, we similarly show that ${2}^{n}\in o\left({3}^{n}\right)=o\left({2}^{n\mathrm{log}3}\right)$$2^n \in o(3^n) = o(2^{n \log 3})$.

Etc.

2

Here a list from Wikipedia, The lower in the table the bigger complexity class; 

note : $$poly(x)=xO(1)poly(x)=xO(1)poly(x) = x^{\mathcal{O}(1)}$$

1
Also, interesting how the table suggests that ${2}^{n\mathrm{log}n}\in o\left(n!\right)$$2^{n \log n} \in o(n!)$. While the table you link to is somewhat accurate, the one linked there (and which you copied) is about complexity classes, which is not a helpful thing to mix in here. Landau notation is not about "time".
Raphael

1
I put this so the name of the complexity classes can be talked directly here. Yes, Landau is more about a specific type of algorithm in Cryptography.
kelalaka

1
I object to some of @Raphael's views. I have been a mathematician and an instructor for many years. I believe, apart from proving those things, a big table like this increases people's intuition easily and greatly. And the names of the asymptotic classes help people remember and communicate a lot.
Apass.Jack

1
@Apass.Jack In my teaching experience, when given a table many students will learn it by heart and fail to order wrt any function not in the table. Note how that effect seems to account for many of questions regarding asymptotic growth that land on this site. That said, of course we'll use lemmata implied by the table if it makes proofs easier, but that comes after learning how to proof the table. (To emphasize that point, people who come here don't need help reading stuff off a table. They need help proving relations.)
Raphael

1
@kelalaka "Yes, Landau is more about a specific type of algorithm in Cryptography." -- that doesn't even make sense. Landau notation is a shorthand to describe properties of mathematical functions. It has nothing to do with algorithms let alone cryptography, per se.
Raphael
Utilizzando il nostro sito, riconosci di aver letto e compreso le nostre Informativa sui cookie e Informativa sulla privacy.