Come trovare 5 valori ripetuti in O (n) tempo?

15

Supponiamo di avere un array di dimensioni $n\ge 6$$n \geq 6$ contenente numeri interi da $1$$1$ a $n-5$$n − 5$ , inclusi, con esattamente cinque ripetuti. Ho bisogno di proporre un algoritmo in grado di trovare i numeri ripetuti in $O\left(n\right)$$O(n)$ tempo. Non posso, per la vita di me, pensare a niente. Penso che l'ordinamento, nella migliore delle ipotesi, sarebbe $O\left(n\mathrm{log}n\right)$$O(n\log n)$ ? Quindi attraversare l'array sarebbe $O\left(n\right)$$O(n)$ , risultando in $O\left({n}^{2}\mathrm{log}n\right)$$O(n^2\log n)$. Tuttavia, non sono davvero sicuro se l'ordinamento sarebbe necessario in quanto ho visto alcune cose complicate con elenco collegato, code, pile, ecc.

16
$O\left(n\mathrm{log}n\right)+O\left(n\right)$$O(n \log n) + O(n)$ non è$O\left({n}^{2}\mathrm{log}n\right)$$O(n^2 \log n)$ . È$O\left(n\mathrm{log}n\right)$$O(n \log n)$ . Sarebbe$O\left({n}^{2}\mathrm{log}n\right)$$O(n^2 \log n)$ se facessi l'ordinamento n volte.
Fondi Monica's Lawsuit,

1

1
@leftaroundabout Questi algoritmi sono dove è la dimensione dell'array e è la dimensione dell'insieme di input. poiché questi algoritmi funzionano in $O\left(k\cdot n\right)$$O(k\cdot n)$$n$$n$$k$$k$$k=n-constant$$k=n-constant$$O\left({n}^{2}\right)$$O(n^2)$
romano Gräf

4
@ RomanGräf sembra che la situazione reale sia questa: gli algoritmi funzionano in , dove è la dimensione del dominio. Quindi, per un problema come gli OP, si ottiene lo stesso se si utilizza un tale algoritmo sul dominio di dimensioni o un algoritmo tradizionale su un dominio di dimensioni illimitate. Ha anche senso. $O\left(\mathrm{log}k\cdot n\right)$$O(\log k \cdot n)$$k$$k$$n$$n$$O\left(n\cdot \mathrm{log}n\right)$$O(n\cdot \log n)$
lasciato circa l'

5
Per , l'unico numero consentito è , secondo la tua descrizione. Ma poi dovrebbe essere ripetuto sei, non cinque, volte. $n=6$$n=6$$1$$1$$1$$1$
Alex Reinking,

Risposte:

22

È possibile creare un ulteriore array di dimensioni . Inizialmente impostare tutti gli elementi dell'array su . Quindi, passare attraverso l'array di input e aumentare di 1 per ogni . Dopodiché controlli semplicemente l'array : passa sopra e se allora viene ripetuto. Lo risolvi in $B$$B$$n$$n$$0$$0$$A$$A$$B\left[A\left[i\right]\right]$$B[A[i]]$$i$$i$$B$$B$$A$$A$$B\left[A\left[i\right]\right]>1$$B[A[i]] > 1$$A\left[i\right]$$A[i]$$O\left(n\right)$$O(n)$ time al costo della memoria che è $O\left(n\right)$$O(n)$ e perché i tuoi numeri interi sono compresi tra e .$1$$1$$n-5$$n-5$

26

La soluzione nella risposta di fade2black è quella standard, ma utilizza lo spazio . Puoi migliorare questo nello spazio come segue:$O\left(n\right)$$O(n)$$O\left(1\right)$$O(1)$

1. Let the array be $A\left[1\right],\dots ,A\left[n\right]$$A[1],\ldots,A[n]$. For $d=1,\dots ,5$$d=1,\ldots,5$, compute ${\sigma }_{d}=\sum _{i=1}^{n}A\left[i{\right]}^{d}$$\sigma_d = \sum_{i=1}^n A[i]^d$.
2. Compute ${\tau }_{d}={\sigma }_{d}-\sum _{i=1}^{n-5}{i}^{d}$$\tau_d = \sigma_d - \sum_{i=1}^{n-5} i^d$ (you can use the well-known formulas to compute the latter sum in $O\left(1\right)$$O(1)$). Note that ${\tau }_{d}={m}_{1}^{d}+\cdots +{m}_{5}^{d}$$\tau_d = m_1^d + \cdots + m_5^d$, where ${m}_{1},\dots ,{m}_{5}$$m_1,\ldots,m_5$ are the repeated numbers.
3. Compute the polynomial $P\left(t\right)=\left(t-{m}_{1}\right)\cdots \left(t-{m}_{5}\right)$$P(t) = (t-m_1)\cdots(t-m_5)$. The coefficients of this polynomial are symmetric functions of ${m}_{1},\dots ,{m}_{5}$$m_1,\ldots,m_5$ which can be computed from ${\tau }_{1},\dots ,{\tau }_{5}$$\tau_1,\ldots,\tau_5$ in $O\left(1\right)$$O(1)$.
4. Find all roots of the polynomial $P\left(t\right)$$P(t)$ by trying all $n-5$$n-5$ possibilities.

This algorithm assumes the RAM machine model, in which basic arithmetic operations on $O\left(\mathrm{log}n\right)$$O(\log n)$-bit words take $O\left(1\right)$$O(1)$ time.

Another way to formulate this solution is along the following lines:

1. Calculate ${x}_{1}=\sum _{i=1}^{n}A\left[i\right]$$x_1 = \sum_{i=1}^n A[i]$, and deduce ${y}_{1}={m}_{1}+\cdots +{m}_{5}$$y_1 = m_1 + \cdots + m_5$ using the formula ${y}_{1}={x}_{1}-\sum _{i=1}^{n-5}i$$y_1 = x_1 - \sum_{i=1}^{n-5} i$.
2. Calculate ${x}_{2}=\sum _{1\le i$x_2 = \sum_{1 \leq i < j \leq} A[i] A[j]$ in $O\left(n\right)$$O(n)$ using the formula
${x}_{2}=\left(A\left[1\right]\right)A\left[2\right]+\left(A\left[1\right]+A\left[2\right]\right)A\left[3\right]+\left(A\left[1\right]+A\left[2\right]+A\left[3\right]\right)A\left[4\right]+\cdots +\left(A\left[1\right]+\cdots +A\left[n-1\right]\right)A\left[n\right].$
3. ${y}_{2}=\sum _{1\le i$y_2 = \sum_{1 \leq i < j \leq 5} m_i m_j$
${y}_{2}={x}_{2}-\sum _{1\le i
4. Calculate ${x}_{3},{x}_{4},{x}_{5}$$x_3,x_4,x_5$ and deduce ${y}_{3},{y}_{4},{y}_{5}$$y_3,y_4,y_5$ along similar lines.
5. The values of ${y}_{1},\dots ,{y}_{5}$$y_1,\ldots,y_5$ are (up to sign) the coefficients of the polynomial $P\left(t\right)$$P(t)$ from the preceding solution.

$d$$d$$O\left({d}^{2}n\right)$$O(d^2n)$$O\left({d}^{2}\right)$$O(d^2)$ space, which performs $O\left(dn\right)$$O(dn)$ arithmetic operations on integers of bit-length $O\left(d\mathrm{log}n\right)$$O(d\log n)$, keeping at most $O\left(d\right)$$O(d)$ of these at any given time. (This requires careful analysis of the multiplications we perform, most of which involve one operand of length only $O\left(\mathrm{log}n\right)$$O(\log n)$$O\left(dn\right)$$O(dn)$$O\left(d\right)$$O(d)$ space using modular arithmetic.

Any interpretation of ${\sigma }_{d}$$\sigma_d$ and ${\tau }_{d}$$\tau_d$, $P\left(t\right)$$P(t)$, ${m}_{i}$$m_i$ and so on? Why $d\in \left\{1,2,3,4,5\right\}$$d \in \{1, 2, 3, 4, 5\}$?
styrofoam fly

3
The insight behind the solution is the summing trick, which appears in many exercises (for example, how do you find the missing element from an array of length $n-1$$n-1$ containing all but one of the numbers $1,\dots ,n$$1,\ldots,n$?). The summing trick can be used to compute $f\left({m}_{1}\right)+\cdots +f\left({m}_{5}\right)$$f(m_1) + \cdots + f(m_5)$ for an arbitrary function $f$$f$, and the question is which $f$$f$ to choose in order to be able to deduce ${m}_{1},\dots ,{m}_{5}$$m_1,\ldots,m_5$. My answer uses familiar tricks from the elementary theory of symmetric functions.
Yuval Filmus

1
@hoffmale Actually, $O\left({d}^{2}\right)$$O(d^2)$.
Yuval Filmus

1
@hoffmale Each of them takes $d$$d$ machine words.
Yuval Filmus

1
@BurnsBA The problem with this approach is that $\left(n-5\right)\mathrm{#}$$(n-5)\#$ is much larger than $\frac{\left(n-4\right)\left(n-5\right)}{2}$$\frac{(n-4)(n-5)}{2}$. Operations on large numbers are slower.
Yuval Filmus

8

There's also a linear time and constant space algorithm based on partitioning, which may be more flexible if you're trying to apply this to variants of the problem that the mathematical approach doesn't work well on. This requires mutating the underlying array and has worse constant factors than the mathematical approach. More specifically, I believe the costs in terms of the total number of values $n$$n$ and the number of duplicates $d$$d$ are $\mathcal{O}\left(n\mathrm{log}d\right)$$\mathcal{O}(n \log d)$ and $\mathcal{O}\left(d\right)$$\mathcal{O}(d)$ respectively, though proving it rigorously will take more time than I have at the moment.

Algorithm

Start with a list of pairs, where the first pair is the range over the whole array, or $\left[\left(1,n\right)\right]$$[(1, n)]$ if 1-indexed.

Repeat the following steps until the list is empty:

1. Take and remove any pair $\left(i,j\right)$$(i, j)$ from the list.
2. Find the minimum and maximum, $\text{min}$$\text{min}$ and $\text{max}$$\text{max}$, of the denoted subarray.
3. If $\text{min}=\text{max}$$\text{min} = \text{max}$, the subarray consists only of equal elements. Yield its elements except one and skip steps 4 to 6.
4. If $\text{max}-\text{min}=j-i$$\text{max} - \text{min} = j - i$, the subarray contains no duplicates. Skip steps 5 and 6.
5. Partition the subarray around $\frac{\text{min}+\text{max}}{2}$$\frac{\text{min}+\text{max}}{2}$, such that elements up to some index $k$$k$ are smaller than the separator and elements above that index are not.
6. Add $\left(i,k\right)$$(i, k)$ and $\left(k+1,j\right)$$(k + 1, j)$ to the list.

Cursory analysis of time complexity.

Steps 1 to 6 take $\mathcal{O}\left(j-i\right)$$\mathcal{O}(j - i)$ time, since finding the minimum and maximum and partitioning can be done in linear time.

Every pair $\left(i,j\right)$$(i, j)$ in the list is either the first pair, $\left(1,n\right)$$(1, n)$, or a child of some pair for which the corresponding subarray contains a duplicate element. There are at most $d⌈{\mathrm{log}}_{2}n+1⌉$$d \lceil \log_2 n + 1\rceil$ such parents, since each traversal halves the range in which a duplicate can be, so there are at most $2d⌈{\mathrm{log}}_{2}n+1⌉$$2d \lceil \log_2 n + 1\rceil$ total when including pairs over subarrays with no duplicates. At any one time, the size of the list is no more than $2d$$2d$.

Consider the work to find any one duplicate. This consists of a sequence of pairs over an exponentially decreasing range, so the total work is the sum of the geometric sequence, or $\mathcal{O}\left(n\right)$$\mathcal{O}(n)$. This produces an obvious corollary that the total work for $d$$d$ duplicates must be $\mathcal{O}\left(nd\right)$$\mathcal{O}(nd)$, which is linear in $n$$n$.

To find a tighter bound, consider the worst-case scenario of maximally spread out duplicates. Intuitively, the search takes two phases, one where the full array is being traversed each time, in progressively smaller parts, and one where the parts are smaller than $\frac{n}{d}$$\frac{n}{d}$ so only parts of the array are traversed. The first phase can only be $\mathrm{log}d$$\log d$ deep, so has cost $\mathcal{O}\left(n\mathrm{log}d\right)$$\mathcal{O}(n \log d)$, and the second phase has cost $\mathcal{O}\left(n\right)$$\mathcal{O}(n)$ because the total area being searched is again exponentially decreasing.

Thank you for the explanation. Now I understand. A very pretty algorithm!
D.W.

5

Leaving this as an answer because it needs more space than a comment gives.

You make a mistake in the OP when you suggest a method. Sorting a list and then transversing it $O\left(n\mathrm{log}n\right)$$O(n\log n)$ time, not $O\left({n}^{2}\mathrm{log}n\right)$$O(n^2\log n)$ time. When you do two things (that take $O\left(f\right)$$O(f)$ and $O\left(g\right)$$O(g)$ respectively) sequentially then the resulting time complexity is $O\left(f+g\right)=O\left(maxf,g\right)$$O(f+g)=O(\max{f,g})$ (under most circumstances).

In order to multiply the time complexities, you need to be using a for loop. If you have a loop of length $f$$f$ and for each value in the loop you do a function that takes $O\left(g\right)$$O(g)$, then you'll get $O\left(fg\right)$$O(fg)$ time.

So, in your case you sort in $O\left(n\mathrm{log}n\right)$$O(n\log n)$ and then transverse in $O\left(n\right)$$O(n)$ resulting in $O\left(n\mathrm{log}n+n\right)=O\left(n\mathrm{log}n\right)$$O(n\log n+n)=O(n\log n)$. If for each comparison of the sorting algorithm you had to do a computation that takes $O\left(n\right)$$O(n)$, then it would take $O\left({n}^{2}\mathrm{log}n\right)$$O(n^2\log n)$ but that's not the case here.

In case your curious about my claim that $O\left(f+g\right)=O\left(maxf,g\right)$$O(f+g)=O(\max{f,g})$, it's important to note that that's not always true. But if $f\in O\left(g\right)$$f\in O(g)$ or $g\in O\left(f\right)$$g\in O(f)$ (which holds for a whole host of common functions), it will hold. The most common time it doesn't hold is when additional parameters get involved and you get expressions like $O\left({2}^{c}n+n\mathrm{log}n\right)$$O(2^cn+n\log n)$.

3

There's an obvious in-place variant of the boolean array technique using the order of the elements as the store (where arr[x] == x for "found" elements). Unlike the partition variant that can be justified for being more general I'm unsure when you'd actually need something like this, but it is simple.

for idx from n-4 to n
while arr[arr[idx]] != arr[idx]
swap(arr[arr[idx]], arr[idx])


This just repeatedly puts arr[idx] at the location arr[idx] until you find that location already taken, at which point it must be a duplicate. Note that the total number of swaps is bounded by $n$$n$ since each swap makes its exit condition correct.

You're going to have to give some sort of argument that the inner while loop runs in constant time on average. Otherwise, this isn't a linear-time algorithm.
David Richerby

@DavidRicherby It doesn't run constant time on average, but the outer loop only runs 5 times so that's fine. Note that the total number of swaps is bounded by $n$$n$ since each swap makes its exit condition correct, so even if the number of duplicate values increases the total time is still linear (aka. it takes $n$$n$ steps rather than $nd$$nd$) .
Veedrac

Oops, I somehow didn't notice that the outer loop runs a constant number of times! (Edited to include your note about the number of swaps and also so I could reverse my downvote.)
David Richerby

1

Subtract the values you have from the sum $\sum _{i=1}^{n}i=\frac{\left(n-1\right)\cdot n}{2}$$\sum_{i=1}^{n} i = \frac{(n-1) \cdot n}{2}$.

So, after $\mathrm{\Theta }\left(n\right)$$\Theta(n)$ time (assuming arithmetic is O(1), which it isn't really, but let's pretend) you have a sum ${\sigma }_{1}$$\sigma_1$ of 5 integers between 1 and n:

${x}_{1}+{x}_{2}+{x}_{3}+{x}_{4}+{x}_{5}={\sigma }_{1}$$x_1 + x_2 + x_3 + x_4 + x_5 = \sigma_1$

Supposedly, this is no good, right? You can't possibly figure out how to break this up into 5 distinct numbers.

Ah, but this is where it gets to be fun! Now do the same thing as before, but subtract the squares of the values from $\sum _{i=1}^{n}{i}^{2}$$\sum_{i=1}^{n} i^2$. Now you have:

${{x}_{1}}^{2}+{{x}_{2}}^{2}+{{x}_{3}}^{2}+{{x}_{4}}^{2}+{{x}_{5}}^{2}={\sigma }_{2}$${x_1}^2 + {x_2}^2 + {x_3}^2 + {x_4}^2 + {x_5}^2 = \sigma_2$

See where I'm going with this? Do the same for powers 3, 4 and 5 and you have yourself 5 independent equations in 5 variables. I'm pretty sure you can solve for $\stackrel{\to }{x}$$\vec{x}$.

Caveats: Arithmetic is not really O(1). Also, you need a bit of space to represent your sums; but not as much as you would imagine - you can do most everything modularly, as long as you have, oh, $⌈\mathrm{log}\left(5{n}^{6}\right)⌉$$\lceil\log(5n^6)\rceil$ bits; that should do it.

Doesn't @YuvalFilmus propose the same solution?

@fade2black: Oh, yes, it does, sorry, I just saw the first line of his solution.
einpoklum

0

Easiest way to solve the problem is to create array in which we will count the apperances for each number in the original array, and then traverse all number from $1$$1$ to $n-5$$n-5$ and check if the number appears more than once, the complexity for this solution in both memory and time is linear, or $O\left(N\right)$$O(N)$

1
This is the same @fade2black's answer (although a bit easier on the eyes)
LangeHaare

0

Map an array to 1 << A[i] and then XOR everything together. Your duplicates will be the numbers where corresponding bit is off.

There are five duplicates, so the xor trick will not break in some cases.
Evil

1
The running time of this is $O\left({n}^{2}\right)$$O(n^2)$. Each bitvector is $n$$n$ bits long, so you each bitvector operation takes $O\left(n\right)$$O(n)$ time, and you do one bit vector operation per element of the original array, for a total of $O\left({n}^{2}\right)$$O(n^2)$ time.
D.W.

@D.W. But given that the machines we normally use are fixed at either 32 or 64-bits, and these don't change at run-time (i.e. they're constant), why shouldn't they be treated as such and assume that the bit operations are in $O\left(1\right)$$O(1)$ instead of $O\left(n\right)$$O(n)$?
code_dredd

1
@ray, I think you answered your own question. Given that the machines we normally use are fixed at 64-bits, the running time to do an operation on a $n$$n$-bit vector is $O\left(n\right)$$O(n)$, not $O\left(1\right)$$O(1)$. It takes something like $n/64$$n/64$ instructions to do some operation on all $n$$n$ bits of a $n$$n$-bit vector, and $n/64$$n/64$ is $O\left(n\right)$$O(n)$, not $O\left(1\right)$$O(1)$.
D.W.

@D.W. What I got out of prev. comments was that a bit vector referred to a single element in an $n$$n$-sized array, with the bit vector being 64-bits, which would be the constant I'm referring to. Obviously, processing an an array of size $n$$n$ will take $O\left(kn\right)$$O(kn)$ time, if we assume there're $k$$k$-bits per element and $n$$n$ the number of elements in the array. But $k=64$$k=64$, so an operation for an array element w/ a constant bit count should be $O\left(1\right)$$O(1)$ instead of $O\left(k\right)$$O(k)$ and the array $O\left(n\right)$$O(n)$ instead of $O\left(kn\right)$$O(kn)$. Are you keeping the $k$$k$ for the sake of completeness/correctness or am I missing something else?
code_dredd

-2
DATA=[1,2,2,2,2,2]

from collections import defaultdict

collated=defaultdict(list):
for item in DATA:
collated[item].append(item)
if len(collated) == 5:
return item.

# n time


4
Welcome to the site. We're a computer science site, so we're looking for algorithms and explanations, not code dumps that require understanding of a particular language and its libraries. In particular, your claim that this code runs in linear time assumes that collated[item].append(item) runs in constant time. Is that really true?
David Richerby

3
Also, you are looking for a value which is repeated five times. In contrast, the OP is looking for five values, which are each repeated twice.
Yuval Filmus
Utilizzando il nostro sito, riconosci di aver letto e compreso le nostre Informativa sui cookie e Informativa sulla privacy.