Skip to content

algorithm.md

Latest commit

 

History

History
105 lines (53 loc) · 6.96 KB

algorithm.md

File metadata and controls

105 lines (53 loc) · 6.96 KB
Raw

Algorithms

[TOC]

An algorithm is a finite sequence of precise instructions for performing a computation or for solving a problem.

Properties

The properties of the algorithm are:

  • Input. An algorithm has input values from a specified set.
  • Output. From each set of input values, an algorithm produces output values from a specified set. The output values are the solution to the problem.
  • Definiteness. The steps of an algorithm must be defined precisely.
  • Correctness. An algorithm should produce the correct output values for each set of input values.
  • Finiteness. An algorithm should produce the desired output after a finite (but perhaps large) number of steps for any input in the set.
  • Effectiveness. It must be possible to perform each step of an algorithm exactly and in a finite amount of time.
  • Generality. The procedure should be applicable for all problems of the desired form, not just for a particular set of input values.

Searching Algorithm

THE LINEAR SEARCH. The linear search algorithm begins by comparing $x$ and $a_1$. When $x = a_1$, the solution is the location of $a_1$, namely, $1$. When $x \neq a_1$, compare $x$ with $a_2$. If $x = a_2$, the solution is the location of $a_2$, namely, $2$. When $x \neq a_2$, compare $x$ with $a_3$. Continue this process, comparing $x$ successively with each term of the list until a match is found, where the solution is the location of that term, unless no match occurs.

THE BINARY SEARCH. The binary search algorithm proceeds by comparing the element to be located to the middle term of the list. The list is then split into two smaller sublists of the same size, or where one of these smaller lists has one fewer term than the other. The search continues by restricting the search to the appropriate sublist based on the comparison of the element to be located and the middle term.

Sorting Algorithm

THE BUBBLE SORT. The bubble sort puts a list into increasing order by successively comparing adjacent elements, interchanging them if they are in the wrong order.

THE INSERTION SORT. In the $j$th step of the insertion sort, the $j$th element of the list is inserted into the correct position in the list of the previously sorted $j - 1$ elements. To insert the $j$the element in the list, a linear search technique is used; the $j$th element is successively compared with the already sorted $j - 1$ elements at the start of the list until the first element that is not less than this element is found, or until it has been compared with all $j - 1$ elements; the $j$th element is inserted in the correct position so that the first $j$ elements are sorted. The algorithm continues until the last element is placed in the correct position relative to the already sorted list of the first $n - 1$ elements.

Greedy Algorithms

Algorithms that make what seems to be the "best" choice at each step are called greedy algorithms.

Cashier's Algorithms

THEOREM: The cashier's algorithm always makes changes using the fewest coins possible when change is made from quarters, dimes, nickels, and pennies.

LEMMA: If $n$ is a positive integer, then $n$ cents in change using quarters, dimes, nickels, and pennies using the fewest coins possible has at most two dimes, at most one nickel, and at most four pennies, and cannot have two dimes and a nickel. The amount of change in dimes, nickels, and pennies cannot exceed 24 cents.

Halting Problem

Halting problem. It asks whether there is a procedure that does this: It takes as input a computer program and input to the program, and determines whether the program will eventually stop when run with this input.

The Growth of Functions

Definition: Let $f$ and $g$ be functions from the set of integers or the set of real numbers to the set of real numbers. We say that $f(x)$ is $O(g(x))$ if there are constants $C$ and $k$ such that $|f(x)| \leq C|g(x)|$ whenever $x > k$. [This is read as "$f(x)$ is big-oh of $g(x)$."]

THEOREM: Let $f(x) = a_n x^n + a_{n-1}x^{n-1} + \cdots + a_1 x + a_0$, where $a_0, a_1, ..., a_{n - 1}, a_n$ are real numbers. Then $f(x)$ is $O(x^n)$.

HEOREM**: Suppose that $f_1(x)$ is $O(g_1(x))$ and that $f_2(x)$ is $O(g_2(x))$. Then $(f_1 + f_2)(x)$ is $O(g(x))$, where $g(x) = max(|g_1(x)|, |g_2(x)|)$ for all $x$.

COROLLARY: Suppose that $f_1(x)$ and $f_2(x)$ are both $O(g(x))$. Then $(f_1 + f_2)(x)$ is $O(g(x))$.

THEOREM Suppose that $f_1(x)$ is $O(g_1(x))$ and $f_2(x)$ is $O(g_2(x))$. Then $(f_1 f_2)(x)$ is $O(g_1(x) g_2(x))$.

Definition Let $f$ and $g$ be functions from the set of integers or the set of real numbers to the set of real numbers. We say that $f(x)$ is $\Omega(g(x))$ If there are constants $C$ and $k$ with $C$ positive such that $|f(x)| \geq C|g(x)|$ whenever $x > k$. [This is read as "$f(x)$ is big-Omega of $g(x)$."]

Definition Let $f$ and $g$ be functions from the set of integers or the set of real numbers to the set of real numbers. We say that $f(x)$ is $\Theta(g(x))$ if $f(x)$ is $O(g(x))$ and $f(x)$ is $\Omega(g(x))$. When $f(x)$ is $\Omega(g(x))$, We say that $f$ is big-Theta of $g(x)$, that $f(x)$ is of order $g(x)$, and that $f(x)$ and $g(x)$ are of the same order.

THEOREM Let $f(x) = a_n x^n + a_{n-1}x^{n-1} + ... + a_1 x + a_0$, where $a_0, a_1, ..., a_n$ are real numbers with $a_n \neq 0$. Then $f(x)$ is of order $x^n$.

Complexity of Algorithms

An analysis of the time required to solve a problem of a particular size involves the time complexity of the algorithm.

An analysis of the computer memory required involves the space complexity of the algorithm.

WORST-CASE COMPLEXITY By the worst-case performance of an algorithm, we mean the largest number of operations needed to solve the given problem using this algorithm on an input of specified size.

AVERAGE-CASE COMPLEXITY The average number of operations used to solve the problem over all possible inputs of a given size is found in this type of analysis.

Commonly Used Terminology for the Complexity of Algorithms:

growth_of_function

Complexity Terminology
$\Theta(1)$ Constant complexity
$\Theta(log\ n)$ Logarithmic complexity
$\Theta(n)$ Linear complexity
$\Theta(n\ log\ n)$ Linearithmic complexity
$\Theta(n^b)$ Polynomial complexity
$\Theta(b^n),\text{ where b > 1}$ Exponential complexity
$\Theta(n!)$ Factorial complexity

TRACTABILITY A problem that is solvable using an algorithm with polynomial (or better) worst-cast complexity is called tractable, because the expectation is that the algorithm will produce the solution to the problem for reasonably sized input in a relatively short time.

References

[1] Kenneth H. Rosen. Discrete Mathematics and Its Applications. 8Edition