Introduction:

NOTE: read the whole assignment brief first before implementing it contains very important information
In this assignment, you will be tasked with using four numerical optimization methods. Gradient Descent with Momentum, Newton’s Method, Simulated Annealing, and Genetic Algorithms. You will be given two problems.

For problem 1 you should use Gradient Descent with Momentum and Newton’s method (both by hand and by code). The second problem should have Simulated Annealing and Genetic Algorithms applied to it by code only. As these were the main methods studied during the labs this is what you must show sufficient implementation and understanding of them in order to pass.

For the first problem, you will be required to find the first and second derivatives in order to apply both methods. You will also be required to provide a python implementation of both methods applied to that problem.
For the second problem you will be required to provide python files that implement the Knapsack problem: one for Simulated Annealing, and the other for Genetic Algorithms