We introduce a new Swarm-Based Descent (SBD) method for non-convex optimization. The swarm consists of agents, each identified with position, x, and mass, m.
There are three key aspects to the SBD dynamics: (i) persistent transition of mass from high to lower ground; (ii) a random choice of marching direction, aligned with the orientation of gradient descent; and (iii) a time stepping protocol, h(x,m), which decreases with m.
The interplay between positions and masses leads to dynamic distinction between `leaders’ and `explorers’. Heavier agents lead the swarm near local minima with small time steps, while lighter agents explore the landscape with larger time steps in search of improved positions, i.e., reduce the ‘loss’ for the swarm. Convergence analysis and numerical simulations demonstrate the effectiveness of SBD method as a global optimizer.