Transactions on Data Analysis in Social Science

Transactions on Data Analysis in Social Science

Classical Versus Metaheuristic Optimization in Management, Engineering, and Data Science: A Comparative Computational Study

Document Type : Original Article

Author
Department of Industrial Engineering, Faculty of Engineering, Fudan University, Shanghai, China
10.22034/tdas.2025.245587
Abstract
Mathematical optimization underpins decision-making in management, engineering and data science, yet there is little clear guidance on when classical methods are sufficient and when metaheuristics are warranted as problem size and complexity increase. This paper reviews classical approaches (the simplex method, gradient-based methods and branch-and-bound) and metaheuristics (genetic algorithms [GA], particle swarm optimization [PSO] and ant colony optimization [ACO]), and compares them computationally on benchmark problems in portfolio optimization, vehicle routing and feature selection in terms of solution quality and computation time over repeated runs. In portfolio optimization, GA and PSO achieved higher mean Sharpe ratios (0.92 and 0.91) than the simplex method (0.85) and gradient descent (0.82), but required longer run times. In a 50-customer vehicle routing problem, ACO and GA found shorter total distances (350 and 355 km) than time-limited branch-and-bound (360 km) in less computation time. In feature selection, GA and PSO achieved higher classification accuracy (0.88 and 0.87) with fewer features than a filter-based baseline (0.81). The results suggest that metaheuristics are advantageous for the nonlinear and combinatorial problems tested, whereas classical methods were faster in the portfolio setting. The paper concludes with practical guidance on algorithm selection and outlines future directions, including hybrid metaheuristics, optimization under uncertainty and integration with machine learning.
Keywords

[1]     Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press. https://doi.org/10.1017/CBO9780511804441
[2]     Nocedal, J., & Wright, S. J. (2006). Numerical optimization (2nd ed.). Springer. https://doi.org/10.1007/978-0-387-40065-5
[3]     Bottou, L., Curtis, F. E., & Nocedal, J. (2018). Optimization methods for large-scale machine learning. SIAM Review, 60(2), 223–311. https://doi.org/10.1137/16M1080173
[4]     Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations (ICLR 2015).
[5]     Boussaïd, I., Lepagnot, J., & Siarry, P. (2013). A survey on optimization metaheuristics. Information Sciences, 237, 82–117. https://doi.org/10.1016/j.ins.2013.02.041
[6]     Bixby, R. E. (2012). A brief history of linear and mixed-integer programming computation. Documenta Mathematica, Extra Volume ISMP, 107–121. https://doi.org/10.4171/DMJ/6/16
[7]     Miettinen, K. (1999). Nonlinear multiobjective optimization. Kluwer Academic Publishers.
[8]     Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical earning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122. https://doi.org/10.1561/2200000016
[9]     Ben-Tal, A., El Ghaoui, L., & Nemirovski, A. (2009). Robust optimization. Princeton University Press. https://doi.org/10.1515/9781400831050
[10]   Dantzig, G. B. (1963). Linear programming and extensions. Princeton University Press. https://doi.org/10.1515/9781400884179
[11]   Klee, V., & Minty, G. J. (1972). How good is the simplex algorithm? In O. Shisha (Ed.), Inequalities III (pp. 159–175). Academic Press.
[12]   Karmarkar, N. (1984). A new polynomial-time algorithm for linear programming. Combinatorica, 4(4), 373–395. https://doi.org/10.1007/BF02579150
[13]   Fletcher, R. (1987). Practical methods of optimization (2nd ed.). Wiley.
[14]   Robbins, H., & Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22(3), 400–407. https://doi.org/10.1214/aoms/1177729586
[15]   Land, A. H., & Doig, A. G. (1960). An automatic method of solving discrete programming problems. Econometrica, 28(3), 497–520. https://doi.org/10.2307/1910129
[16]   Gomory, R. E. (1958). Outline of an algorithm for integer solutions to linear programs. Bulletin of the American Mathematical Society, 64(5), 275–278. https://doi.org/10.1090/S0002-9904-1958-10224-4
[17]   Wolsey, L. A. (1998). Integer programming. Wiley.
[18]   Holland, J. H. (1975). Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press.
[19]   Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680. https://doi.org/10.1126/science.220.4598.671
[20]   Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of ICNN’95—International Conference on Neural Networks (Vol. 4, pp. 1942–1948). IEEE. https://doi.org/10.1109/ICNN.1995.488968
[21]   Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 26(1), 29–41. https://doi.org/10.1109/3477.484436
[22]   Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), 53–66. https://doi.org/10.1109/4235.585892
[23]   Blum, C., & Roli, A. (2003). Metaheuristics in combinatorial optimization: Overview and conceptual comparison. ACM Computing Surveys, 35(3), 268–308. https://doi.org/10.1145/937503.937505
[24]   Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82. https://doi.org/10.1109/4235.585893
[25]   Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2), 182–197. https://doi.org/10.1109/4235.996017
[26]   Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
[27]   Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1706.10059
[28]   Hillier, F. S., & Lieberman, G. J. (2021). Introduction to operations research (11th ed.). McGraw Hill.
[29]   Simchi-Levi, D., Kaminsky, P., & Simchi-Levi, E. (2008). Designing and managing the supply chain: Concepts, strategies, and case studies (3rd ed.). McGraw-Hill/Irwin.
[30]   Markowitz, H. (1952). Portfolio selection. The Journal of Finance, 7(1), 77–91. https://doi.org/10.2307/2975974
[31]   Toth, P., & Vigo, D. (Eds.). (2014). Vehicle routing: Problems, methods, and applications (2nd ed.). Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9781611973594
[32]   Arora, J. S. (2017). Introduction to optimum design (4th ed.). Academic Press.
[33]   Sra, S., Nowozin, S., & Wright, S. J. (Eds.). (2011). Optimization for machine learning. MIT Press. https://doi.org/10.7551/mitpress/8996.001.0001
[34]   Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182.
[35]   Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281–305.
[36]   Bengio, Y., Lodi, A., & Prouvost, A. (2021). Machine learning for combinatorial optimization: A methodological tour d’horizon. European Journal of Operational Research, 290(2), 405–421. https://doi.org/10.1016/j.ejor.2020.07.063
[37]   Shi, Y., & Eberhart, R. (1998). A modified particle swarm optimizer. In 1998 IEEE International Conference on Evolutionary Computation Proceedings (pp. 69–73). IEEE. https://doi.org/10.1109/ICEC.1998.699146
[38]   Sharpe, W. F. (1966). Mutual fund performance. The Journal of Business, 39(1), 119–138. https://doi.org/10.1086/294846
[39]   Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. J., Jones, E., Kern, R., Larson, E., ... SciPy 1.0 Contributors. (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17, 261–272. https://doi.org/10.1038/s41592-019-0686-2
[40]   Fortin, F.-A., De Rainville, F.-M., Gardner, M.-A., Parizeau, M., & Gagné, C. (2012). DEAP: Evolutionary algorithms made easy. Journal of Machine Learning Research, 13, 2171–2175.
[41]   Miranda, L. J. V. (2018). PySwarms: A research toolkit for particle swarm optimization in Python. Journal of Open Source Software, 3(21), 433. https://doi.org/10.21105/joss.00433
[42]   Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
[43]   Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.
[44]   Derrac, J., García, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3–18. https://doi.org/10.1016/j.swevo.2011.02.002
[45]   Talbi, E.-G. (2009). Metaheuristics: From design to implementation. Wiley. https://doi.org/10.1002/9780470496916
[46]   Esmaeili Moshiran, A., & Babaei, S. (2020). Load balancing of servers in cloud computing using the lion optimization algorithm. Transactions on Machine Intelligence, 3(4), 240–250. https://doi.org/10.47176/TMI.2020.240
[47]   Liu, H., & Chen, M. (2025). Optimized coordination of overcurrent relays in microgrids using the Grey Wolf Optimization algorithm. Transactions on Machine Intelligence, 8(4), 233–247. https://doi.org/10.22034/tmi.2025.253006
Volume 7, Issue 2
Spring 2025
Pages 140-153

  • Receive Date 20 February 2025
  • Revise Date 03 May 2025
  • Accept Date 02 June 2025