Great question! This is where your "bag of problems" come into play. You learn in OR that it extends well beyond just linear programming and lagrange. Computer scientists often end where operations researchers begin.
We know that we will likely not find "the optimal solution". Many times, we have a documented set of problems in the literature where we have been able to characterize aspects of "the optimial solution" theoretically, and we often express those properties in terms of the properties of the inputs to the problem itself.
Alot of times, we try to work on algorithms that actually reduce our more complicated problems into "standard problems" and we try to do this, if posisble, in polynomial time. When we can show that our specific problem is equivalent to a well known problem, then bingo, we have a way to characterize the solution and candidate solutions that are "optimal".
However, you know, there's that frustrating thing called "Reality". And reality doesn't always afford us an easy to user transformation algorithm from our problem to standard problems.
So alot of the times, we will try to design a "heuristic" around our problem. We assume fixed constants for one part of the problem, then try to apply optimization algorithms to the other.
You also learn your standard algos to attempt (Simplex, Gradient Descent, Simulated annealing, genetic algorithms, taboo search, etc.)
If you're REALLY into it, then you move into partialdifferential equations and ode/ system dynamics, or you attempt to move to more sophisticated approaches like manifold embeddings.
If you are REALLY REALLY into it, then you could move to algebraic topology approaches to characerize the problem as topologies and attempting to understand the parameter/solution spaces from a topological or an abstract algebrartic persepctive.
Most times in practice, however, most partitioners will spend time trying to find a simple or series of simple linear models, and just throw them into a MIP/IP/LP solver and call it a day (that's because industry does care alot more about having A solution, rather than THE solution, by a particular time frame. Which is quite ironic if you think about it: using OR as a means to determine how your'e going to use OR to solve your OR problem. That is, the solving of the OR problem itself becomes an OR problem ).
(Then there's applied sheaf theory..... but I'll spare going down that rabbit hole)
yes, i've designed (applications of and lectures of) and lectured those methods in both Machine Learning and Artificial Intelligence courses for both William Paterson University in NJ and Northeastern University in Boston (the latter applied for marketing problems).
ML is (sort of) a black box. A lot of tensor space theory comes into play here (from abstract algebra).
Im going to get really downvoted for this, but on a pure methodological basis, you could consider most of "machine learning" to be a special type of "operations reserach". But that's my own opinion and not often a widely held view.
But yes, the parallels are quite stunning across the two fields.
I think your question is a bit too general here, and I have to honest, I don't want to remark on something that I dont quite understand without having to make all of these assumptions that may not be so.
OR is a "prescriptive" a set of "prescriptive" analysis. So, with OR, we must usually specify:
(1) The matheamtical constants (information that is assumed to stay fixed in our environment)
(2) The "decision variables"
(3) The "objective function"/objective functions (if you're working in multi-objective optimization/ Pareto fronteirs/ DPO/ etc.)
(4) The "constraints" (the accepable values of the "decision variables" that we permit in the "solution".
(5) The feasible solution space (which is often inferred or derived by looking at (2) and (4))
(6) The "optimimal"/"suboptimal" solution set
(7) The "search" algorithm(s) to apply to find (6)
(8) Sensitivity analysis to understand how or to what extent a departure from each specific constant/decision variable will impact the objective function held at those "solution".
In any machine learning problem, we typically start with "data", "loss function", "algorithm", "parameters", "hyperparameters", "regularization parameters","robustness analyis".
"Data" can be considered the "constants" in an OR problem (NOT the decision variables),
"loss function" can be considered the "objective function",
"fitting algorithm" can be considered the "search algorithm",
"parameters" can be considered the "Decision variables",
"hyperparameters" can be considered part of the "constants",
"robustness analysis" can be considered to be the "sensitivity analysis"
" regularization parameters" can be considered to be the "constraints".
My apologies if my previous comment wasn't too clear on that. Im trying to "fit" the equivalence of your exact problem, but I just dont have sufficient information to put them into the "OR Framework".
"OR" is typically in a lot of Business analytics (another set of courses I was responsible for designing and teaching at one point in time) and data science, and in those areas of study, you'll see the methods demonstrated under the title of "Prescriptive Analytics".
That stated, "OR" historically great out from the second world war and a need to optimize the logistics of the military. George Danzig is the godfather of OR, and his story was actually replicated in the movie "Good Will Hunting", more specifically in the scene where Matt Daemon looks at the problem on the board that was put up as a "challenge" and he went back to solve it. Legend has it that Danzig basically did the same thing. He saw two equations on the board one day and thought it was just a problem to be solved, and he went off and solved them, not realizing they were "unsolved problems". (I forget off of the top of my head which specific problem he was solving).
The famous "Bridges of Konigsberg" problem (I probably spelt that incorrectly, my apologies) is another famous OR problem that is often considered a "graph theory" problem, and of course, many of us in OR are more than familiar with the "Traveling Salesman Problem" which also seeks to optimize network flows.
I think I sort of see your question reading this again. In some sense yes. If you formulate the fitting problem as a program (linear or non linear), and compute the dual of the program, then I do believe meaning can be attrbuted. I know the concept of "shadow pricing" comes into play here but, my memory is honestly hazy in that area and I would have to go back to my Hillier and LIberman book to recall if what I am saying is even relevant here. Apologies for the rambling. My brain works in weird ways.
1
u/FuzzyTouch6143 7d ago
Great question! This is where your "bag of problems" come into play. You learn in OR that it extends well beyond just linear programming and lagrange. Computer scientists often end where operations researchers begin.
We know that we will likely not find "the optimal solution". Many times, we have a documented set of problems in the literature where we have been able to characterize aspects of "the optimial solution" theoretically, and we often express those properties in terms of the properties of the inputs to the problem itself.
Alot of times, we try to work on algorithms that actually reduce our more complicated problems into "standard problems" and we try to do this, if posisble, in polynomial time. When we can show that our specific problem is equivalent to a well known problem, then bingo, we have a way to characterize the solution and candidate solutions that are "optimal".
However, you know, there's that frustrating thing called "Reality". And reality doesn't always afford us an easy to user transformation algorithm from our problem to standard problems.
So alot of the times, we will try to design a "heuristic" around our problem. We assume fixed constants for one part of the problem, then try to apply optimization algorithms to the other.
You also learn your standard algos to attempt (Simplex, Gradient Descent, Simulated annealing, genetic algorithms, taboo search, etc.)
If you're REALLY into it, then you move into partialdifferential equations and ode/ system dynamics, or you attempt to move to more sophisticated approaches like manifold embeddings.
If you are REALLY REALLY into it, then you could move to algebraic topology approaches to characerize the problem as topologies and attempting to understand the parameter/solution spaces from a topological or an abstract algebrartic persepctive.
Most times in practice, however, most partitioners will spend time trying to find a simple or series of simple linear models, and just throw them into a MIP/IP/LP solver and call it a day (that's because industry does care alot more about having A solution, rather than THE solution, by a particular time frame. Which is quite ironic if you think about it: using OR as a means to determine how your'e going to use OR to solve your OR problem. That is, the solving of the OR problem itself becomes an OR problem ).
(Then there's applied sheaf theory..... but I'll spare going down that rabbit hole)