PART SIX of long document
Note—The differences we observe between experiment, perhaps what we think of as reality, and our simulations of reality hindered by finite computer memory, hence limited numerical discretization of space and time, limited processor speed, various kinds of limited I/O speeds and bandwidths, together with our choices in numerical algorithms, their various convergence properties and instabilities, and what physics we exclude from our numerical models, e.g., an inviscid hydrocode, among other considerations, makes me believe that this approach to creating a virtual world in the sense of The Matrix move would quickly lead intelligent beings within the simulations to conclude that the nature of their universe is inconsistent with the basic laws of physics they experience even at the classical level. For a different reason, this, if we can believe in what current physics tells us about the deep future, will also be true for intelligent beings born eons from now, for there will come a time when galaxies will become isolated islands, and though the inhabitants of these islands will be able to discover and develop physics pretty much as we know it, they will not be able to reconcile it with a sensible cosmology consistent with the abundance of elements in their galaxy the way we can reconcile the abundance of elements we observe with a Big Bang cosmology. Has this already happened to us? Has it happened to our antecedents? Where are the SUSY sparticles? Or the next missing particles?
Introductory Linear Algebra concurrent with Ordinary Differential Equations:
The laws of nature, and many human made models, such as economic models, can be expressed as differential equations. However, except for a small number of exceptions, most differential equations cannot be solved analytically, instead being subjected to approximate numerical methods. That is, differential equations are converted into difference equations to be solved by approximate numerical methods. These numerical methods invariably involve solving systems of linear equations, best expressed in matrix form. An introductory level linear algebra course will give the student a basic understanding of what numerical methods do to solve systems of linear equations.
Note—By ordinary differential equation, it is meant differential equations involving only one independent variable, and one or more their derivatives. Ordinary differential equations involve equations with derivatives in the sense of calculus I. More generally, systems of coupled ordinary differential equations may be solved, sometimes exactly, using methods from linear algebra as well; also see eigenvalues and eigenvectors and the method of eigenfunction expansion. Partial differential equations are a type of differential equation involving an unknown function (or functions) of several independent variables and their partial derivatives with respect to those variables. Partial differential equations involved derivatives in the sense of calculus III, often involving differential vector operators used by physicists and engineers.
Some future motivation for at least introductory level mastery of linear algebra. Electricity and magnetism at the junior level and above is expressed in the language of partial differential equations, as are the subjects of heat transfer, diffusion theory, and many other fields including mathematical finance. We have already stated that numerical methods involving partial differential equations ultimately involve solving systems of linear equations. Some problems in classical mechanics are problems involving metric preserving orthogonal transformations. That is, in classical mechanics the length of an object (a metrical concept) is independent of the reference frame (coordinate system) we choose. Linear algebra provides the matrix algebra of such transformations. In non-relativistic quantum mechanics, unitary transformations connect a wave function solving Schrodinger’s wave equation between two points in time. In special relativity, it is a four-vector length that is preserved as expressed by Lorentz transformations. In Dirac’s equation, a (special) relativistic equation for the electron, we deal with matrix transformations which preserve probability currents. The bottom line: having some knowledge of elementary linear algebra is essential to an eventual understanding of many physical systems and concepts independent of, but in conjunction with corresponding numerical methods.
Advanced undergraduate course in linear algebra (optional, but highly recommended): The need to solve systems of linear equations derives from many sources. The approximate numerical solution of partial and ordinary differential equations has already been mentioned. Multiple linear least square regression to fit a line through data requires the solution of systems of linear equations. Problems in linear optimization, e.g., the simplex method involve solving systems of linear inequalities. Advanced courses in classical, continuum, quantum, special and general relativistic mechanics are bested treated in a coordinate free, linear algebraic language proceeding to tensor algebra and calculus. This list is far from complete, but the reader should get the idea as to the deep importance of linear algebra in both applied and pure mathematics.
Ordinary Differential Equations:
When a student is first introduced to partial differential equations, it is shown how some very important partial differential equations can be reduced (by separation of variables) into a system of ordinary differential equations. Hence the reason a first course in differential equations is a course in ordinary differential equations. Advanced courses in mathematical physics at the advanced undergraduate or beginning graduate level introduce the student to other methods of treating partial differential equations such as integral transform methods, Green’s functions and the use of the Dirac delta function, which opens a path to the theory of distributions and probability that a graduate student or post doctoral fellow in physics or engineering may wish to pursue on their own in greater depth.
Complex Variables concurrent with Partial Differential Equations:
When recently an undergraduate engineering student asked me if complex numbers have meaning, I took his question to mean if complex numbers have a correspondence to physical ‘reality’. I explained to him that complex numbers are useful in describing the motion of a damped harmonic oscillator, modeled by an ordinary differential equation. I then thought of the importance of the phases of quantum mechanical wave functions in the Bohm-Ahronov experiment, but I kept quiet for the moment. This free flow of ideas and questions further led me to think about Hamilton’s quarternionic extension of the complex number system and the connection of this number system to spinors and Dirac’s relativistic quantum mechanical partial differential equation for the electron and positron. The Lie algebra of the Lorentz group then came to mind, an algebra shared with the direct product group SU(2) × SU(2), which is compact while the Lorentz group is not. I was struck by the magnitude of the difference in experience between my current self and the young man I had once been three decades ago pondering the same type of questions as the young man standing before me. I had often wished for a mentor.
More mundanely, in certain idealized problems of aerodynamics and hydrodynamics, methods from complex variables lead to solutions to flow line fields. Particularly important is the generalization of integration as understood from a calculus II level. Integration in complex variables is deeply connected with infinite series expansions and much more. Incidentally, we can construct ‘larger’ sets of numbers from ‘smaller’ sets of numbers: from natural numbers we can construct the integers, from the integers the rationals, from the rationals the real numbers, from the real numbers the complex numbers, from the complex numbers the quaternions, from the quaternions the Cayley numbers, and so on, but we progressively abandon more structure so that we get progressively less axiomatic richness. Does this limit how wild physics theories can be? Probably not. One can create illimitably rich number systems by defining direct products.
A student running through a calculus sequence is not likely to escape some homework involving numerical methods. Computational fluid dynamics (CFD), hydrodynamics, radiation hydrodynamics and many other computationally intense fields including statistics drive students deep into numerical methods. A great reference, heavy on linear algebra, is “Numerical Recipes” in Fortran 77, or C, or C++, etc. One need not take a course in numerical methods. They can be mastered with self study and experimentation with computers. Having a good, general understanding in numerical methods, however is essential to understanding how we model physical systems with computers. Creating approximate numerical models of physical systems aids in our understanding of ‘real’ systems. One of the best books I’ve ever run into is the book on numerical methods by Richtmyer and Morton, “Difference Methods for Initial Value Problems.” Richtmyer worked at Los Alamos during WW II, and became the leader of its theoretical division after the war. This old book of limited availability covers many areas from shock physics to diffusion, heat flow, particle transport, and fluid (hydrodynamics) in one space variable including the method of artificial viscosity which was pioneered in “A method for the Numerical Calculation of Hydrodynamic Shocks” by J. Von Neumann and R. D. Richtmyer, Journal of Applied Physics, Vol. 21, March, 1950. Also useful in this vein of reference books I’ve used is, “Numerical Methods in Applied Physics and Astrophysics” by R. L. Bowers, and J. R. Wilson, Jones and Bartlett, 1991. I recommended “Computational Plasma Physics, with Applications to Fusion and Astrophysics” by Toshiki Tajima, Kansai Research Establishment, Japan Atomic Energy Research Institute, Kyoto, Westview Press, 2004. I enjoyed working through “Nuclear Reactor Analysis” by James J. Duderstadt, and Louis J. Hamilton, Wiley Interscience, 1976. I also like, “Computational Methods of Neutron Transport” by E. E. Lewis, and W. F. Miller, Jr., American Nuclear Society, 1993. A friend of mine from Los Alamos recommended to me the three volume series “Computational Fluid Dynamics” 4th ed. Vols. I, II, and III by K. A. Hoffmann and S. T. Chiang; they are very good books. Working these three books out will give you a strong foundation in the numerical treatment of partial differential equations in Computational Fluid Dynamics and Computation Fluid Turbulence.
Underlying the applied numerical methods above are the following texts. “Explosives Engineering” by Paul. W. Cooper, Wiley-VCH, 1996, and “Explosive Effects and Applications” edited by J. A. Zukas and W. P. Walters. An essential text and reference book is “Physics of Shock Waves and High-Temperature Hydrodynamic Phenomena” by Ya. B. Zeldovich, and Yu. P. Razier, edited by W. D. Hayes, and R. F. Probstein, Dover Publications Inc., first translated from Russian and published in English in Volume I in 1966 and in Volume II in 1967. You will deal with much applied thermodynamics in Zeldovich and Razier. “Shock Compression of Condensed Materials” by R. F. Trunin is another book translated from the All Russian Research Institute of Experimental Physics, Sarov, which was translated into English by Cambridge University Press in 1998. The book covers condensed matter physics phenomenology and data tables of materials experiencing up to 10 TPa in large scale underground nuclear tests. Lastly, I recommend “Foundations of Radiation Hydrodynamics” by Dimitri Mihalas and Barbara Weibel-Mihalas Dover Publications Inc., 1984. You would be well on your way to readings in cosmology and astrophysics with the book by S. Eliezer, A. G. Ghatak and H. Hora cited above treating equations of state.
An old but great reference book is “Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables” edited by Milton Abramowitz and Irene A. Stegun, Dover Publications Inc., 9th printing, November 1970. Another essential handbook is “Tables of Integrals, Series, and Products” 2nd ed. by I. S. Gratsheyn and I. M. Ryzhik, American Press Inc, 1980. Lastly, there is “Handbook of Differential Equations” 3rd ed. by Zwillinger, Academic Press, 1997. I’m happy with the 2nd edition.
Probability and Statistics for Physicists, Mathematicians, and Engineers:
Here is a list of topics/ideas that I see as critical: The Central Limit Theorem for the sample mean. Propagation of errors. Least Square Estimation. Maximum Likelihood Estimation (for statistical physics through logistic regression). Singular Value Decomposition and other Principle Component (PC) methods with applications ranging from Molecular Dynamics to pricing a portfolio of stocks and options, or pricing long term weather derivatives for North America based on El Niņo/La Niņa climatology. A doctoral candidate in physics will have had to have passed at least a one semester course in statistical physics. Problems in econophysics require a deeper foundation in graduate level (measure theoretic) analysis and probability theory, together with courses in stochastic processes modeled by stochastic differential equations. Note—Monte Carlo methods (numerical methods first developed at LANL during WWII to solve neutron transport problems in early nuclear weapons) are good enough, but a good scientist should always pine for a deeper understanding.
Outside of the Calculus of Variations, the areas of linear and nonlinear programming, linear and nonlinear optimization are good things to be familiar with. I like “Optimization Theory with Applications” by D. A. Pierre, Dover Publications Inc., 1969, 1986.
Design of Experiments:
One of the best, most useful applications of Probability and Statistics, with Optimization Theory is the field of Designed Experiments, or Design of Experiments (DOE). I like “Design and Analysis of Experiments” 5th ed. or above by Douglas C. Montgomery, Wiley, 1997.