Studying Self-Studying Scientific Computing for future graduate study

[geologist]

I like to start a graduate program in scientific computing within the next 4-5 years, once my kids are a little older, my wife finishes her degree, and I have more money saved. In the meantime, I’m not going to waste time. Using the MIT challenge completed by Scott Young as inspiration, I’ve made the decision to complete my own challenge, spread over two years (due to kids and work), with a focus in scientific computing. Specifically, I’d like to complete the approximate equivalent of a BS in scientific computing, with a focus on environmental science/geoscience applications. I’ll aim to study 10-20 hours per week.I’ve completed an MS in geology (circa 2010) and work in environmental consulting, so some of what I learn will be applicable to my profession. My main motivation for completing this is because I find the subject fascinating and it gives me something to focus on (as opposed to dabbling). I also understand that this doesn’t replace official education, but with family and work obligations, going back to school isn’t practical, and I don’t have the patience to wait until I’m 37 to study this topic.I’ve created a general outline of the types of courses/topics I would need to take. For those who have gone through a similar program, is there anything significant I’ve left out?

· Calculus I-III (currently using Elementary calculus by Keisler with supplemental videos on MIT open and Udemy)

· Differential Equations (Elementary calculus has a chapter at the end that covers this)

· Introductory Linear Algebra course/textbook

· Statistics with R course on Coursera (introductory probability, inferential statistics, linear regression, Bayesian statistics).

· Introduction to Computer science and programming using Python (on edx or MIT open)

· Computational thinking and data science (same as above)

· QGIS 3.0 for GIS professionals (Udemy)

· DataCamp GIS with R tutorials

· Calculus based Physics 1 & 2 (book/course TBD)

· Applied Groundwater Modeling (textbook)

· Global Warming Science (MIT open) or Principles of Planetary Climate (textbook)

· Modeling Environmental Complexity (MIT open) or Environmental Modeling: Finding simplicity in complexity (textbook)

· Introduction to High-performance computing (online course or book, TBD) or equivalent

· Independent project, posted to personal website, github, or similar

 

[jedishrfu]

There's a good book that can get you started in Computational Physics by Gould, Tobochnik and Christian:

https://www.amazon.com/dp/1974427471/?tag=pfamazon01-20

There's also a java-based simulation library and examples that can be downloaded at: www.compadre.org/osp

I used in a course several years ago and it was quite a lot of fun to run simulations given the equations of motion.

One in particular was the three body problem where they simulated a few of the more arcane versions like Euler, LaGrange and Montgomery. by providing the initial values for each body:

public class ThreeBodyInitialConditions {
  static final double sn = Math.sin(Math.PI/3);
  static final double half = Math.cos(Math.PI/3);
  static final double x1 = 0.97000436;
  static final double v1 = 0.93240737/2;
  static final double y1 = 0.24308753;
  static final double v2 = 0.86473146/2;
  static final double v = 0.8; // inital speed
  // state = {x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t}
  // EULER places masses on a line

  static final double[] EULER = {
    0, 0, 0, 0, 1, 0, 0, v, -1, 0, 0, -v, 0
  };
  // LAGRANGE places masses on an equilateral triangle

  static final double[] LAGRANGE = {
    1, 0, 0, v, -half, -v*sn, sn, -v*half, -half, v*sn, -sn, -v*half, 0
  };
  static final double[] MONTGOMERY = {
    x1, v1, -y1, v2, -x1, v1, y1, v2, 0, -2*v1, 0, -2*v2, 0
  };
}

The App is here:

/*
 * Open Source Physics software is free software as described near the bottom of this code file.
 *
 * For additional information and documentation on Open Source Physics please see:
 * <http://www.opensourcephysics.org/>
 */

package org.opensourcephysics.sip.ch05;
import org.opensourcephysics.controls.*;
import org.opensourcephysics.frames.*;

/**
 * ThreeBodyApp models the three body problem by extending
 * AbstractSimulation and implementing the doStep method.
 *
 * @author Wolfgang Christian, Jan Tobochnik, Harvey Gould
 * @version 1.0
 */
public class ThreeBodyApp extends AbstractSimulation {
  PlotFrame frame = new PlotFrame("x", "y", "Three-Body Orbits");
  ThreeBody trajectory = new ThreeBody();

  /**
   * Constructs ThreeBodyApp and initializes the drawing.
   */
  public ThreeBodyApp() {
    frame.addDrawable(trajectory);
    frame.setSquareAspect(true);
    frame.setSize(450, 450);
  }

  /**
   * Initializes the simulation;
   */
  public void initialize() {
    trajectory.odeSolver.setStepSize(control.getDouble("dt"));
    trajectory.initialize(ThreeBodyInitialConditions.MONTGOMERY);
    frame.setPreferredMinMax(-1.5, 1.5, -1.5, 1.5);
  }

  /**
   * Resets the animation into a predefined state.
   */
  public void reset() {
    control.setValue("dt", 0.1);
    enableStepsPerDisplay(true);
    initialize();
  }

  /**
   * Does an animation step.
   */
  protected void doStep() {
    trajectory.doStep();
    frame.setMessage("t="+decimalFormat.format(trajectory.state[4]));
  }

  /**
   * Starts the Java application.
   * @param args  command line parameters
   */
  public static void main(String[] args) {
    SimulationControl.createApp(new ThreeBodyApp());
  }
}

/*
 * Open Source Physics software is free software; you can redistribute
 * it and/or modify it under the terms of the GNU General Public License (GPL) as
 * published by the Free Software Foundation; either version 2 of the License,
 * or(at your option) any later version.

 * Code that uses any portion of the code in the org.opensourcephysics package
 * or any subpackage (subdirectory) of this package must must also be be released
 * under the GNU GPL license.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston MA 02111-1307 USA
 * or view the license online at http://www.gnu.org/copyleft/gpl.html
 *
 * Copyright (c) 2007  The Open Source Physics project
 *                     http://www.opensourcephysics.org
 */

and the model is here:

/*
 * Open Source Physics software is free software as described near the bottom of this code file.
 *
 * For additional information and documentation on Open Source Physics please see:
 * <http://www.opensourcephysics.org/>
 */

package org.opensourcephysics.sip.ch05;
import java.awt.*;
import org.opensourcephysics.display.*;
import org.opensourcephysics.numerics.*;

/**
 * ThreeBody models the gravitational three-body problem.
 *
 * @author Wolfgang Christian, Jan Tobochnik, Harvey Gould
 * @version 1.0
 */
public class ThreeBody implements Drawable, ODE {
  int n = 3; // number of interacting bodies
  // state= {x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t}

  double[] state = new double[4*n+1];
  double[] force = new double[2*n], zeros = new double[2*n];
  ODESolver odeSolver = new RK45MultiStep(this);
  Mass mass1 = new Mass(), mass2 = new Mass(), mass3 = new Mass();

  /**
   * Draws the three bodies.
   *
   * @param panel
   * @param g
   */
  public void draw(DrawingPanel panel, Graphics g) {
    mass1.draw(panel, g);
    mass2.draw(panel, g);
    mass3.draw(panel, g);
  }

  /**
   * Steps the time using an ode solver.
   */
  public void doStep() {
    odeSolver.step();
    mass1.setXY(state[0], state[2]);
    mass2.setXY(state[4], state[6]);
    mass3.setXY(state[8], state[10]);
  }

  /**
   * Initialize the three-body problem with the given state.
   * @param initial state
   */
  void initialize(double[] initState) {
    System.arraycopy(initState, 0, state, 0, 13); // copies initstate to state
    mass1.clear();                                // clears data from old trail
    mass2.clear();
    mass3.clear();
    mass1.setXY(state[0], state[2]);
    mass2.setXY(state[4], state[6]);
    mass3.setXY(state[8], state[10]);
  }

  /**
   * Computes the forces using pairwise interactions assuming equal mass.
   */
  void computeForce(double[] state) {
    System.arraycopy(zeros, 0, force, 0, force.length); // sets force array elements to 0
    for(int i = 0;i<n;i++) {
      for(int j = i+1;j<n;j++) {
        double dx = state[4*i]-state[4*j];
        double dy = state[4*i+2]-state[4*j+2];
        double r2 = dx*dx+dy*dy;
        double r3 = r2*Math.sqrt(r2);
        double fx = dx/r3;
        double fy = dy/r3;
        force[2*i] -= fx;
        force[2*i+1] -= fy;
        force[2*j] += fx;
        force[2*j+1] += fy;
      }
    }
  }

  /**
   * Gets the rate using the given state.
   *
   * @param state double[]
   * @param rate double[]
   */
  public void getRate(double[] state, double[] rate) {
    computeForce(state); // force array alternates fx and fy
    for(int i = 0;i<n;i++) {
      int i4 = 4*i;
      rate[i4] = state[i4+1];    // x rate is vx
      rate[i4+1] = force[2*i];   // vx rate is fx
      rate[i4+2] = state[i4+3];  // y rate is vy
      rate[i4+3] = force[2*i+1]; // vy rate is fy
    }
    rate[state.length-1] = 1; // time rate is last
  }

  /**
   * Gets the state: x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t.
   *
   * @return the state
   */
  public double[] getState() {
    return state;
  }

  class Mass extends Circle {
    Trail trail = new Trail();
    // Draws the mass.

    public void draw(DrawingPanel panel, Graphics g) {
      trail.draw(panel, g);
      super.draw(panel, g);
    }

    // Clears trail
    void clear() {
      trail.clear();
    }

    // Sets postion and adds to trail
    public void setXY(double x, double y) {
      super.setXY(x, y);
      trail.addPoint(x, y);
    }
  }
}

/*
 * Open Source Physics software is free software; you can redistribute
 * it and/or modify it under the terms of the GNU General Public License (GPL) as
 * published by the Free Software Foundation; either version 2 of the License,
 * or(at your option) any later version.

 * Code that uses any portion of the code in the org.opensourcephysics package
 * or any subpackage (subdirectory) of this package must must also be be released
 * under the GNU GPL license.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston MA 02111-1307 USA
 * or view the license online at http://www.gnu.org/copyleft/gpl.html
 *
 * Copyright (c) 2007  The Open Source Physics project
 *                     http://www.opensourcephysics.org
 */

 

[geologist]

There's a good book that can get you started in Computational Physics by Gould, Tobochnik and Christian:

https://www.amazon.com/dp/1974427471/?tag=pfamazon01-20

There's also a java-based simulation library and examples that can be downloaded at: www.compadre.org/osp

I used in a course several years ago and it was quite a lot of fun to run simulations given the equations of motion.

One in particular was the three body problem where they simulated a few of the more arcane versions like Euler, LaGrange and Montgomery. by providing the initial values for each body:

public class ThreeBodyInitialConditions {
  static final double sn = Math.sin(Math.PI/3);
  static final double half = Math.cos(Math.PI/3);
  static final double x1 = 0.97000436;
  static final double v1 = 0.93240737/2;
  static final double y1 = 0.24308753;
  static final double v2 = 0.86473146/2;
  static final double v = 0.8; // inital speed
  // state = {x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t}
  // EULER places masses on a line

  static final double[] EULER = {
    0, 0, 0, 0, 1, 0, 0, v, -1, 0, 0, -v, 0
  };
  // LAGRANGE places masses on an equilateral triangle

  static final double[] LAGRANGE = {
    1, 0, 0, v, -half, -v*sn, sn, -v*half, -half, v*sn, -sn, -v*half, 0
  };
  static final double[] MONTGOMERY = {
    x1, v1, -y1, v2, -x1, v1, y1, v2, 0, -2*v1, 0, -2*v2, 0
  };
}

The App is here:

/*
 * Open Source Physics software is free software as described near the bottom of this code file.
 *
 * For additional information and documentation on Open Source Physics please see:
 * <http://www.opensourcephysics.org/>
 */

package org.opensourcephysics.sip.ch05;
import org.opensourcephysics.controls.*;
import org.opensourcephysics.frames.*;

/**
 * ThreeBodyApp models the three body problem by extending
 * AbstractSimulation and implementing the doStep method.
 *
 * @author Wolfgang Christian, Jan Tobochnik, Harvey Gould
 * @version 1.0
 */
public class ThreeBodyApp extends AbstractSimulation {
  PlotFrame frame = new PlotFrame("x", "y", "Three-Body Orbits");
  ThreeBody trajectory = new ThreeBody();

  /**
   * Constructs ThreeBodyApp and initializes the drawing.
   */
  public ThreeBodyApp() {
    frame.addDrawable(trajectory);
    frame.setSquareAspect(true);
    frame.setSize(450, 450);
  }

  /**
   * Initializes the simulation;
   */
  public void initialize() {
    trajectory.odeSolver.setStepSize(control.getDouble("dt"));
    trajectory.initialize(ThreeBodyInitialConditions.MONTGOMERY);
    frame.setPreferredMinMax(-1.5, 1.5, -1.5, 1.5);
  }

  /**
   * Resets the animation into a predefined state.
   */
  public void reset() {
    control.setValue("dt", 0.1);
    enableStepsPerDisplay(true);
    initialize();
  }

  /**
   * Does an animation step.
   */
  protected void doStep() {
    trajectory.doStep();
    frame.setMessage("t="+decimalFormat.format(trajectory.state[4]));
  }

  /**
   * Starts the Java application.
   * @param args  command line parameters
   */
  public static void main(String[] args) {
    SimulationControl.createApp(new ThreeBodyApp());
  }
}

/*
 * Open Source Physics software is free software; you can redistribute
 * it and/or modify it under the terms of the GNU General Public License (GPL) as
 * published by the Free Software Foundation; either version 2 of the License,
 * or(at your option) any later version.

 * Code that uses any portion of the code in the org.opensourcephysics package
 * or any subpackage (subdirectory) of this package must must also be be released
 * under the GNU GPL license.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston MA 02111-1307 USA
 * or view the license online at http://www.gnu.org/copyleft/gpl.html
 *
 * Copyright (c) 2007  The Open Source Physics project
 *                     http://www.opensourcephysics.org
 */

and the model is here:

/*
 * Open Source Physics software is free software as described near the bottom of this code file.
 *
 * For additional information and documentation on Open Source Physics please see:
 * <http://www.opensourcephysics.org/>
 */

package org.opensourcephysics.sip.ch05;
import java.awt.*;
import org.opensourcephysics.display.*;
import org.opensourcephysics.numerics.*;

/**
 * ThreeBody models the gravitational three-body problem.
 *
 * @author Wolfgang Christian, Jan Tobochnik, Harvey Gould
 * @version 1.0
 */
public class ThreeBody implements Drawable, ODE {
  int n = 3; // number of interacting bodies
  // state= {x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t}

  double[] state = new double[4*n+1];
  double[] force = new double[2*n], zeros = new double[2*n];
  ODESolver odeSolver = new RK45MultiStep(this);
  Mass mass1 = new Mass(), mass2 = new Mass(), mass3 = new Mass();

  /**
   * Draws the three bodies.
   *
   * @param panel
   * @param g
   */
  public void draw(DrawingPanel panel, Graphics g) {
    mass1.draw(panel, g);
    mass2.draw(panel, g);
    mass3.draw(panel, g);
  }

  /**
   * Steps the time using an ode solver.
   */
  public void doStep() {
    odeSolver.step();
    mass1.setXY(state[0], state[2]);
    mass2.setXY(state[4], state[6]);
    mass3.setXY(state[8], state[10]);
  }

  /**
   * Initialize the three-body problem with the given state.
   * @param initial state
   */
  void initialize(double[] initState) {
    System.arraycopy(initState, 0, state, 0, 13); // copies initstate to state
    mass1.clear();                                // clears data from old trail
    mass2.clear();
    mass3.clear();
    mass1.setXY(state[0], state[2]);
    mass2.setXY(state[4], state[6]);
    mass3.setXY(state[8], state[10]);
  }

  /**
   * Computes the forces using pairwise interactions assuming equal mass.
   */
  void computeForce(double[] state) {
    System.arraycopy(zeros, 0, force, 0, force.length); // sets force array elements to 0
    for(int i = 0;i<n;i++) {
      for(int j = i+1;j<n;j++) {
        double dx = state[4*i]-state[4*j];
        double dy = state[4*i+2]-state[4*j+2];
        double r2 = dx*dx+dy*dy;
        double r3 = r2*Math.sqrt(r2);
        double fx = dx/r3;
        double fy = dy/r3;
        force[2*i] -= fx;
        force[2*i+1] -= fy;
        force[2*j] += fx;
        force[2*j+1] += fy;
      }
    }
  }

  /**
   * Gets the rate using the given state.
   *
   * @param state double[]
   * @param rate double[]
   */
  public void getRate(double[] state, double[] rate) {
    computeForce(state); // force array alternates fx and fy
    for(int i = 0;i<n;i++) {
      int i4 = 4*i;
      rate[i4] = state[i4+1];    // x rate is vx
      rate[i4+1] = force[2*i];   // vx rate is fx
      rate[i4+2] = state[i4+3];  // y rate is vy
      rate[i4+3] = force[2*i+1]; // vy rate is fy
    }
    rate[state.length-1] = 1; // time rate is last
  }

  /**
   * Gets the state: x1, vx1, y1, vy1, x2, vx2, y2, vy2, x3, vx3, y3, vy3, t.
   *
   * @return the state
   */
  public double[] getState() {
    return state;
  }

  class Mass extends Circle {
    Trail trail = new Trail();
    // Draws the mass.

    public void draw(DrawingPanel panel, Graphics g) {
      trail.draw(panel, g);
      super.draw(panel, g);
    }

    // Clears trail
    void clear() {
      trail.clear();
    }

    // Sets postion and adds to trail
    public void setXY(double x, double y) {
      super.setXY(x, y);
      trail.addPoint(x, y);
    }
  }
}

/*
 * Open Source Physics software is free software; you can redistribute
 * it and/or modify it under the terms of the GNU General Public License (GPL) as
 * published by the Free Software Foundation; either version 2 of the License,
 * or(at your option) any later version.

 * Code that uses any portion of the code in the org.opensourcephysics package
 * or any subpackage (subdirectory) of this package must must also be be released
 * under the GNU GPL license.
 *
 * This software is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston MA 02111-1307 USA
 * or view the license online at http://www.gnu.org/copyleft/gpl.html
 *
 * Copyright (c) 2007  The Open Source Physics project
 *                     http://www.opensourcephysics.org
 */

Thanks. From what I've heard, once you learn one programming language (in my case python), it's easier to learn another. What level of math and physics should I have reviewed before reading this book?

 

[Vanadium 50]

Introduction to High-performance computing (online course or book, TBD) or equivalent

Where are you planning on getting access to a supercomputer?

 

[geologist]

Where are you planning on getting access to a supercomputer?

That is a very good question. To be honest, I came across that course title or a variation of it on some of the scientific computing course listings I found (This one being one: https://www.soe.ucsc.edu/departments/applied-mathematics-statistics/teaching/scicam-masters-program).

With regards to what I would have access to, I was thinking more along the lines of this edx.org course (High-performance Computing for Reproducible Genomics https://www.edx.org/course/high-performance-computing-reproducible-harvardx-ph525-6x-1)

or this one (High Performance Finite Element Modeling https://www.edx.org/course/high-performance-finite-element-modeling-kthx-hpfem01-1x
), which has in its description "Learners that achieve top grades in this first course, in a series of two courses, will be offered access to a supercomputer, and more advanced simulations of turbulent flow." Thanks for asking the question, otherwise I wouldn't have found this one.

 

[f95toli]

Where are you planning on getting access to a supercomputer?

You don't necessary need access to supercomputer to learn the basics. Many years ago I was involved in a project where I had to learn how to use MPI and I just used a normal desktop computer; we did the same for testing the software we wrote (which was basically a Monte Carlo simulation where we simulated the same system over an over with added noise). Once I was reasonably sure everything was working it was quite easy to run the same thing on the university cluster (but with tens of thousands of iterations).

If someone is really ambitious and want access to a real supercomputer I guess you could just buy some time from a cloud based service such as Amazon's AWS.

 

[berkeman]

Specifically, I’d like to complete the approximate equivalent of a BS in scientific computing, with a focus on environmental science/geoscience applications.

I’ve completed an MS in geology (circa 2010) and work in environmental consulting, so some of what I learn will be applicable to my profession. My main motivation for completing this is because I find the subject fascinating

Good for you, it sounds like a very fun plan, and a useful career move in the long run.

What do you know already about weather simulations? That seems like an area of computing that is directly applicable to your interests. What programming languages are typically used for the large (and microclimate) simulations?

From what I've heard, once you learn one programming language (in my case python), it's easier to learn another.

I guess that's true, but I'd spend some time finding out what languages are most typically used for environmental simulations and be sure to learn one or two of those early on. Especially if there are some special features to a particular language that is used a lot, like if Object Oriented programming languages are used a lot (seems unlikely), or if multi-threaded RISC code is used a lot for the larger simulations.

 

[geologist]

Good for you, it sounds like a very fun plan, and a useful career move in the long run.

What do you know already about weather simulations? That seems like an area of computing that is directly applicable to your interests. What programming languages are typically used for the large (and microclimate) simulations?

I'm not too familiar with weather simulations, although I'd like to go through the text Principles of Planetary Climate once I build up my math skills (I know weather and climate modeling is not the exact same). I'm more familiar with groundwater modeling, having taken one course many years ago and occasionally exposed to it at my current job.