Saving the state of a program versus running the program

[fog37]

TL;DR Summary

Saving data and saving the state of a program (persistence)

Hello Forum,

I am not completely clear on the idea of saving the state of a Python program. We can write a .py program which has instructions, formulas, variables, user inputs, and uses data from an external file, etc., and run such file. However, running the program is different from saving its state.

I see how, sometimes, the data generated by the program during its execution needs to be saved in an external file using JSON or pickle. Not sure if that is saving the state...Saving the state seems to imply that the entire program is saved as if we were taking a snapshot of it.
Without saving the state, everything is temporarily saved in RAM and disappears after the program has run.

Some applications work with some kind of "persistent" state. Saving the state seems to essentially mean that we store values away outside of the program and whenever we need the values we read the from that outside place. External files can be JSON, YAML, TXT, or it could be another program, like a database.

People that work with machine learning create ML models and train them (set ideal model weights) before deploying the model.
After training, the model can be used with the weights obtained during training. But that is only possible if we save the state of the model. Otherwise, if we run the ML model every time, the weight would change every time. Saving the state of the model means saving the trained model into a different file to reload it later when we need to use it. The model will be identical without having to run all the data processing and training phase. The model file (json, binary, etc.) contains the model properties (layer type, activation functions, weights, etc.)

Is my understanding correct? Any corrections?

Thanks!

 

[PeterDonis]

I am not completely clear on the idea of saving the state of a Python program.

Why do you think there is a single, well-defined idea of "saving the state" of a Python program?

 

[DaveE]

Programs have internal registers that contain data that is only necessary in a temporary context. The simplest example may be the Program Counter, which contains the memory address of the next instruction to be executed. This is a fundamental state of the program, i.e. where you are in the execution. However, it is only sometimes necessary to save it. The uP may save it if your program jumps away to do something else, so it knows where to resume. But for the simplest programs, it has no value to the outside world, and therefore is discarded when the program doesn't need it any more.

Of course uPs are way more complicated than this, with several internal registers that control "the state" of the program being executed. Normally this sort of internal information isn't counted when people talk about the programs data. "Data", normally exists before and/or after the program is executed. OTOH, computers are complicated and so are the ways that people talk about them. I imagine that my answer isn't universally agreed with.

 

[fog37]

Why do you think there is a single, well-defined idea of "saving the state" of a Python program?

Well, not sure...I know that we can pickle only certain types of Python data structures but we don't generally pickle an entire Python script... However, we can convert data to json or even an entire script. The acts of serializing them seems equivalent to saving the state...

 

[PeterDonis]

I know that we can pickle only certain types of Python data structures but we don't generally pickle an entire Python script

Why would you want to pickle an entire script? You already have its source code.

The acts of serializing them seems equivalent to saving the state

I think you have not thought very carefully about what "the state" even means, or whether what "the state" means is not a single thing but depends on the particular application you are running.

 

[fog37]

Well, you may be right. I believe "state" means the current state of the program or data...

In the case of machine learning models, it seems like the state of the entire model gets saved...

Why would you want to pickle an entire script? You already have its source code.I think you have not thought very carefully about what "the state" even means, or whether what "the state" means is not a single thing but depends on the particular application you are running.

 

[FactChecker]

An operating system might be able to completely save the state of a program, like the "hibernate" operation of a Windows system. That saves the state of all running programs. I have never seen a programmable method of saving the complete state of a single program that is done by itself. It might be a handy way of occasionally saving intermediate restart points of a long-running program. But there might be issues with it like the first thing it might do on a restart is to save itself.

 

[PeterDonis]

I believe "state" means the current state of the program or data...

In other words, "state" means "state". Not a very helpful definition.

In the case of machine learning models, it seems like the state of the entire model gets saved

What does "the state of the entire model" mean? Just saying "the state" does not tell anyone anything useful, as noted above.

Whatever meaning you assign to "state" for machine learning models, it will be specific to that kind of application. Other applications will have different meanings of "state". So, contrary to the assumption you made in the OP of this thread, there is not a single notion of "saving the state of a program". There are many different notions, corresponding to the many different kinds of application programs.

 

[PeterDonis]

An operating system might be able to completely save the state of a program, like the "hibernate" operation of a Windows system. That saves the state of all running programs.

Only in the sense that the computer can be restarted and all the programs will be as they were when it was hibernated. But this "state" is not accessible to the programmer--there's no function provided by the OS that any program can call that returns the "state" of the program, when it was last hibernated or at any other time. Each particular kind of application has to write "save state" and "restore state" functions for itself.

 

[FactChecker]

Summary:: Saving data and saving the state of a program (persistence)
...
After training, the model can be used with the weights obtained during training. But that is only possible if we save the state of the model.

Saving some state variables of a model is very different from saving the "state" of a running program. Neural network weights are a specific set of parameters of the model that can be stored in a file. That is routine.

 

[PeterDonis]

Saving some state variables of a model is very different from saving the "state" of a running program.

That depends on how you define the "state" of a running program. As I have already pointed out, there is not one single definition of that term that applies everywhere.

In your example of a computer hibernating, the hibernation routine doesn't even have the concept of "running program". It just saves the entire contents of RAM and CPU registers to disk (or to non-volatile memory). Any information about "running programs" is implicit in the contents of RAM and CPU registers; the hibernation routine doesn't have any way of extracting it, nor does it need to. So it would be more accurate to say that hibernation saves the "state" of the computer as a whole, not of particular running programs.

 

[DaveC426913]

My concept of saving the state - probably the simplest defintion - is such that - once the state is saved - you can shut down the running program and at some later time restart it exactly where you left off.

This can be as simple as storing the value of all variables and then - on relaunch - having a boot-up routine that retrieves and sets those variables. It may or may not return to the last routine that had been run and start executing again from there.

There is no need to save the program (i.e. lines of code) itself, since it's generally a good idea to have them saved in a non-volatile format elsewhere.

 

[FactChecker]

So it would be more accurate to say that hibernation saves the "state" of the computer as a whole, not of particular running programs.

Good point. And that makes it simpler in many ways. If a program tried to save its own state, it would need to be careful not to change registers or any internal variables as it did that. It might be possible for the operating system to supply a method to do it, but I have never seen such a thing. If a program runs on a virtual machine in a "sandbox", I suppose that it would be possible.

 

[FactChecker]

It's not clear to me what the OP had in mind since the example given is simply saving the results of neural network training. That is a simple and routine thing.

 

[DaveE]

It may be helpful to look into RTOSs. This is something that multi-tasking real time operating systems do all the time. It is also what simple uPs do when they process interrupts. Sometimes it's simple like interrupts, sometimes it's quite complex, like a task switching in a RTOS.

This is what your phone does when you are listening to music and you get a phone call. There are a wide range of implementations in the real world, so I think you won't get a simple or definitive answer.

 

[anorlunda]

My concept of saving the state - probably the simplest defintion - is such that - once the state is saved - you can shut down the running program and at some later time restart it exactly where you left off.

That's a good definition, but it is a feature seldom available. A true state save means the program can be interrupted at any instruction and the contents of all hardware registers, the display window, the mouse position, the state of keyboard keys and mouse buttons, menu dropdowns, and all program data are saved to be restored later.

If you had a true state save created when the user had typed PRIN, then when it restored the STO would appear on the screen and the program would wait for the user to type T and RETURN. That's not particularly user friendly and therefore undesirable. Undesirability is a good reason for not providing the feature.

I learned that lesson 50 years ago when building a training simulator. I created a state save according to your definition. Later the simulator could be initialized to that saved state to begin a lesson. However, I found that the state save was too literal and the customers were unhappy. All those user interface things the customer preferred all user interfaces to be initialized to a neutral state, not to the saved state.

Visualize the printer printing a report. If the state was save when it was halfway through a page, then when it was restored months or years later, it would finish printing the remainder of the report, but the piece of paper in the printer would not be the half printed report. Nobody wants that.

 

[PeterDonis]

This is what your phone does when you are listening to music and you get a phone call.

Actually, what a smartphone does when task switching bears little if any resemblance to a simple "just save the state" in terms of RAM and CPU registers.

In most cases, a smartphone task switches the same way a desktop computer does: both applications are still in RAM and can still execute instructions, the only difference is which one is taking input from the user. (Depending on how the screen is configured, both may still be displaying output to the user. Other output, such as sound, will generally be controlled by just one app, whichever one has highest priority, as in your example of a phone call interrupting music.)

In cases where an app has to be "saved to disk" to reclaim memory, smartphone OS's do not have any OS-provided generic "save" mechanism. Instead, they send the app an event that says "you're about to be saved to disk and removed from memory, make sure you store whatever state you need to re-start". And then it's up to the app to decide what "state" it, specifically, needs to save.

 

[phinds]

So it would be more accurate to say that hibernation saves the "state" of the computer as a whole, not of particular running programs.

Exactly.

 

[FactChecker]

One important issue that may not have been mentioned is the interaction with hardware and external devices. To store a program state and stop it without corrupting disk drives, losing communications, and damaging other external devices is a tricky business. And restarting it as though nothing had happened would be even harder. I assume that a program running in a virtual machine would have the hardware protected by the actual machine, but restarting it would probably not be guaranteed.

 

[DaveE]

Actually, what a smartphone does when task switching bears little if any resemblance to a simple "just save the state" in terms of RAM and CPU registers.

In most cases, a smartphone task switches the same way a desktop computer does: both applications are still in RAM and can still execute instructions, the only difference is which one is taking input from the user. (Depending on how the screen is configured, both may still be displaying output to the user. Other output, such as sound, will generally be controlled by just one app, whichever one has highest priority, as in your example of a phone call interrupting music.)

In cases where an app has to be "saved to disk" to reclaim memory, smartphone OS's do not have any OS-provided generic "save" mechanism. Instead, they send the app an event that says "you're about to be saved to disk and removed from memory, make sure you store whatever state you need to re-start". And then it's up to the app to decide what "state" it, specifically, needs to save.

Yes, I was trying to ignore the fact that modern phones have multiple cores and complex OSs. The fact that things stay in RAM strikes me as irrelevant. They were and still are stored, somewhere. The idea of saving the state implies to me that you have some critical resource like a uP register that can't be shared and so has to be copied to somewhere else. But even this is a gross simplification of modern processors that often have multiple sets of registers to facilitate rapid task switching as well as multiple forms of memory. In the wide world of computer cores and OSs there are few simple or definitive answers about task switching.

Honestly, I don't think these questions make much sense, unless you are working in a specific environment, or you restrict the problem to a very simple uP. I have work with good embedded developers that literally spent months learning to use a new RTOS, let alone design one. In my case, a HW guy, the last uP I completely 100% understood was the 8080.

 

[DaveC426913]

If you had a true state save created when the user had typed PRIN, then when it restored the STO would appear on the screen and the program would wait for the user to type T and RETURN. That's not particularly user friendly and therefore undesirable. Undesirability is a good reason for not providing the feature.

Indeed, that was the gist of my comment:

"It may or may not return to the last routine that had been run and start executing again from there."

You generally don't want it returning to the exact spot it left off. Often that is right before a crash.

(There was a sci-fi story about that somewhere. A guy had a time machine in a lab(?) and could go to any point in time, but he had to return to the exact same time and location in his lab every time. A lab which was just slightly more on-fire every time he returned. Or something like that.)

 

[PeterDonis]

The fact that things stay in RAM strikes me as irrelevant. They were and still are stored, somewhere.

Some data is stored somewhere, but a program that is running might not have all relevant information that is in RAM stored to disk or other nonvolatile storage at all times. That's why smartphone OS's send events to apps that are about to be removed from memory, so they can make sure any state that isn't currently stored can be stored.

 

[sysprog]

Maybe have a look at IBM checkpoint/restart e.g. http://www.bitsavers.org/pdf/ibm/37...int_Restart_Supervisor_Call_Logic_Dec1985.pdf