Petr Zemek
Blog of a Software Engineer
Links
Categories
Restarting a Python Script Within Itself
Sometimes, you may wish to check within a script when a configuration file or the script itself changes, and if so, then automatically restart the script. In this post, you will see a way of doing this in Python.
Consider the following scenario. You have a Python script that runs as a daemon and regularly performs the prescribed tasks. Examples may be a web server, a logging service, and a system monitor. When the script starts, it reads its configuration from a file, and then enters an infinite loop. In this loop, it waits for inputs from the environment and acts upon them. For example, a web server may react to a request for a page, which results into sending a response to the user.
From time to time, it may be necessary to restart the script. For example, if you fix a bug in it or change its configuration. One way of doing so is to kill the script and run it again. However, this requires manual intervention, which you may forget to do. When you fix a vulnerability in the script, you want to be sure that you do not forget to restart the script. Otherwise, someone may exploit the vulnerability if you did not restart the script. It would be nice if there existed a way of restarting the script within itself after it detected that its sources or a configuration file changed. In the rest of this post, we will show such a way.
For the purpose of the present post, let us assume that the script has the following structure:
That is, it processes the arguments and loads the configuration from the configuration files. After that, the script waits for inputs and processes them in an infinite loop.
Next, we describe how to watch files for changes. After that, we show how to restart the script.
Checking Watched Files For Changes
First, we define the paths to the files whose change we want to watch:
We watch the global configuration file, the local configuration file, and the script itself, whose path can be obtained from the special global variable __file__ . When the script starts, we get and store the time of the last modification of these files by using os.path.getmtime() :
Then, we add a check if any of these files have changed into the main loop:
If either of the files that we watch has changed, we restart the script. The restarting is described next.
Restarting the Script
We restart the script by utilizing one of the exec*() functions from the os module. The exact version and arguments depend on how you run the script. For example, on Linux or Mac OS, you can make the file executable by putting the following line to the top of the file
Then, you can run the script via
In such a situation, to restart the script, use the following code:
Otherwise, when you run the script via
Either way, do not forget to import the sys module:
To explain, the arguments of os.execv() are the program to replace the current process with and arguments to this program. The __file__ variable holds a path to the script, sys.argv are arguments that were passed to the script, and sys.executable is a path to the Python executable that was used to run the script.
The os.execv() function does not return. Instead, it starts executing the current script from its beginning, which is what we want.
Concluding Remarks
If you use the solution above, please bear in mind that the exec*() functions cause the current process to be replaced immediately, without flushing opened file objects. Therefore, if you have any opened files at the time of restarting the script, you should flush them using f.flush() or os.fsync(fd) before calling an exec*() function.
Of course, the presented solution is only one of the possible ways of restarting a Python script. Depending on the actual situation, other approaches, like killing the script externally and starting it afterwards, may be more suitable for you. Moreover, there exist other methods of checking whether a watched file has changed and acting upon such a change. If you know of another way of restarting a Python program within itself, please share it by posting a comment.
Complete Source Code
The complete source code for this post is available on GitHub.
29 Comments
This looks to be exactly what I am looking for.
I am very new to Python though so I am struggling figuring out how exactly to implement this.
I have a cron job that runs (lets call it program A) every night at midnight and goes to a webservice and checks to see if i have updated my code to my program (lets call it program B). If I have, it downloads the code and overwrites my program B code. Until know I couldn’t get program A to restart program B. So I set up a cron job that just reboots the PI every night. I don’t like this and would really like Program B to pick up the change and restart itself only if there has been a change. I am struggling on where to put this while loop in my code.
lets say i have a stop watch program that essentially shows a clock and a stop watch at the same time. Its always waiting on external inputs from GPIO to start and stop the stop watch and record their times. I tried putting your loop at the top and the bottom but my code either goes into an infinite loop or doesn’t display the main program. Any advice you could give would be appreciated.
You have to put the checking and restarting into a place that is periodically executed. For example, in a Tk application, this can be done by using root.after() as follows:
Just save the code into a file, make the file executable, and run it. Then, whenever you modify the file (its mtime attribute is changed), the application is automatically restarted. You can try it by yourself: run the application, modify the file in a text editor, and after at most two seconds, the application should be restarted.
But what if I accidentally introduce an error in the new version of this .py file? I would want the old version to keep on running (and output the error), and *try* restarting again when the file is next updated.
Your solution would have to be manually restarted after the problem is fixed.
I’m running a script on a beaglebone black with debian
Here is the part that gets the error.
this is the error
PS. I guess I need to figure out how to post a code block
thanks
bill
Hi Bill! Make sure that your script is executable, i.e. do
Then, the code you posted should work (I tested it).
Hi. Once I make the python file executable I got a new error:
I’ve checked and tried some shebangs but nothing works.
I don’t know what os.execv(__file__, sys.argv) exacly does, but seems a popen command at all.
This is what i did to relaunch the script:
Python script calling relaunch will launch itself again. I’m sure this solution do not cover memory handling or other aspects, but I need the script reloading at any cost 😀
Hi erm3nda. To fix the Exec format error , put the following line to the top of the file:
This line informs the system to run the script in Python. When this line is not present, the system does not know how to execute the script when it is relaunched.
Also, what you are doing is blocking your script with an error and launching a new instance within that.
An easy way to restart the Python Shell programmatically is as follows
That was one way I found.
@Petr Zemek about “#!/usr/bin/env python”
I saw that I tried some shebangs and still doesn’t work.
This time im working on a Windows machine and shebang does exactly nothing on it.
I’ve checked .py assoc with pythonw.exe instead python.exe and still doesn’t work.
I’m really stucked with that because that would be dead simple and is not.
Another way would be to create a simple Launcher just for that.
Nice code, just what I was looking for. Thanks!
What if the new script does eventually not work as expected and you revert back to the backed-up original? In that case
Yes, you are right, != is better. I have updated the code. Thanks!
1st of all, let me thank you very much for your explanation. I inserted your code fragment into a timer. There are two routines: 1 routine compares the NOW-time to the saved times in a list and switches the respective GPIO, when NOW- and SAVE-time are the same. This routine runs Daemon-like in the background and is started by a cronjob @reboot. The other routine let’s me change the saved times (switch times) and the names of the channels. Now, when I’m changing times or names, I save the new list with ‘pickle’ to a file. When this file is changed, the switch-routine restarts itself and reloads the pickle file to adopt the new settings. I know, it’s surely not the fine English style of coding, but it works flawlessly, that’s the essential.
2 things to remark: First of all, your code runs flawlessly unter Python 3 as well. Just use
as interpreter description (I assume it is installed…). Secondly, you can check every single file you want. Just use
Just for the beginners who don’t know – like me. (Trial and error – method…) 🙂
I also got the same problem.
It seems that __file__ refers to the compiled bytecode file with .pyc instead of the original .py.
Does this break the os.exec statement when trying to exec the bytecode directly?
Here’s my solution. It can relaunch the file even after you introduced errors.
man i will take back this
You have any idea what kind of problem is there?
If you are running on Linux/MacOS, make sure that your script is executable, i.e. do
When it complains about Exec format error , make sure that you have the following line at the beginning of the file:
Alternatively (or when you are on Windows), try changing the os.execv() line from
Man thanks for your quickly respond.
I am running on Ubuntu 14.04.
I try both of your solutions but nothing change.
You have any other idea?
From the traceback
it seems that you did not change the name of the file in your script. Try changing the line
and run the script via
thnx man… i see it and i fixed it. Now, the script run ok… i don’t run script via
but with the previous method
Great! I’m glad I could help.
Excellent….
Simply this was excellent.I went on finding for days to make a restart method for my GUI brain game.I found no way.But simply two lines from your dictums helped me out.
This is perfect. Thank you much….
I’ve put it in my excepthook function, adding the feature to control how many times should it be restarted before stopping it. Maybe this could be useful 😀
it’s a big help to me , thanks a lot
Hi, thanks a lot for sharing this, it helped me a lot. At the same time, I have encountered some issues with it as well, wondering if you could help me. I created a button with a tkinter gui, when I press the button, it would run the code
. It worked perfectly for the first time, but when I press the same button on the restarted tkinter gui, the gui was closed (which is good), the new one did not show up. Within the console, it did not show anything, it was still recognized as the program is running i guess. Which made me press “control c” to interrupt it, then it says
Fatal Python error: Py_Initialize: can’t initialize sys standard streams
Do you have any ideas? Than you so much!
Ha! I have managed to restart the program in another way, but I think its a way that everybody knows.
I simply just use the following code.
I know it is like a “primary school” stuff, but it worked for me well, thanks anyways.
I want to restart a script at a particular time for that I have made a above code and I have executed “chmod a+x dummy_code.py” also but still getting an [Errorno 8] Exec format error. Please let me what could be the possible problem.
Hi. I suggest ensuring that /home/pi/Desktop/rasp_2016/dummy_code.py
- begins with #!/usr/bin/env python , and
- is executable.
Как создать «перезапуск кода» на Python?
Всем доброго, изучаю питон самостоятельно, написал программу которая переводит числа в квадрат, в конце если пользователь пишет «Да» что бы либо он перезагружал либо зациклил этот код.
Чтобы не искать, все в низу.
#Переводчик квадратов чисел
#Приветствие
print(«Добро пожаловать в \»Переводчик квадратов чисел\»»)
start=input(«Начинаем работать?(Да,Нет): » )
#консоль начала работы
if start==»Да»:
number=int(input(«Введите число,которое нужно перевести: » ))#ввод числа
elif start==»Нет»:
print(«Досвидания =(«)
exit()
else:
restart=bool(print(«Напишите Да/Нет(С заглавной буквы)»))
if restart==False:
exit()
#список квадратов
if number==0:
print(«Результат:0»)
elif number==1:
print(«Результат:1»)
elif number==2:
print(«Результат:4»)
elif number==3:
print(«Результат:9»)
elif number==4:
print(«Результат:16»)
elif number==5:
print(«Результат:25»)
elif number==6:
print(«Результат:36»)
elif number==7:
print(«Результат:49»)
elif number==8:
print(«Результат:64»)
elif number==9:
print(«Результат:81»)
elif number==10:
print(«Результат:100»)
elif number==11:
print(«Результат:121»)
elif number==12:
print(«Результат:144»)
elif number==13:
print(«Результат:169»)
elif number==14:
print(«Результат:196»)
elif number==15:
print(«Результат:225»)
elif number==16:
print(«Результат:256»)
elif number==17:
print(«Результат:289»)
elif number==18:
print(«Результат:324»)
elif number==19:
print(«Результат:361»)
elif number==20:
print(«Результат:400»)
elif number==21:
print(«Результат:441»)
elif number==22:
print(«Результат:484»)
elif number==23:
print(«Результат:529»)
elif number==24:
print(«Результат:576»)
elif number==25:
print(«Результат:625»)
elif number==26:
print(«Результат:676»)
elif number==27:
print(«Результат:729»)
elif number==28:
print(«Результат:784»)
elif number==29:
print(«Результат:841»)
elif number==30:
print(«Результат:900»)
elif number==31:
print(«Результат:961»)
elif number==32:
print(«Результат:1024»)
elif number==33:
print(«Результат:1089»)
elif number==34:
print(«Результат:1156»)
elif number==35:
print(«Результат:1225»)
elif number==36:
print(«Результат:1296»)
elif number==37:
print(«Результат:1369»)
elif number==38:
print(«Результат:1444»)
elif number==39:
print(«Результат:1521»)
elif number==40:
print(«Результат:1600»)
elif number==41:
print(«Результат:1681»)
elif number==42:
print(«Результат:1764»)
elif number==43:
print(«Результат:1849»)
elif number==44:
print(«Результат:1936»)
elif number==45:
print(«Результат:2025»)
elif number==46:
print(«Результат:2116»)
elif number==47:
print(«Результат:2209»)
elif number==48:
print(«Результат:2304»)
elif number==49:
print(«Результат:2401»)
elif number==50:
print(«Результат:2500»)
elif number==51:
print(«Результат:2601»)
elif number==52:
print(«Результат:2704»)
elif number==53:
print(«Результат:2809»)
elif number==54:
print(«Результат:2916»)
elif number==55:
print(«Результат:3025»)
elif number==56:
print(«Результат:3136»)
elif number==57:
print(«Результат:3249»)
elif number==58:
print(«Результат:3364»)
elif number==59:
print(«Результат:3481»)
elif number==60:
print(«Результат:36»)
elif number==61:
print(«Результат:36»)
elif number==62:
print(«Результат:36»)
elif number==63:
print(«Результат:36»)
elif number==64:
print(«Результат:36»)
elif number==65:
print(«Результат:36»)
elif number==66:
print(«Результат:36»)
elif number==67:
print(«Результат:36»)
elif number==68:
print(«Результат:36»)
elif number==69:
print(«Результат:36»)
#финальная консоль
close=input(«Хотите продолжить?(Да/Нет)»)
if close==»Нет»:
print(«До свидания =(«)
exit()
elif close==»Да»:
вот здесь нужно добавить перезапуск
Я знаю что можно было побыстрее весь код обернуть, но как то вот так, с английским худо поэтому на русском (
