I've trained a model and deployed it but it takes up to 20 seconds to classify (importing dependencies, loading the model, making the classification), given an input. Is there anything I can do to permanently save the model into RAM and use it whenever I want?
Asked
Active
Viewed 66 times
-2
-
Permanently save into RAM? No. That's one of the defining characteristics of RAM is that anything stored there is not permanent. – mypetlion Aug 12 '19 at 18:05
-
By permanently I mean from the moment I load the model it remains in memory – Youcef zy'm Aug 12 '19 at 20:00
2 Answers
0
You don't save your model, you load it and use it. Just define the model variable as a module variable. See How to create module-wide variables in Python?
cookiemonster
- 614
- 6
- 11
-
I think this ll take the same amount of time because it does the same thing import deps, loading model ( as a global variable ), feed forward the given input – Youcef zy'm Aug 12 '19 at 22:05
-
No you would actually load the model only once at the start of the app. Have a look at https://pytorch.org/tutorials/intermediate/flask_rest_api_tutorial.html#prediction – cookiemonster Aug 13 '19 at 09:39
0
I don't know about your system configuration, but personally I use a "service" on local machine and a web-service when deploying on the client side. That allow you to load one time the dependencies and your model, then it remain in memory as long as your service is running. This way you can access your model barely instantly.
Lethos
- 26
- 3
-
You can see an exemple here : https://stackoverflow.com/questions/32404/how-do-you-run-a-python-script-as-a-service-in-windows Also there is multiple way of doing such a process, but the idea behind is the same, you just need a program that keep running somewhere (local, cloud, etc) and that provide you the output you want based on the input you send. – Lethos Aug 13 '19 at 12:09