Quite often we hear disputes about what is better to use Node.js or Python. But what if one day, you come up with a great idea to combine those, for instance to use some Python libraries in your application, but you just do not have any idea how to integrate it with your Node.js application. Of course you can always build API on top of Python backend(Flack, etc), but in that case you need to build, host and manage one more application, when you just need to run a single Python script. That's why I want to give you step by step instructions on how to achieve this.
To begin with, let's briefly define what are both languages.
Python is an object-oriented programming language used to create dynamic web applications. The syntax of Python and dynamic typing make it an ideal programming language for scripting.
Why you might need to run Python scripts from Node.js
- Node js can't run heavy multitask as it works on a single thread basis, making it difficult for Node Js to execute CPU-bound tasks. Whenever any request is received, the system will complete the request once and only accept the other request. This process slows down the system and ends up in a massive delay.
- Python is much more suitable for that back-end applications, that use numerical computations, big data solutions, and machine learning.
- Python has one of the largest ecosystems out of any programming community and a lot of open-source libraries. For instance some of built-in AI libraries (Pylearn2, Tensorflow for neural networks, and Scikit-learn for data analysis).
Сall your Python script from Node.js using Child Process module
You will find several libraries that can help you with this issue. As for me, I prefer to use built in child_process. The node:child_process module provides the ability to spawn subprocesses. This capability is primarily provided by the <mark>child_process.spawn() function.</mark>
First, we need to create a server.js file as the server for our app. Where we suppose that <mark>x</mark> and <mark>y</mark> are large arrays.
The first parameter, that was passed to the <mark>spawnSync</mark> is <mark>command <string></mark> (the command to run, in this case is <mark>python3</mark> - the version of python that is required to run your libraries), the second is <mark>args <string></mark> (list of string arguments). You can also pass the third parameter <mark>options</mark>, the more detailed information can be found by the link.
I propose to consider in more detail the list I am passing as the second argument:
- '/usr/src/app/scripts/python-script.py' - is the path of the script I need to execute
- 'first_function' - is the name of the function I want to run inside that file
- '/usr/src/app/scripts/args.json' - is the path to the file with arguments to this function. Actually you can directly pass those arguments using <mark>JSON.stringify()</mark>function in Node.js, but as for me it's better to write those arguments down into the file and then read that from Python script. One of the reasons is that you might need to pass as arguments large arrays of data and this will cause errors
- '/usr/src/app/scripts/results.json' - is the path, where I want my Python script to write results, after it is finished, so I can read that from Node.js.
Passed arguments can be found in <mark>sys.argv</mark>array in Python script. The <mark>sys.argv</mark> list is a list that contains command line arguments in a python program. Whenever we run a python program from a command line interface, we can pass different arguments to the program. The program stores all the arguments and the file name of the python file in the <mark>sys.argv</mark> list. This can be seen in more detail by writing a simple example script:
In two words, in Node.js script we write down to the file all required arguments, run <mark>spawnSync</mark> passing list of arguments and after Python script reads passed arguments from the file, makes all calculations and writes down to the file all results. At the moments all this results can be read in Node.js from file.
Run Python script, that uses specific Python libraries
One of the main goals is to have ability to use built-in AI or other libraries. It's always not a good idea to install dependencies and libraries globally in a system, it can lead to version conflicts, deployment difficulties, and problems with multiple developers working on a project at the same time, for example. That's why the perfect way is to use Docker. You can build the image yourself, but the easiest solution is to use an existing one.
For now we can use <mark>scikit-learn</mark>, <mark>numpy</mark>, <mark>pandas</mark> and <mark>simplejson</mark> libraries on Python script, as they are available in the created docker container.
By following the steps above, you can create a new process and execute a Python script from your Node.js application. This can be useful in cases where you need to integrate Python code into your Node.js application or when you need to perform certain tasks that are better suited for Python. Described simple app can be downloaded from the Github, there you can also find instructions how to run it.
Happy coding and wish you to have a nice day!