Introduction
Python live task are simple live task that allows you to write python code that takes a Resource as input and return a Resource. It only supports python code and already installed package. It can be useful to manipulate an existing resource.
The python live task only support 1 input and 1 output. If you want to manipulate multiple resources, you can use ResourceSet.
When you write the code of a python live task, there are 2 important variables available in your code context :
sources : type List[Resource] - variable already initialized with the input
Resources (an array containing a Table for example)
targets : type List[Resource]- variable not initialized, the outputs
Resources must be assigned to this variable. This will be the outputs of the task.
The context of this a python live task is the same as a Task run method . This means that you can do the same things as you would in a normal run method of a Task. You can import bricks or libraries and call methods of the Task, like self.log_info_message.
This is the only live task that has the same context as a normal Task.
Here is a simple example of a Live task that takes a Table as input/output and transpose it. It takes just 1 input and 1 output Table.
# Import modules
from gws_core import Table
# access task method to log a messages
self.log_info_message('Transposing table')
# Transpose the input table
table: Table = sources[0].transpose()
# set the new table a output or the live task
targets = [table]A third variable named working_dir is also available, see detail bellow.
How to develop a live task ?
The live task config integrate a light IDE to develop the live task. If you know what you are doing, this might be enough, but we recommend developing the live task in a real IDE before integrating it to the lab. The live task is just a script with some constraints on input/output, you can easily code it outside the lab and import the code in your lab task. This will save you some time and frustration.
You can develop it in the Codelab, here is how we recommend to do it.
If you are not familiar with the Codelab, please check the following link: https://constellab.community/tech-doc/doc/developer-guide/dev-environment-v2/getting-started
Ready to develop your live task ? Let's go 💪
Let's say we want to develop a live task that takes a Table as input, transpose it and return the transposed Table.
To do this we will develop and test our live task in a Jupyter notebook in Codelab. Open you Codelab, upload your data and create a Jupyter notebook file with the gws environment activated. You can find how to do this here : https://constellab.community/tech-doc/doc/developer-guide/dev-environment-v2/jupyter-notebook
Here is the code to do it.
# Initialize notebook to test live task
import env
cwd = env.activate() #activate GWS environment -> cwd will contain the current working dir as abolute path
# The import of any brick after the env.activate
from gws_core import File, TableImporter
# create a File resource
file = File('/lab/user/data/iris.csv')
# Import the File to Table
source = TableImporter.call(file, {"header": 0})
# convert to array
sources = [source]
#Start Live task code
from gws_core import Table
targets = [sources[0].transpose()]
#End live task code
# print result Dataframe for notebook
print(targets[0].get_data())Let's break it down
The first part until the #Start live task code comment, is the part that configure the notebook to be close to the configuration of a real live task. First we import and activate the gws environment. By using the File and TableImporter we create a Table resource from a file and assigne this Table to the source variable. We use the source variable to imitate the real live task. With this, the following code will be executed in a similar environment that the live task.
You will need an Importer (here TableImporter) to convert a file to the python Resource object. Any importer can be called using the call method. The first parameter must be a File and the seconds contains t.he parameters to call the importer.
The second part is the live task code : from #Start live task code to #End live task code . This is the code we are testing and once we are happy with it, we will be able to copy it to the live task.
the Third part after the #End live task code is for testing purpose in the notebook. This is useful to check the result. This part does not need to be copied in the live task..
Note: by doing this, we don't have the exact same context as the python live task. As this is executed in a task, you will not be able to use method from the task like the self.log_info_message
Log message
The python live task has the same context as a normal task, you can call the log methods. The logs will be available in the task messages. Theses methods can be useful to debug a live task.
To log a simple methods you can do : self.log_info_message('Hello world')
To log progresss you can do : self.update_progress_value(10, 'Reached 10%')
Here is a more detail information about the log methods : https://constellab.community/bricks/gws_core/latest/doc/developer-guide/task/task#log-messages
Working directory
Beside sources and targets variable, a third variable named working_dir is available. This variable contains the path of an empty temp directory if you need to create file or folder .After the execution of the live task, this directory is deleted. If a resource was referencing this directory (like a File or Folder), don't worry, it will be transferred before the deletion.
