This documentation is under construction. Please give us your feedbacks by contacting us at hub@gencovery.com.
What is a resource ?
A Resource
is a fundamental object used in any Brick
.
It represents data used or generated by tasks, or manually uploaded by a user in the DataBox
. A task only takes resources as inputs and generates resources as outputs. A resource contains different Views
that are used to generate visualizations of the resource (like chart or table).
Resources are objects generated by tasks
that store data and that can be viewed in different ways.
For example, the object Table
is a resource
that stores 2d array data with tags and can be viewed as Excel-like sheet, Line plots, Bar plots, etc.
How to create a resource ?
To create a new resource in your brick, you need to create a new class that:
- Extends the
Resource
class (import from gws_core
)- This class contains few methods to simplify resource management.
- Is decorated with the
@resource_decorator
(import from gws_core
)- See the
resource_decorator
for more information on how to decorate your resource
- Does not have mandatory parameters in the constructor and the constructor must call the parent constructor. This is because the system instantiates resources without parameters.
Your resource class must meet these conditions, otherwise the resource will be ignored by the system.
Here is an example of our Robot
resource.
from gws_core import resource_decorator
from gws_core import Resource
@resource_decorator(unique_name="Robot", human_name="Robot")
class Robot(Resource):
age: int = IntRField()
position: List[float] = ListRField()
weight: float = FloatRField()
WARNING: the unique_name parameters must never be changed once the brick is released. It must be unique in the brick and it is the only way to retrieve the resource. If you change it, it will be considered as a new resource type and the previous unique_name will no longer exist (the system won't be able to retrieve resources generated with the old unique_name). You can still update other parameters including python class name, human name or short description.
We recommend that one python file contains only one resource and the name of the file is the name of the resource with undercores. Here the file is named robot.py
RField
All output resources of a task are automatically saved after the task is run. The system will save only the attributes of the class that are marked as an RField
and will ignore all other attributes.
Here is an example of a string attribute marked as an RField that will be saved in the Resource.
my_attribute = StrRField(default_value="Hello world")
The my_attribute
will be saved after the task has been run and the system will automatically provide it when the resource is used in another task or is viewed.
There are 2 types of RFields :
- Searchable: this is only for small data. It will be stored in the database and therefore it can be searched using a full text search in the Databox
- Non searchable: this is for medium and large data. It will be stored in a file and the non-searchable files are lazy loaded (only loaded when there are requested) when the resource is reused.
Here is the list of the available RField to save different types of data.
Primitive RFields
IntRField
: RField to save int values, by default they are searchableFloatRField
: RField to save float values, by default they are searchableBoolRRifled
: RField to save bool values, by default they are searchableStringRField
: RField to save str values, by default they are not searchable
Object RFields
ListRField
: RField to save json list. It does not support python classes, please only provide serializable objects. DictRField
: RField to save json dictionary. It does not support python classes, please only provide serializable objects.SerializableRField
: RField to save python object. The python class of the object must be passed to the SerializableRField. The python class must extend the class SerializableObjectJson
, have an empty constructor and implements the methods :serialize
: method called when saving the resource to convert the python object to json to be stored. deserialize
: class method called when the resource object is created and the RField loaded to instantiate the python object from the previously generated json object.
Special RFields
FileRfield
: RField to dump and load resource directly to and from a file . You must create a class that extends the FileRField
to use it (you can check the code of the DataFrameRField
that uses the FileRField
to dump DataFrame
into a file using read_pickle
function). The sub class of FileRfield must implement 2 methods :dump_to_file
: method called when saving a resource to save the RField to a file. The RField and file path are passed to this method (dump the resource in the provided file, not elsewhere). load_from_file
: method called when the resource object is created and the RField
loaded to instantiate the Rfield
from the file.
DataFrameRField
: RField to save pandas Dataframe object in a file. WARNING : It uses the “read_pickle” function to deserialize the Dataframe, so only use it for one Dataframe that you trust. ResourceRField
: RField to link another resource inside a resource. The associated resource MUST be an input of the task that generated the resource containing the ResourceRField.
Technical information
You can add technical information
to a resource to attach more information to the resource. These informations are always attached to the views and are visible when a user generates a view.
The technical information object is a dictionary
of TechnicalInfo
objects. A TechnicalInfo
is a key/value
object. More attributes will be added to it later.
To attach a TechnicalInfo
to a resource, create the TechnicalInfo
object :
technical_info = TechnicalInfo(key='key', value='value')
And then add it to the resource by calling :
my_resource.add_technical_info(technical_info)
The technical information will be automatically saved and accessible on the resource's views. You can also get the technical information by calling get_technical_info(key: str): TechnicalInfo
It is also possible to add specific technical information for a view by calling add_technical_info
on the view object
Views
A resource can have different views, to allow the user to navigate through the data of the resource with different views.
See Resource Views section for more details.