This documentation is under construction. Please give us your feedbacks by contacting us at firstname.lastname@example.org.
What is a resource ?
Resource is a fundamental object used in any
It represents data used or generated by tasks, or manually uploaded by a user in the
DataBox. A task only takes resources as inputs and generates resources as outputs. A resource contains different
Views that are used to generate visualizations of the resource (like chart or table).
Resources are objects generated by
tasks that store data and that can be viewed in different ways.
For example, the object
Table is a
resource that stores 2d array data with tags and can be viewed as Excel-like sheet, Line plots, Bar plots, etc.
How to create a resource ?
To create a new resource in your brick, you need to create a new class that:
- Extends the
Resource class (
import from gws_core)
- This class contains few methods to simplify resource management.
- Is decorated with the
import from gws_core)
- See the
resource_decorator for more information on how to decorate your resource
- Does not have mandatory parameters in the constructor and the constructor must call the parent constructor. This is because the system instantiates resources without parameters.
Your resource class must meet these conditions, otherwise the resource will be ignored by the system.
Here is an example of our
from gws_core import resource_decorator
from gws_core import Resource
age: int = IntRField()
position: List[float] = ListRField()
weight: float = FloatRField()
WARNING: the unique_name parameters must never be changed once the brick is released. It must be unique in the brick and it is the only way to retrieve the resource. If you change it, it will be considered as a new resource type and the previous unique_name will no longer exist (the system won't be able to retrieve resources generated with the old unique_name). You can still update other parameters including python class name, human name or short description.
We recommend that one python file contains only one resource and the name of the file is the name of the resource with undercores. Here the file is named
All output resources of a task are automatically saved after the task is run. The system will save only the attributes of the class that are marked as an
RField and will ignore all other attributes.
Here is an example of a string attribute marked as an RField that will be saved in the Resource.
my_attribute = StrRField(default_value="Hello world")
my_attribute will be saved after the task has been run and the system will automatically provide it when the resource is used in another task or is viewed.
There are 2 types of RFields :
- Searchable: this is only for small data. It will be stored in the database and therefore it can be searched using a full text search in the Databox
- Non searchable: this is for medium and large data. It will be stored in a file and the non-searchable files are lazy loaded (only loaded when there are requested) when the resource is reused.
Here is the list of the available RField to save different types of data.
IntRField : RField to save int values, by default they are searchable
FloatRField: RField to save float values, by default they are searchable
BoolRRifled : RField to save bool values, by default they are searchable
StringRField : RField to save str values, by default they are not searchable
ListRField : RField to save json list. It does not support python classes, please only provide serializable objects.
DictRField : RField to save json dictionary. It does not support python classes, please only provide serializable objects.
SerializableRField : RField to save python object. The python class of the object must be passed to the SerializableRField. The python class must extend the class
SerializableObjectJson , have an empty constructor and implements the methods :
serialize : method called when saving the resource to convert the python object to json to be stored.
deserialize : class method called when the resource object is created and the RField loaded to instantiate the python object from the previously generated json object.
FileRfield : RField to dump and load resource directly to and from a file . You must create a class that extends the
FileRField to use it (you can check the code of the
DataFrameRField that uses the
FileRField to dump
DataFrame into a file using
read_pickle function). The sub class of FileRfield must implement 2 methods :
dump_to_file : method called when saving a resource to save the RField to a file. The RField and file path are passed to this method (dump the resource in the provided file, not elsewhere).
load_from_file : method called when the resource object is created and the
RField loaded to instantiate the
Rfield from the file.
DataFrameRField : RField to save pandas Dataframe object in a file. WARNING : It uses the “read_pickle” function to deserialize the Dataframe, so only use it for one Dataframe that you trust.
ResourceRField : RField to link another resource inside a resource. The associated resource MUST be an input of the task that generated the resource containing the ResourceRField.
You can add
technical information to a resource to attach more information to the resource. These informations are always attached to the views and are visible when a user generates a view.
The technical information object is a
TechnicalInfo objects. A
TechnicalInfo is a
key/value object. More attributes will be added to it later.
To attach a
TechnicalInfo to a resource, create the
TechnicalInfo object :
technical_info = TechnicalInfo(key='key', value='value')
And then add it to the resource by calling :
The technical information will be automatically saved and accessible on the resource's views. You can also get the technical information by calling
get_technical_info(key: str): TechnicalInfo
It is also possible to add specific technical information for a view by calling
add_technical_info on the view object
A resource can have different views, to allow the user to navigate through the data of the resource with different views.
See Resource Views section for more details.