Virtual en agent allows you to execute agent code in virtual environment. With this you can use a Pip, Conda or an Conda R environment. Virtual env agent takes a mandatory parameter that define the virtual environment.
As they use virtual environment to be executed, theses agent can only take File or Folder as input/output. Theses agent are not run in the context of a normal Task.
When you write the code of an agent, there are 2 important variables available in your code context :
-
source_paths : type List[string] - list of string containing the path of files and folders defined as inputs. The list size and order matches the inputs defined in the agent.
-
target_folders : typeList[string] - This variable must be set containing a list of path of output files and folders. The list size and order matches the outputs defined in the agent.
Pip env agent
This agent will allow you to run a python script inside a pip virtual environment. This environment requires a PipFile, here is the documentation about it : https://pipenv-fork.readthedocs.io/en/latest/basics.html . Juste write the text in the Pipenv configuration.
Conda/Mamba env agent
This agent will allow you to run a python script inside a Conda or Mamba virtual environment. This environment requires an environment yaml file, here is the documentation about it : https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually. Just write the content of the yml file in the Conda/Mamba env configuration.
Conda/Mamba R agent
This agent will allow you to run a R script inside a Conda or Mamba virtual environment. This is using the same yaml environment file as Conda.
This agent must have the r-base dependency in its environment.
Here is an example of the environment.yml file for Conda/Mamba R agent
name: .venv
channels:
- conda-forge
dependencies:
- r-base
Develop a virtual env agent
The agent config integrate a light IDE to develop the agent. If you know what you are doing, this might be enough, but you can also develop the agent in a real IDE before integrating it to the lab. The agent is just a script with some constraints on input/output, you can easily code it outside the lab and import the code in your lab task. This will save you some time and frustration.
You can develop it in the Codelab, here is how we recommend to do it.
Ready to develop your agent ? Let's go 💪
Install dependencies manually
The easiest way to test an virtual env agent is to install required dependencies directly in the Codelab so it is available in the notebook.
Create a virtual environment
You can also create a virtual environment and run Jupyter notebook inside this environment. This only works for Conda and R Conda env.
- Create you environment manually. You must include the
ipykernel package into your environment.
- Run the notebook using this environment.
Write and test the code
Once you notebook is ready you can develop the agent. Simply declare a source_path and target_folder variables then you can write you virtual env agent code !
Here is a virtual env agent that uses pyjwt to encode the content of a file to a jwt token and write the result in a text file.
source_paths = ['/lab/user/data/example.txt']
# Start of agent
import jwt
import os
# read source_path file
with open(source_paths[0], "r", encoding="utf-8") as fp:
data = fp.read()
encoded_jwt = jwt.encode({"text": data}, "secret", algorithm="HS256")
# write result
result_path = "result.txt"
with open(result_path, "w", encoding="utf-8") as fp:
fp.write(encoded_jwt)
target_paths = [result_path]
# End of agent
Let's break it down.
First we declare the source_paths variable, this is to initialize the code in the same context as an agent.
Then we import the jwt module which is from the pyjwt package installed in the virtual env. After that we read the source_path file, encode its content using the jwt module and write the result in a file named result.txt in the current directory. Then we set the output paths in the target_paths variable.
Here is the environment.yml file :
name: .venv3
channels:
- conda-forge
dependencies:
- python=3.8
- pyjwt==2.3.0
- ipykernel # use for notebook
Error
For virtual env agent, If an exception occurred, the line number the exception tells will be shifted by 2 (if no parameters). So if you have an error tells you that there is a problem line 12, it correspond to line 10 of your code in agent. If there are parameters the shift will be increased by one for each parameter.
