Streamlit

Streamlit is a popular Python package for rapidly building data apps. You can find more information and examples on the Streamlit website: https://streamlit.io/. With this you can integrate interactive dashboards in Constellab that aggregate informations of your scenario, across multiple scenarios or generate scenarios and notes.

When you're deploying a Streamlit app in Constellab, the system takes care of deployment, security, and app cleanup, so you don't need to worry about those aspects.

Introduction

To create a streamlit dashboard, you will need to generate a StreamlitApp resource. Here is the documentation about this resource. This is a special resource where the default view is the streamlit interactive dashboard. 

The StreamlitApp contains sub resources which allows you to view others resources that were used in the dashboard. 

Build streamlit app

There is 3 ways of building a StreamlitApp :

• With a streamlit agent
• With a streamlit virtual env agent
• With a custom task in you brick

Agent

The Streamlit agent is a special type of task that lets you create a Streamlit dashboard app connected to your lab's data. 

The inputs of the agent will be automatically added to the StreamlitApp and will be ready to use in the streamlit code. The sources variable will contains the list of input resources, similar to the python agent. 

Here is an example of a simple agent that generates a StreamlitApp. It takes a Table as input to generate an app with the dataframe and the possibility to select 2 columns to generate a scatter plot. 

# This is a template for a streamlit agent.
# This generates an app with one dataframe as input. Then the user can select 2 columns to plot a scatter plot.

import plotly.express as px
import streamlit as st
from pandas import DataFrame

# Your Streamlit app code here
st.title("Dashboard example")

# show a table from file_path which is a csv file full width
if sources:
    df: DataFrame = sources[0].get_data()

    # show the dataframe
    st.dataframe(df)

    # add a select widget with the columns names with no default value
    # set the selectbox side by side
    col1, col2 = st.columns(2)

    with col1:
        x_col = st.selectbox("Select x column", options=df.columns, index=0)

    with col2:
        y_col = st.selectbox("Select y column", options=df.columns, index=1)

    if x_col and y_col:
        # Generate a scatter plot with plotly express
        fig = px.scatter(df, x=x_col, y=y_col)
        st.plotly_chart(fig)

Here is the result app :

Virtual env agent

You can create a StreamlitApp using a virtual env streamlit agent. These agents work like virtual environment agents and support Conda, Mamba, and Pip for managing environments.

The virtual env streamlit agents only accept files and folders as input and need an environment file. In your code, you can use the source_paths variable, which is a list of paths for the input files and folders.

Custom task

You can build a StreamlitApp in you brick. This allows you to complex app with multiple files.

Create a custom task that has a StreamlitResource as output. In this task you will build the StreamlitResource, add resource to it, set parameters and provide the code for the streamlit dashboard. View StreamlitResource documentation for more information about its methods.

Inputs

To add input to you StreamlitResource, simply call the add_resource method. The inputs resources will be available under the sources variable (with the same order as added) in the streamlit code.   

Any resource can be added to the streamlit app. We recommend added pre-computed data so the streamlit code does not do heavy computation which will result in a slow dashboard. 

Config

You can add parameters to generate pre-configured dashboard. It can be useful to limit access to certains data for example. 

To set the parameters you can call the set_param or set_params methods. The parameters will be available as a dict under the params variable in the streamlit code.

Code

You can now provide the code of your streamlit app. To do this you have 2 options:

Provide a folder (recommended)

Provide a folder containing you streamlit app code. Call set_streamlit_folder and provide the path of the folder that contains your streamlit code. This allows you to provide a complete app with multiple files.

Here is few rules for the folder

• The folder must contains the entry point main.py file.
• The folder name must start with an underscore so the streamlit code is not executed when the brick is loaded.

Set code dynamically

Set code dynamically (from a string) by calling set_streamlit_code. Only call this method is the code is autonomous and doesn't rely on others files (this will not work as only this code will be copied to the streamlit app).

You can also call the set_streamlit_code_path which will read the contains of the file and save it as dynamic code. 

Example

Here is a example of a streamlit app using a folder and multiple files.

First we create the structure for our StreamlitApp and our task. 

The folder _dashboard_code contains the streamlit code. The streamlit_generator.py file contains the task that will create the dashboard. The _dashboard_code folder maintains the main.py file (entrypoint for the app). It also contains a sub module plot imported by main.py to work with multiple file. The _dashboard_code folder a a common python module structure. 

Here is the content of the main.py file. 

# This is a template for a streamlit agent.
# This generates an app with one dataframe as input. Then the user can select 2 columns to plot a scatter plot.

from typing import List

import streamlit as st
from gws_core import Resource
from pandas import DataFrame
from plot.plot_generator import generate_scatter_plot

# thoses variable will be set by the streamlit app
# don't initialize them, there are create to avoid errors in the IDE
sources: List[Resource]
params: dict

# Your Streamlit app code here
st.title(params.get('title'))

# show a table from file_path which is a csv file full width
if sources:
    df: DataFrame = sources[0].get_data()

    # show the dataframe
    st.dataframe(df)

    # add a select widget with the columns names with no default value
    # set the selectbox side by side
    col1, col2 = st.columns(2)

    with col1:
        x_col = st.selectbox("Select x column", options=df.columns, index=0)

    with col2:
        y_col = st.selectbox("Select y column", options=df.columns, index=1)

    generate_scatter_plot(df, x_col, y_col)

And the content of plot_generator sub file.

import plotly.express as px
import streamlit as st
from pandas import DataFrame


def generate_scatter_plot(dataframe: DataFrame, x_col: str, y_col: str) -> None:
    if x_col and y_col:
        # Generate a scatter plot with plotly express
        fig = px.scatter(dataframe, x=x_col, y=y_col)
        st.plotly_chart(fig)

Then we create a task that will generates the StreamlitApp. It takes a table as input and a title as config.

import os

from gws_core import (ConfigParams, ConfigSpecs, InputSpec, InputSpecs,
                      OutputSpec, OutputSpecs, StreamlitResource, StrParam,
                      Table, Task, TaskInputs, TaskOutputs, task_decorator,
                      dashboard_decorator, Dashboard, DashboardType)

@dashboard_decorator("GenerateDashboard", dashboard_type=DashboardType.STREAMLIT)
class GenerateDashboard(Dashboard):

    # retrieve the path of the app folder, relative to this file
    # the dashboard code folder starts with a underscore to avoid being loaded when the brick is loaded
    def get_app_folder_path(self):
        return os.path.join(
            os.path.abspath(os.path.dirname(__file__)),
            "_test_streamlit_dashboard"
        )

      
@task_decorator("StreamlitGenerator", human_name="Generate dashboard for table",
                short_description="Task to generate a custom Streamlit dashboard from a table")
class StreamlitGenerator(Task):

    input_specs: InputSpecs = InputSpecs({'table': InputSpec(Table, human_name="Table")})
    output_specs: OutputSpecs = OutputSpecs({
        'streamlit_app': OutputSpec(StreamlitResource, human_name="Streamlit app")
    })
    config_specs: ConfigSpecs = {
        'title': StrParam(human_name='Dashboard title')
    }

    def run(self, params: ConfigParams, inputs: TaskInputs) -> TaskOutputs:

        # build the streamlit resource with the code and the resources
        streamlit_resource = StreamlitResource()

        # set the input in the streamlit resource
        table: Table = inputs.get('table')
        streamlit_resource.add_resource(table)

        # set the param of the streamlit resource
        title = params.get_value('title')
        streamlit_resource.set_param('title', title)

        # set dashboard reference
        streamlit_resource.set_dashboard(GenerateDashboard())

        return {'streamlit_app': streamlit_resource}

That's it 🎉, the streamlit app is ready to be used. You can find the complete code of this in the academy brick.

Custom task with virtual environment

In task that generates a StreamlitResource you can also use a virtual environment. Just override the get_shell_proxy method and return a env ShellProxy.  

from gws_core import (dashboard_decorator, Dashboard, DashboardType, ShellProxy, CondaShellProxy)

@dashboard_decorator("GenerateStreamlitTestApp", dashboard_type=DashboardType.STREAMLIT)
class TestDashboard(Dashboard):

    # retrieve the path of the app folder, relative to this file
    # the dashboard code folder starts with a underscore to avoid being loaded when the brick is loaded
    def get_app_folder_path(self):
        return os.path.join(
            os.path.abspath(os.path.dirname(__file__)),
            "_test_streamlit_dashboard"
        )

    def get_shell_proxy(self) -> ShellProxy:
        env_path = os.path.join(
            self.get_app_folder_path(),
            'streamlit_env.yml',
        )
        return CondaShellProxy(env_path)

Use brick classes in streamlit normal app

In a streamlit app the gws environment to work with classes from your brick (like resources, tasks, database...) is automatically loaded. With this you can interact with the database to create real time dashboard, generate scenarios or notes...

To do this, simply import your classes from your brick.

Here is an example of a streamlit app that creates an empty scenario when the button is clicked. 

import streamlit as st
from gws_core import ScenarioProxy, Logger, User, CurrentUserService

# Your Streamlit app code here
st.title("Create scenario app")

if st.button("Create scenario"):
    st.write("Creating scenario")
    Logger.info("Creating scenario from streamlit app")
    # Call ScenarioProxy to create the scenario
    scenario_proxy = ScenarioProxy(title="Super streamlit")
else:
    st.write("No creating")
  

Develop the streamlit app

When you're deploying a Streamlit app in Constellab, the system takes care of deployment, security, and app cleanup, so you don't need to worry about those aspects.

Run the app in dev mode

To run the app in dev mode create a folder with a main.py file and a dev_config.json file.

The main.py file will contains your streamlit code.

The dev_config.json file will be used simulate the real execution when the dashboard is generated by a Task. Here is the content of the file. 

{
    "app_dir_path": "",
    "source_ids": [],
    "params": {
        "title": "Dev mode"
    },
    "env_type": "NORMAL",
    "env_file_path": ""
}

• app_dir_path : string. Path of the folder containing the streamlit main.py file. It is recommended to set a relative path (relative to the config json file).
• source_ids : List[string] . You can pass a list of resource input to simulate the real use. The resource must be defined in the dev environment.
• To find the resource id, open the resource in Constellab dev environment and retrieve the id from the url or open Resource info. If you don't have the resources, in dev mode, run the task that generates the dashboard and use the resource from the generated StreamlitResource.
• The order in the array is important, it must be the same as your order in the code.
• params : dict. Dictionnary of parameters to pass to the app. The keys must be the same as the keys defined in the StreamlitResource. Here title is the same as when we gyuyugcalled streamlit_resource.set_param('title', title)
• env_type : 'NONE' | 'CONDA' | 'MAMBA' | 'PIP'. Type of environment to run the streamlit app. Default toNONE which is the lab environment.
• env_file_path: string. If the env is not normal, the env_file_path must be the path of the environment file (yml or txt). It is recommended to set a relative path (relative to the config json file)

Here is how you folder should look like. 

With this, the sources and params variables will be provided in your main.py file when the app is run. You can declare them in your main.py flie but do not initialize them, they will be provided automatically based on dev_config.json file. 

# main.py file
# thoses variable will be set by the streamlit app
# don't initialize them, there are create to avoid errors in the IDE
sources: List[Resource]
params: dict

Once this file is defined, you can run the streamlit app in dev mode using the following command : gws streamlit run-dev dev_config.json .  Find information about the CLI here. A message is print that show in which url you can access the dashboard:

Example

Here is a example of a streamlit app using a folder and multiple files.

First we create the structure for our StreamlitApp and our task. 

The folder _dashboard_code contains the streamlit code. The streamlit_generator.py file contains the task that will create the dashboard. The _dashboard_code folder maintains the main.py file (entrypoint for the app). It also contains a sub module plot imported by main.py to work with multiple file. The _dashboard_code folder a a common python module structure. 

Here is the content of the main.py file. 

# This is a template for a streamlit agent.
# This generates an app with one dataframe as input. Then the user can select 2 columns to plot a scatter plot.

from typing import List

import streamlit as st
from gws_core import Resource
from pandas import DataFrame
from plot.plot_generator import generate_scatter_plot

# thoses variable will be set by the streamlit app
# don't initialize them, there are create to avoid errors in the IDE
sources: List[Resource]
params: dict

# Your Streamlit app code here
st.title(params.get('title'))

# show a table from file_path which is a csv file full width
if sources:
    df: DataFrame = sources[0].get_data()

    # show the dataframe
    st.dataframe(df)

    # add a select widget with the columns names with no default value
    # set the selectbox side by side
    col1, col2 = st.columns(2)

    with col1:
        x_col = st.selectbox("Select x column", options=df.columns, index=0)

    with col2:
        y_col = st.selectbox("Select y column", options=df.columns, index=1)

    generate_scatter_plot(df, x_col, y_col)

  And the content of plot_generator sub file.  

import plotly.express as px
import streamlit as st
from pandas import DataFrame


def generate_scatter_plot(dataframe: DataFrame, x_col: str, y_col: str) -> None:
    if x_col and y_col:
        # Generate a scatter plot with plotly express
        fig = px.scatter(dataframe, x=x_col, y=y_col)
        st.plotly_chart(fig)

 Then we create a task that will generates the StreamlitApp. It takes a table as input and a title as config.  

import os

from gws_core import (ConfigParams, ConfigSpecs, InputSpec, InputSpecs,
                      OutputSpec, OutputSpecs, StreamlitResource, StrParam,
                      Table, Task, TaskInputs, TaskOutputs, task_decorator,
                      dashboard_decorator, Dashboard, DashboardType)

@dashboard_decorator("GenerateDashboard", dashboard_type=DashboardType.STREAMLIT)
class GenerateDashboard(Dashboard):

    # retrieve the path of the app folder, relative to this file
    # the dashboard code folder starts with a underscore to avoid being loaded when the brick is loaded
    def get_app_folder_path(self):
        return os.path.join(
            os.path.abspath(os.path.dirname(__file__)),
            "_test_streamlit_dashboard"
        )

@task_decorator("StreamlitGenerator", human_name="Generate dashboard for table",
                short_description="Task to generate a custom Streamlit dashboard from a table")
class StreamlitGenerator(Task):

    input_specs: InputSpecs = InputSpecs({'table': InputSpec(Table, human_name="Table")})
    output_specs: OutputSpecs = OutputSpecs({
        'streamlit_app': OutputSpec(StreamlitResource, human_name="Streamlit app")
    })
    config_specs: ConfigSpecs = {
        'title': StrParam(human_name='Dashboard title')
    }

    def run(self, params: ConfigParams, inputs: TaskInputs) -> TaskOutputs:

        # build the streamlit resource with the code and the resources
        streamlit_resource = StreamlitResource()

        # set the input in the streamlit resource
        table: Table = inputs.get('table')
        streamlit_resource.add_resource(table)

        # set the param of the streamlit resource
        title = params.get_value('title')
        streamlit_resource.set_param('title', title)

        # set dashboard reference
        streamlit_resource.set_dashboard(GenerateDashboard())

        return {'streamlit_app': streamlit_resource}

The Dashboard class is used to retrieve the path of the streamlit folder. With this method, the pass of the folder can change as the StreamlitResource uses the Dashboard class to retrieve the folder path. Do not change the unique name of the Dashboard in the @dashboard_decorator.  

That's it 🎉, the streamlit app is ready to be used. You can find the complete code of this in the academy brick.

Custom components

In gws_core we developed few custom component for streamlit app. They can be imported from gws_core.streamlit package. 

You can find the code of those component here.

Select resource

The ResourceSearchInput class allows you to show a resource selector in your streamlit dashboard. It allows user to search resource by name and you can prefilter the result (only showing resource of a type for example). 

from gws_core.streamlit import ResourceSearchInput

selected_resource: ResourceModel | None = ResourceSearchInput() \
    # only show the flagged resources
    .add_flagged_filter(True) \
    # only show not archived resource
    .add_is_archived_filter(False) \
    # order by name
    .add_order_by(ResourceModel.name) \
    # call select
    .select(placeholder="Search for folder")

if selected_resource:
    st.write(selected_resource.name)

Rich text

The rich_text_editor method allows you to include the rich text editor (similar to note) in your dashboard.

from gws_core.streamlit import rich_text_editor
from json import dump, load

folder = sources[0]
file_path = os.path.join(folder.path, 'test.json')

# initialising the rich text from a json file
rich_text: RichText = None
if os.path.exists(file_path):
    # load json file to rich text
    with open(file_path, 'r') as f:
        rich_text = RichText.from_json(load(f))
else:
    rich_text = RichText()

# calling component
result = rich_text_editor(placeholder='My note', initial_value=rich_text,
                          key="note")

if result:
    # saving modified rich text to json file
    with open(file_path, 'w') as f:
        dump(result.to_dto_json_dict(), f)

Dataframe paginator

Streamlit doesn't have a way of paginate big dataframe natively. You can use the dataframe_paginated method from gws_core.streamlit. It support row and column pagination.

Here is an example on how to use it:

from gws_core.streamlit import dataframe_paginated
from pandas import DataFrame

my_df : DataFrame = sources[0]

dataframe_paginated(my_df, paginate_rows=True, row_page_size_options=[50,100 ,250], 
                   paginate_columns=False, column_page_size_options=None)

Troubleshoot

Connecting / Connection lost / Connection timed out

If you encounter connection issues such as "Connecting/Connection lost/Connection timed out" when opening the Streamlit app, please close the resource or view and then reopen it. This should resolve the issue and allow you to use the app again.

Port 8501 already is use

When developing a streamlit app, if you encounter error 'Port 8501 already is use', this means that you have another streamlit app that is already running. Streamlit app can be running : 

• in a terminal of the codelab, in this case, kill the terminal
• in the lab in dev mode. In Monitoring > Other > Streamlit app you can check if there is an app running and stop it. You can also kill the dev environment.

If you can't find the running streamlit app, another solution might be to restart the lab from the lab manager section.