TaskFileDownloader

Download external files that are required by a task. To init this class, you need to provide the brick name of the task and the message dispatcher of the task. For example: TaskFileDownloader(brick_name=MyTaskClass.get_brick_name(), message_dispatcher=self.message_dispatcher)

It is recommended to use MyTaskClass.get_brick_name() because this defines the destination of the downloader file. With this, the destination will be the same even is your task is overriden by another task (and the file will not be duplicated).

:param FileDownloader: description :type FileDownloader: type

Attributes

brick_name: strdestination_folder: strmessage_dispatcher: MessageDispatcher

Functions

__init__

summary

brick_name : str

brick name of the task, use : MyTaskClass.get_brick_name()

message_dispatcher : MessageDispatcher

object to log the progress message of the downloading in the task. use : self.message_dispatcher

decompress_file

Unzip a file to a given path

file_path : str

The path to the file to unzip

destination_folder : str

Return type : str

download_file

Download a file from a given url to a given file path

url : str

The url to download the file from

filename : str

headers : Dict

The headers to send with the request

timeout : float

The timeout for the request

destination_folder : str

The destination folder to save the file to, if None, the default destination folder is used

Return type : str

download_file_if_missing

Download a file from a given url if the file does not already exist. This class is useful for downloading a file that is required for a task. If used within a task, it automatically logs the download progress and the time it took to download the file.

url : str

url to download the file from

filename : str

name of the file once downloaded. This filename must be unique for the brick. If a file downloader tries to download a file with the same name, it considers that the file has already been downloaded and will not download it. If you want to force the download of a file (new version for example), change the filename (adding v2 for example).

headers : Dict

http header to attach to the download request

timeout : float

timeout of the download request

decompress_file : bool - False

if true the file is decompress (support .zip and .tar.gz) after the download and the zip file is deleted

Return type : str

Have you developed a brick?