Download external files that are required by a task. To init this class, you need to provide the brick name of the task and the message dispatcher of the task. For example: TaskFileDownloader(brick_name=MyTaskClass.get_brick_name(), message_dispatcher=self.message_dispatcher)
It is recommended to use MyTaskClass.get_brick_name() because this defines the destination of the downloader file. With this, the destination will be the same even is your task is overriden by another task (and the file will not be duplicated).
:param FileDownloader: description :type FileDownloader: type
Attributes
brick_name: strdestination_folder: strmessage_dispatcher: MessageDispatcherFunctions
summary
use : self.message_dispatcher, defaults to None
| Name | Type | Default value | Description |
|---|---|---|---|
| brick_name | str | Brick name of the task, use : mytaskclass.get_brick_name() | |
| message_dispatcher | MessageDispatcher | Object to log the progress message of the downloading in the task. , optional |
Unzip a file to a given path
| Name | Type | Default value | Description |
|---|---|---|---|
| file_path | str | The path to the file to unzip `` | |
| destination_folder | str |
str |
Download a file from a given url to a given file path
| Name | Type | Default value | Description |
|---|---|---|---|
| url | str | The url to download the file from `` | |
| filename | str | ||
| headers | Dict | The headers to send with the request `dict` | |
| timeout | float | The timeout for the request `` | |
| destination_folder | str | The destination folder to save the file to, if none, the default destination folder is used `` |
str |
Download a file from a given url if the file does not already exist. This class is useful for downloading a file that is required for a task. If used within a task, it automatically logs the download progress and the time it took to download the file.
tries to download a file with the same name, it considers that the file has already been downloaded
and will not download it. If you want to force the download of a file (new version for example), change the filename (adding v2 for example).
| Name | Type | Default value | Description |
|---|---|---|---|
| url | str | Url to download the file from | |
| filename | str | Name of the file once downloaded. this filename must be unique for the brick. if a file downloader | |
| headers | Dict | Http header to attach to the download request, defaults to none [str, str], optional | |
| timeout | float | Timeout of the download request, defaults to none , optional | |
| decompress_file | bool | False | If true the file is decompress (support .zip and .tar.gz) after the download and the zip file is deleted, defaults to false , optional |
str |