Base downloader
Defines an abstract base class to download documents.
We download the documents so they can be processed by other jobs
BaseDownloader
Bases: ABC
Base class for document downloader.
download(source, output, **kwargs)
abstractmethod
Download source to the output directory.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
str
|
The source to be downloaded. |
required |
output
|
str
|
The output directory where the downloaded source will be saved. |
required |
**kwargs
|
Any
|
Additional keyword arguments. |
{}
|
Returns:
| Type | Description |
|---|---|
list[str] | None
|
list[str] | None: A list of file paths of successfully downloaded files. If no files are downloaded, an empty list should be returned. Returning None is only for backward compatibility and should be avoided in new implementations. |