kedro.io.CSVHTTPDataSet¶

class kedro.io.CSVHTTPDataSet(fileurl, auth=None, load_args=None)[source]¶

Bases: kedro.io.core.AbstractDataSet

CSVHTTPDataSet loads the data from HTTP(S) and parses as Pandas dataframe. Does not support versioning or data uploading.

Example:

from kedro.io import CSVHTTPDataSet


data_set = CSVHTTPDataSet(
    fileurl="https://people.sc.fsu.edu/~jburkardt/data/csv/cities.csv",
    auth=None,
    load_args=None)
data = data_set.load()

Methods

`CSVHTTPDataSet.__init__`(fileurl[, auth, …])	Creates a new instance of `CSVHTTPDataSet` pointing to a concrete csv file over HTTP(S).
`CSVHTTPDataSet.exists`()	Checks whether a data set’s output already exists by calling the provided _exists() method.
`CSVHTTPDataSet.from_config`(name, config[, …])	Create a data set instance using the configuration provided.
`CSVHTTPDataSet.get_last_load_version`()	Versioned datasets should override this property to return last loaded version
`CSVHTTPDataSet.get_last_save_version`()	Versioned datasets should override this property to return last saved version.
`CSVHTTPDataSet.load`()	Loads data by delegation to the provided load method.
`CSVHTTPDataSet.release`()	Release any cached data.
`CSVHTTPDataSet.save`(data)	Saves data by delegation to the provided save method.

__init__(fileurl, auth=None, load_args=None)[source]¶

Creates a new instance of CSVHTTPDataSet pointing to a concrete csv file over HTTP(S).

Parameters:

fileurl (str) – A URL to fetch the CSV file.
auth (Union[Tuple[str], AuthBase, None]) – Anything requests.get accepts. Normally it’s either
`` ('login', 'password') –
load_args (Optional[Dict[str, Any]]) – Pandas options for loading csv files. Here you can find all available arguments: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html All defaults are preserved.

Return type:

None

exists()¶

Checks whether a data set’s output already exists by calling the provided _exists() method.

Return type:	`bool`
Returns:	Flag indicating whether the output already exists.
Raises:	`DataSetError` – when underlying exists method raises error.

classmethod from_config(name, config, load_version=None, save_version=None)¶

Create a data set instance using the configuration provided.

Parameters:	name (`str`) – Data set name. config (`Dict`[`str`, `Any`]) – Data set config dictionary. load_version (`Optional`[`str`]) – Version string to be used for `load` operation if the data set is versioned. Has no effect on the data set if versioning was not enabled. save_version (`Optional`[`str`]) – Version string to be used for `save` operation if the data set is versioned. Has no effect on the data set if versioning was not enabled.
Return type:	`AbstractDataSet`
Returns:	An instance of an `AbstractDataSet` subclass.
Raises:	`DataSetError` – When the function fails to create the data set from its config.

get_last_load_version()¶

Versioned datasets should override this property to return last loaded version

Return type:	`Optional`[`str`]

get_last_save_version()¶

Versioned datasets should override this property to return last saved version.

Return type:	`Optional`[`str`]

load()¶

Loads data by delegation to the provided load method.

Return type:	`Any`
Returns:	Data returned by the provided load method.
Raises:	`DataSetError` – When underlying load method raises error.

release()¶

Release any cached data.

Raises:	`DataSetError` – when underlying exists method raises error.
Return type:	`None`

save(data)¶

Saves data by delegation to the provided save method.

Parameters:	data (`Any`) – the value to be saved by provided save method.
Raises:	`DataSetError` – when underlying save method raises error.
Return type:	`None`