Metadata-Version: 2.1
Name: ecmwfspec
Version: 0.0.4
Summary: fsspec implementation for ECMWF File Storage System
Project-URL: Homepage, https://github.com/observingClouds/ecmwfspec
Requires-Dist: fsspec
Requires-Dist: dask
Requires-Dist: universal-pathlib
Requires-Dist: typing_extensions; python_version < "3.11"
Requires-Dist: xarray<=2024.11.0
Provides-Extra: tests
Requires-Dist: mock; extra == "tests"
Requires-Dist: pre-commit; extra == "tests"
Requires-Dist: netCDF4; extra == "tests"
Requires-Dist: numpy; extra == "tests"
Requires-Dist: pandas; extra == "tests"
Requires-Dist: xarray; extra == "tests"
Requires-Dist: zarr<3.0.0; extra == "tests"
Requires-Dist: pytest; extra == "tests"
Requires-Dist: pytest-cov; extra == "tests"
Description-Content-Type: text/markdown

[![CI](https://github.com/observingClouds/ecmwfspec/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/observingClouds/ecmwfspec/actions?query=workflow%3ATests)

# ecmwfspec

This is work in progress! This unofficial repository aims to provide an fsspec driver for the [ECMWF File Storage System](https://confluence.ecmwf.int/display/UDOC/ECFS+user+documentation).

Pull requests (also for additional protocols like MARS and FDB) are welcomed!

```python
import fsspec

with fsspec.open("ec:/<user>/path/to/file", "r") as f:
    print(f.read())
```
## Loading datasets

```python

import fsspec
import xarray as xr

url = fsspec.open("ec:/arch/project/file.grib").open()
ds = xr.open_dataset(url, engine='cfgrib')  # does not work until https://github.com/ecmwf/cfgrib/issues/326 is solved
```


### Usage in connection with gribscan
This is just an example what ecmwfspec can be used for.

```python
import gribscan
import json
import xarray as xr

file_to_scan = "ec:///path/to/some/grib/file/forecast+0038.grib2"
index_file = "index.json"
reference_file = "reference.json"

gribscan.write_index(gribfile=file_to_scan, idxfile=index_file)  # required currently patch https://github.com/gribscan/gribscan/commit/7a5e595759f48e3118964091358f1b2e9eb32b37 to work with fsspec paths
magician = gribscan.magician.HarmonieMagician()  # use magician fitting the grib file
refs = gribscan.grib_magic(
                        filenames=[index_file],
                        magician=magician,
                        global_prefix="",
                    )


with open(reference_file, "w") as outfile:
    json.dump(refs["heightAboveGround"], outfile)
ds = xr.open_zarr(f"reference::{reference_file}")
ds.u.max()  # with the help of ecmwfspec the data is now fetched if it is not locally cached
```
