Create intake description files
...............................

To get rid of multi readings of the metadata of a set of files and to quicky access your dataset of interest (or a subset), the user is encouraged to create a single .json file that describes the metadata of the full set. This .json file will be read to speed up the creation of any future xarray dataset.

The creation of the .json file requires the use of **auto-kerchunk** pre-processing tool.

First create the python environment dedicated to auto-kerchunk tool::

  source "/appli/anaconda/versions/4.8.3/etc/profile.d/conda.csh"
  conda create -c conda-forge -n auto-kerchunk python==3.9
  conda activate auto-kerchunk
  conda install xarray kerchunk ujson h5py zarr fsspec dask rich typer zstandard intake intake-xarray

And install *dask-hpcconfig* and *auto-kerchunk* modules to your environment following these instructions (use your own extranet login!)::

  python -m pip install git+https://username@gitlab.ifremer.fr/iaocea/dask-hpcconfig.git
  python -m pip install git+https://username@gitlab.ifremer.fr/iaocea/auto-kerchunk.git

or from IAOCEA projects downloaded locally::

  git clone https://gitlab.ifremer.fr/iaocea/auto-kerchunk
  git clone https://gitlab.ifremer.fr/iaocea/dask-hpcconfig

  cd pathto/dask-hpcconfig
  python -m pip install .
  cd pathto/auto-kerchunk
  python -m pip install .


Create the metadata and intake configuration files related to your own dataset:

.. literalinclude::  _static/auto_kerchunk.pbs.txt