Getting Started
About
The repository was created from a ProjectPythia cookbook template. This template brings with it all machinery to enable full featured GitHub workflows, including building a docker image, running and rendering Jupyter Notebooks and compiling a website using Sphinx and the JupyterBook theme.
Customizing
If there is a package missing for your use-case you can just add the package to binder/environment.yml and activate a new build (see below). Notebooks might be added to the notebooks-folder.
Build workflow
We work on pre-building the environment and toolkit so you can deploy this on the platform of your choice! Making the Open Radar Science stack available across different environments is essential to ensure reproducibility. We take advantage of “containerization”, which enables packaging all of these tools together so you can deploy it on the platform of your choice. In this case, we are deploying a software stack which includes a variety of different languages (ex. Python, C++, C) and deploy it on a cloud platform. We use Docker as our main containerization tool.
Build on GitHub - PullRequest
repo2docker-action is used to build the docker image
docker layers will be cached from ghcr.io
image is not pushed, since GITHUB_TOKEN
writeis not available as per security policy
book is built in a second step using docker-run-action, zipped and uploaded as artifact
book is deployed to gh-pages, link is added to pr comment
Build on GitHub - Push
repo2docker-action is used to build the docker image
book is built in a second job directly inside ghcr.io, zipped and uploaded as artifact
book is deployed to gh-pages
Depending on the changes in the repo the build times can be quite low as docker layer caching is facilitated.
Build & run locally
If you want to build and run locally using repo2docker you would need to remove (temporarily) the binder/Dockerfile. Then you would need to invoke the build with:
$ repo2docker --appendix "`cat binder/appendix.txt`" .
This will build the docker image locally and fire up a container running jupyterlab.
Run locally
If you want to just run locally using repo2docker you will just use the provided binder/Dockerfile. Then you would need to invoke the run with:
$ repo2docker .
This will fetch the docker image from ghcr and fire up a container running jupyterlab.
Health check
The complete build workflow is run as nightly build in GitHub CI to early detect problems. The Github action builds the environment, executes the notebooks, and checks the different links within the content. This ensures that the content is still executable, and we do not run into issues when building the environment or running the computational workflows.
Running these health checks results in the following badge:
Which is included in our README, letting users know whether the content is still executable. If this workflow fails, it also notifies the developers of this repository that there is an issue.
Conclusion
Docker images are built using GHA with all essential components inside (and pushed to ghcr.io on push)
On binder we are using now our prebuild images on ghcr.io. As we are using
appendixinstead ofpostBuildno compilations are necessary on the binder side, the images is used as is with some minor adaptions by binder.On GHA we are using our prebuild images as layer cache, but layers can’t be updated on pull requests due to security reasons.
The erad2022 package on ghcr.io represents always the status of the most recent commit to the repo.
Running locally as well as building & running is easily possible.
Using the PANGEO Cloud with Binder
1. Make Sure You Have a Github Account
The first step is to make sure you have a Github Account.
Here is the link if you do have one already:
2. Log into Pangeo Binder
Next, sign in and authenticate the Pangeo Binder, which is the platform we will use for the workshop:
The JupyterHub instance we use for this course is relatively small in memory and compute power (~10 GB of memory, 2 CPU cores). For more information about all of the open computational resrouces available within the Pangeo community, check out the Pangeo Cloud documentation.
3. Launch our Environment
Now that we have our authentication set up, we can access our content!
Use the following link to launch into the binder:
If you are having issues with that (ex. it is taking a long time), try using the following link:
https://hub.aws-uswest2-binder.pangeo.io/user/{your_github_username}/lab
Where you replace {your_github_username} with your github username (ex. mgrover1)
Running notebooks
If you finally succeeded with the above procedure you will have a JupyterLab instance up and running. Then you can select the notebooks as usual from the right navigation and simply run through them. If you encounter any problems please raise an issue here.