NEMO_Nowcast Package Development

Continuous Integration

 Pytest with Coverage Status Codecov Testing Coverage Report CodeQL analysis

Documentation

 Documentation Status Sphinx linkcheck

Package

 Releases Python Version from PEP 621 TOML Issue Tracker

Meta

 Licensed under the Apache License, Version 2.0 Licensed under the BSD-3-Clause License Git on GitHub
pre-commit The uncompromising Python code formatter Hatch project

Python Versions

Python Version

The NEMO_Nowcast package is developed and tested using Python 3.13.

Getting the Code

Git on GitHub

Clone the code and documentation repository from GitHub with:

$ git clone git@github.com:43ravens/NEMO_Nowcast.git

Development Environment

Setting up an isolated development environment using Conda is recommended. Assuming that you have Miniconda3 installed, you can create and activate an environment called nemo-nowcast that will have all of the Python packages necessary for development, testing, and building the documentation with the commands:

$ cd  NEMO_Nowcast
$ conda env create -f envs/environment-dev.yaml
$ conda activate nemo-nowcast

The NEMO_Nowcast package is installed in editable install mode as part of the conda environment creation process. That means that the package is installed from the cloned repo in such a way that it call be updated as the repo evolves with a simple git pull.

To deactivate the environment use:

(nemo-nowcast)$ conda deactivate

Coding Style

pre-commit The uncompromising Python code formatter

The NEMO_Nowcast package uses the Git pre-commit hooks managed by pre-commit to maintain consistent code style and and other aspects of code, docs, and repo QA.

To install the pre-commit hooks in a newly cloned repo, activate the conda development environment, and run pre-commit install:

$ cd NEMO_Nowcast
$ conda activate nemo-nowcast
(nemo-nowcast)$ pre-commit install

Note

You only need to install the hooks once immediately after you make a new clone of the NEMO_Nowcast repository and build your Development Environment.

Building the Documentation

Documentation Status

The documentation for the NEMO_Nowcast package is written in reStructuredText and converted to HTML using Sphinx. Creating a Development Environment as described above includes the installation of Sphinx. Building the documentation is driven by the docs/Makefile. With your nemo-nowcast development environment activated, use:

(nemo-nowcast)$ (cd docs && make clean html)

to do a clean build of the documentation. The output looks something like:

Removing everything under '_build'...
Running Sphinx v8.1.3
loading translations [en]... done
making output directory... done
loading intersphinx inventory 'python' from https://docs.python.org/3/objects.inv ...
loading intersphinx inventory 'gomssnowcast' from https://gomss-nowcast-system.readthedocs.io/en/latest/objects.inv ...
loading intersphinx inventory 'salishseanowcast' from https://salishsea-nowcast.readthedocs.io/en/latest/objects.inv ...
building [mo]: targets for 0 po files that are out of date
writing output...
building [html]: targets for 18 source files that are out of date
updating environment: [new config] 18 added, 0 changed, 0 removed
reading sources... [100%] nowcast_system/workers
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets...
copying static files...
Writing evaluated template result to /media/doug/warehouse/43ravens/projects/NEMO_Nowcast/docs/_build/html/_static/language_data.js
Writing evaluated template result to /media/doug/warehouse/43ravens/projects/NEMO_Nowcast/docs/_build/html/_static/basic.css
Writing evaluated template result to /media/doug/warehouse/43ravens/projects/NEMO_Nowcast/docs/_build/html/_static/documentation_options.js
Writing evaluated template result to /media/doug/warehouse/43ravens/projects/NEMO_Nowcast/docs/_build/html/_static/js/versions.js
copying static files: done
copying extra files...
copying extra files: done
copying assets: done
writing output... [100%] nowcast_system/workers
generating indices... genindex py-modindex done
highlighting module code... [100%] nemo_nowcast.workers.sleep
writing additional pages... search done
copying images... [100%] architecture/MessageBroker.png
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in _build/html.

The HTML rendering of the docs ends up in docs/_build/html/. You can open the index.html file in that directory tree in your browser to preview the results of the build.

If you have write access to the repository on GitHub, whenever you push changes to GitHub the documentation is automatically re-built and rendered at https://nemo-nowcast.readthedocs.io/en/latest/.

Running the Unit Tests

The test suite for the NEMO_Nowcast package is in NEMO_Nowcast/tests/. The pytest tool is used for test parametrization and as the test runner for the suite.

With your nemo-nowcast development environment activated, use:

(nemo-nowcast)$ cd NEMO_Nowcast/
(nemo-nowcast)$ pytest

to run the test suite. The output looks something like:

================================ test session starts =================================
platform linux -- Python 3.13.0, pytest-8.3.3, pluggy-1.5.0
Using --randomly-seed=3451085175
rootdir: /media/doug/warehouse/43ravens/projects/NEMO_Nowcast
configfile: pyproject.toml
plugins: randomly-3.15.0, cov-6.0.0, anyio-4.6.2.post1
collected 319 items

tests/test_manager.py ................................................................
...................                                                             [ 26%]
tests/workers/test_rotate_logs.py .................                             [ 31%]
tests/test_message_broker.py ...................                                [ 37%]
tests/workers/test_clear_checklist.py .........                                 [ 40%]
tests/test_log_aggregator.py .................                                  [ 45%]
tests/test_scheduler.py ...................                                     [ 51%]
tests/test_cli.py .................                                             [ 56%]
tests/workers/test_sleep.py .........                                           [ 59%]
tests/test_next_workers.py ......                                               [ 61%]
tests/test_worker.py .................................................................
...........................                                                     [ 90%]
tests/test_message.py ......                                                    [ 92%]
tests/workers/test_awaken.py ........                                           [ 94%]
tests/test_config.py .................                                          [100%]

================================ 319 passed in 18.66s ================================

You can monitor what lines of code the test suite exercises using the coverage.py and pytest-cov tools with the commands:

(nemo-nowcast)$ cd NEMO_Nowcast/
(nemo-nowcast)$ pytest --cov=./

The test coverage report will be displayed below the test suite run output.

Alternatively, you can use

(nemo-nowcast)$ pytest --cov=./ --cov-report html

to produce an HTML report that you can view in your browser by opening NEMO_Nowcast/htmlcov/index.html.

Continuous Integration

Pytest with Coverage Status Codecov Testing Coverage Report

The NEMO_Nowcast package unit test suite is run and a coverage report is generated whenever changes are pushed to GitHub. The results are visible on the repo actions page, from the green checkmarks beside commits on the repo commits page, or from the green checkmark to the left of the “Latest commit” message on the repo code overview page . The testing coverage report is uploaded to codecov.io

The GitHub Actions workflow configuration that defines the continuous integration tasks is in the .github/workflows/pytest-with-coverage.yaml file.

Version Control Repository

Git on GitHub

The NEMO_Nowcast package code and documentation source files are available as a Git repository at https://github.com/43ravens/NEMO_Nowcast.

Issue Tracker

Issue Tracker

Development tasks, bug reports, and enhancement ideas are recorded and managed in the issue tracker at https://github.com/43ravens/NEMO_Nowcast/issues

Licenses

Licensed under the Apache License, Version 2.0 Licensed under the BSD-3-Clause License

The NEMO_Nowcast framework code and documentation are copyright 2016 – present by Doug Latornell, 43ravens.

They are licensed under the Apache License, Version 2.0. https://www.apache.org/licenses/LICENSE-2.0 Please see the LICENSE file for details of the license.

The fileutils module from the boltons project is included in the NEMO_Nowcast package. It is copyright 2016 by Mahmoud Hashemi and used under the terms of the boltons BSD license.

Release Process

Releases Hatch project

Releases are done at Doug’s discretion when significant pieces of development work have been completed.

The release process steps are:

  1. Use hatch version release to bump the version from .devn to the next release version identifier

  2. Edit docs/CHANGES.rst to update the version identifier and replace unreleased with the release date

  3. Commit the version bump and change log update

  4. Create and annotated tag for the release with Git -> New Tag… in PyCharm or git tag -e -a vyy.n

  5. Push the version bump commit and tag to GitHub

  6. Use the GitHub web interface to create a release, editing the auto-generated release notes into sections:

    • Features

    • Bug Fixes

    • Documentation

    • Maintenance

    • Dependency Updates

  7. Use the GitHub Issues -> Milestones web interface to edit the release milestone:

    • Change the Due date to the release date

    • Delete the “when it’s ready” comment in the Description

  8. Use the GitHub Issues -> Milestones web interface to create a milestone for the next release:

    • Set the Title to the next release version, prepended with a v; e.g. v25.1

    • Set the Due date to the end of the year of the next release

    • Set the Description to something like v25.1 release - when it's ready :-)

    • Create the next release milestone

  9. Review the open issues, especially any that are associated with the milestone for the just released version, and update their milestone.

  10. Close the milestone for the just released version.

  11. Use hatch version minor,dev to bump the version for the next development cycle, or use hatch version major,minor,dev for a year rollover version bump

  12. Edit docs/CHANGES.rst to add a new section for the unreleased dev version

  13. Commit the version bump and change log update

  14. Push the version bump commit to GitHub