NEMO_Nowcast Package Development

Licensed under the Apache License, Version 2.0 Licensed under the BSD-3-Clause License Python Version Git on GitHub The uncompromising Python code formatter Documentation Status GitHub Workflow Status Codecov Testing Coverage Report Issue Tracker Install with conda

Python Versions

Python Version

The SalishSeaNowcast package is developed and tested using Python 3.8 or later. The package uses some Python language features that are not available in versions prior to 3.6, in particular:

Getting the Code

Git on GitHub

Clone the code and documentation repository from GitHub with:

$ git clone git@github.com:43ravens/NEMO_Nowcast.git

or

$ git clone https://github.com/43ravens/NEMO_Nowcast.git

if you don’t have ssh key authentication set up on GitHub.

Development Environment

Setting up an isolated development environment using Conda is recommended. Assuming that you have Anaconda Python Distribution or Miniconda3 installed, you can create and activate an environment called nemo-nowcast that will have all of the Python packages necessary for development, testing, and building the documentation with the commands:

$ cd  NEMO_Nowcast
$ conda env create -f environment-dev.yaml
$ source activate nemo-nowcast
(nemo-nowcast)$ pip install --editable nemo_nowcast

The --editable option in the pip install command above installs the NEMO_Nowcast package from the cloned repo via symlinks so that the installed nemo-nowcast package will be automatically updated as the repo evolves.

To deactivate the environment use:

(nemo-nowcast)$ source deactivate

Coding Style

The uncompromising Python code formatter

The NEMO_Nowcast package uses the black code formatting tool to maintain a coding style that is very close to PEP 8.

black is installed as part of the Development Environment setup.

To run black on the entire code-base use:

$ cd NEMO_Nowcast
$ conda activate nemo-nowcast
(nemo-nowcast)$ black ./

in the repository root directory. The output looks something like:

reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/nemo_nowcast/workers/clear_checklist.py
reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/nemo_nowcast/config.py
reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/tests/workers/test_clear_checklist.py
reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/tests/test_config.py
reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/nemo_nowcast/worker.py
reformatted /media/doug/warehouse/MEOPAR/NEMO_Nowcast/tests/test_worker.py
All done! ✨ 🍰 ✨
6 files reformatted, 26 files left unchanged.

Building the Documentation

Documentation Status

The documentation for the NEMO_Nowcast package is written in reStructuredText and converted to HTML using Sphinx. Creating a Development Environment as described above includes the installation of Sphinx. Building the documentation is driven by the docs/Makefile. With your nemo-nowcast development environment activated, use:

(nemo-nowcast)$ (cd docs && make clean html)

to do a clean build of the documentation. The output looks something like:

rm -rf _build/*
sphinx-build -b html -d _build/doctrees   . _build/html
Running Sphinx v1.4.1
making output directory...
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 2 source files that are out of date
updating environment: 2 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

Build finished. The HTML pages are in _build/html.

The HTML rendering of the docs ends up in docs/_build/html/. You can open the index.html file in that directory tree in your browser to preview the results of the build.

If you have write access to the repository on GitHub, whenever you push changes to GitHub the documentation is automatically re-built and rendered at http://nemo-nowcast.readthedocs.io/en/latest/.

Running the Unit Tests

The test suite for the NEMO_Nowcast package is in NEMO_Nowcast/tests/. The pytest tool is used for test parametrization and as the test runner for the suite.

With your nemo-nowcast development environment activated, use:

(nemo-nowcast)$ cd NEMO_Nowcast/
(nemo-nowcast)$ pytest

to run the test suite. The output looks something like:

============================ test session starts ============================
platform linux -- Python 3.6.7, pytest-4.0.1, py-1.7.0, pluggy-0.8.1
rootdir: /media/doug/warehouse/43ravens/projects/gomss-nowcast/NEMO_Nowcast, inifile:
collected 300 items

tests/test_cli.py .................                                                       [  5%]
tests/test_config.py .............                                                        [ 10%]
tests/test_log_aggregator.py .................                                            [ 15%]
tests/test_manager.py ...............................................................
...................                                                                       [ 43%]
tests/test_message.py ......                                                              [ 45%]
tests/test_message_broker.py ...................                                          [ 51%]
tests/test_next_workers.py ......                                                         [ 53%]
tests/test_scheduler.py ...................                                               [ 59%]
tests/test_worker.py ................................................................
..............                                                                            [ 85%]
tests/workers/test_awaken.py ........                                                     [ 88%]
tests/workers/test_clear_checklist.py .........                                           [ 91%]
tests/workers/test_rotate_logs.py .................                                       [ 97%]
tests/workers/test_sleep.py .........                                                     [100%]

========================= 300 passed in 16.77 seconds =========================

You can monitor what lines of code the test suite exercises using the coverage.py and pytest-cov tools with the commands:

(nemo-nowcast)$ cd NEMO_Nowcast/
(nemo-nowcast)$ pytest --cov=./

The test coverage report will be displayed below the test suite run output.

Alternatively, you can use

(nemo-nowcast)$ pytest --cov=./ --cov-report html

to produce an HTML report that you can view in your browser by opening NEMO_Nowcast/htmlcov/index.html.

Continuous Integration

GitHub Workflow Status Codecov Testing Coverage Report

The NEMO_Nowcast package unit test suite is run and a coverage report is generated whenever changes are pushed to GitHub. The results are visible on the repo actions page, from the green checkmarks beside commits on the repo commits page, or from the green checkmark to the left of the “Latest commit” message on the repo code overview page . The testing coverage report is uploaded to codecov.io

The GitHub Actions workflow configuration that defines the continuous integration tasks is in the .github/workflows/pytest-coverage.yaml file.

Version Control Repository

Git on GitHub

The NEMO_Nowcast package code and documentation source files are available as a Git repository at https://github.com/43ravens/NEMO_Nowcast.

Issue Tracker

Issue Tracker

Development tasks, bug reports, and enhancement ideas are recorded and managed in the issue tracker at https://github.com/43ravens/NEMO_Nowcast/issues

Release Packages

Install with conda

Versioned releases of the NEMO_Nowcast package are available as Conda packages on Anaconda.org.

The latest release package can be installed with:

$ conda install -c gomss-nowcast nemo_nowcast

Licenses

Licensed under the Apache License, Version 2.0 Licensed under the BSD-3-Clause License

The NEMO_Nowcast framework code and documentation are copyright 2016-2021 by Doug Latornell, 43ravens.

They are licensed under the Apache License, Version 2.0. http://www.apache.org/licenses/LICENSE-2.0 Please see the LICENSE file for details of the license.

The fileutils module from the boltons project is included in the NEMO_Nowcast package. It is copyright 2016 by Mahmoud Hashemi and used under the terms of the boltons BSD license.