Scorer for NIST OpenFAD 2022 Challenge Activity Classification- (AC) and Temporal Activity Detection Task (TAD). For official evaluation information, evaluation plan as well as documentation please visit the OpenFAD Website.
NOTE: Please ensure to call pip from your local python interpreter in order to install the scorer package and dependecies in your local python environment by using
python -m pipinstead ofpip.
- python 3.6 or later
- setuptools
python -m pip install setuptoolsNOTE: Running install will also install dependent python packages !
python -m pip install .After running the install above there should be a fad-scorer CLI tool in your
python environment.
# Install dependencies
python -m pip install .[tests]
# Run unit/integration tests
python -m pytest# General Help
fad-scorer -h
# Command Specific Help
fad-scorer score-ac -h
# Score Activity Classification Task
fad-scorer score-ac -r tests/testdata/ac_ref_ec_1.csv -y tests/testdata/ac_hyp_ec_1.csv
# - using verbose flag for additional info
fad-scorer -v score-ac -r tests/testdata/ac_ref_ec_1.csv -y tests/testdata/ac_hyp_ec_1.csv
# Score Temporal Activity Detection Task
fad-scorer score-tad -r tests/testdata/tad_ref_smoothcurve.csv -y tests/testdata/tad_hyp_smoothcurve.csv
# Plot Results
fad-scorer plot-results -f tmp/scoring_results.h5Documentation uses the pdoc package.
# Install dependencies
python -m pip install .[documentation]
# generate docs and provide access via http server
pdoc -d numpy ./fad21 -nThe scorer uses the HDF5 standard-format for storing results on file but extracts them to CSV by default so no extra tools are needed to inspect system- and activity-level scores.
However, there are several CLI- and UI-based tools available if you wish to
inspect the hdf5 files directly. For example h5dump, h5ls:
- Ubunut/Debian platforms:
sudo apt-get install h5utils - OsX:
brew install hdf5
- 2021-2022 Lukas Diduch (lukas.diduch@nist.gov)
- Jeffrey Byrne (jeff@visym.com)
- Jonathan Fiscus (jonathan.fiscus@nist.gov)
- YooYoung Lee (yooyoung.lee@nist.gov)
Full details can be found at: http://nist.gov/data/license.cfm
NIST-developed software is provided by NIST as a public service. You may use,
copy, and distribute copies of the software in any medium, provided that you
keep intact this entire notice. You may improve, modify, and create derivative
works of the software or any portion of the software, and you may copy and
distribute such modifications or works. Modified works should carry a notice
stating that you changed the software and should note the date and nature of
any such change. Please explicitly acknowledge the National Institute of
Standards and Technology as the source of the software.
NIST-developed software is expressly provided "AS IS." NIST MAKES NO WARRANTY
OF ANY KIND, EXPRESS, IMPLIED, IN FACT, OR ARISING BY OPERATION OF LAW,
INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND DATA ACCURACY. NIST NEITHER
REPRESENTS NOR WARRANTS THAT THE OPERATION OF THE SOFTWARE WILL BE
UNINTERRUPTED OR ERROR-FREE, OR THAT ANY DEFECTS WILL BE CORRECTED. NIST DOES
NOT WARRANT OR MAKE ANY REPRESENTATIONS REGARDING THE USE OF THE SOFTWARE OR
THE RESULTS THEREOF, INCLUDING BUT NOT LIMITED TO THE CORRECTNESS, ACCURACY,
RELIABILITY, OR USEFULNESS OF THE SOFTWARE.
You are solely responsible for determining the appropriateness of using and
distributing the software and you assume all risks associated with its use,
including but not limited to the risks and costs of program errors, compliance
with applicable laws, damage to or loss of data, programs or equipment, and the
unavailability or interruption of operation. This software is not intended to
be used in any situation where a failure could cause risk of injury or damage
to property. The software developed by NIST employees is not subject to
copyright protection within the United States.