Assessing metadata completeness

[davemfish]

From our design document:

To meet the requirements of a data manager, our software should provide a command-line-interface for validating and reporting on metadata completeness for an entire project (or a data dump from an analyst), with user-friendly output that highlights missing metadata files, or values within files, vocabularies, etc.

A basic geometamaker describe will create a document with many blank attributes that could benefit from more user-input. It could be useful to provide a summary, or coverage report, that tells a user which attributes are still blank in a given metadata document, or for each document in a directory. This could incentivize metadata completeness.

We could also report on metadata coverage for an entire directory/project; reporting whether or not metadata exists for each "describe-able" file. Although I'm not sure how useful that is because a geometamaker describe -r dir/ will generate metadata for 100% of the describe-able files. So rather than asking for a coverage report, why wouldn't a user just ask to describe the whole directory?

[davemfish]

After talking with Emily P, it sounds like the most useful thing is not necessarily assessing "completeness" or "coverage", but validating documents against a specific schema, such as one defined for the DataHub. Here's an example, natcap/data.naturalcapitalproject.stanford.edu#89

This approach doesn't require any new geometamaker features. It's based on subclassing existing models and using jsonschema for validation.

[empavia]

@davemfish In addition to ensuring that YAML files contain the necessary elements for the Data Hub, there has been a discussion about providing descriptions for the fields. For instance, if we state that the Lineage field is required, is there a way to clarify what that element should contain?

I'm not sure if this issue is the appropriate place to add this. I was considering whether it could be developed alongside a schema or if it should be its own section in the documentation.
There have been some questions around what should be in each section, which currently only seems to be described here

Happy to place this in it's own issue. I am also happy to create the example YAML myself.

[davemfish]

@davemfish In addition to ensuring that YAML files contain the necessary elements for the Data Hub, there has been a discussion about providing descriptions for the fields. For instance, if we state that the Lineage field is required, is there a way to clarify what that element should contain?

Yeah, good question, @empavia . A couple things come to mind.

First, we have API documentation that includes brief descriptions of each field, for each different model in geometamaker.models. That is rendered directly from the source code. See the docstring under each field. If the descriptions we want to include are generally applicable to all metadata, then I think this approach is suitable. And we can certainly update any of the existing docstrings to be more descriptive.

On the other hand, if the description you want to include is more project-specific, like, "for this project, the lineage field should include xyz...", then let's look closer at the example in natcap/data.naturalcapitalproject.stanford.edu#89.

In that example, I sub-classed some of the geometamaker models and redefined some fields to add more constraints. But I neglected to add any docstrings for those fields. It looks like the resulting jsonschema file does contain descriptions based on docstrings, but they are absent from the fields that were re-defined in the new classes. So if you were creating your own classes like I did in that example, you could also write your own docstrings that would show up in the jsonschema file. And we could also talk about other options for rendering that documentation elsewhere.

[empavia]

@davemfish Thanks, Dave. The first links you provided for the docs are what I was envisioning and looking for. Couldn't find them when I was originally looking. Those should work for our purposes, and I can work on more descriptions where they may be necessary.