NEWS

===============
# TNRS 0.4.0
===============

## NEW FEATURES

* "TNRS_local" gains a "within" argument that confines the search to one or
  more taxa at family rank or above, for instance within = "Culicidae" or
  within = "Ixodida".  Against an all-life source such as the Catalogue of
  Life, a misspelt or missing name otherwise comes back as whatever it most
  resembles anywhere in the tree of life, a tick as a bird, a mosquito as a
  fungus; a submitted family prefix does not prevent this, since as in the
  web service the family is scored rather than used to narrow the search.
  Every source can be confined to a family; the Catalogue of Life and the
  Mammal Diversity Database record the ranks above it too.  The local name
  table now carries kingdom, phylum, class and order for this, and a synonym
  takes its classification from its accepted name, which the Catalogue of
  Life leaves blank on synonym rows.  A source built by an earlier version
  has to be rebuilt before it can be searched this way; a checklist
  registered with "TNRS_local_add_source" can supply the ranks as Darwin Core
  columns.

* New dataset "gbif_triage_sample": a few hundred raw mosquito and tick
  name strings from GBIF occurrence downloads, with the number of records
  carrying each and GBIF's own reading of it, chosen to cover every kind of
  agreement and disagreement with "TNRS_local".  The "TNRS_triage" help
  page walks through using it, and through doing the same with a download
  of your own.

* New function "TNRS_triage" compares two resolutions of the same names,
  from two sources, two services or two settings, and sorts each name into
  one of seven tiers from "both agree at species" to "both placed it and
  they contradict each other", with a suggested reading where one side is
  clearly the fuller.  On 16,600 raw GBIF names the last tier, the review
  queue, held well under 1% of records.

* New built-in source "phylacine": the PHYLACINE 1.2.1 mammal list (CC0),
  with the name each of its 5,999 species carries in IUCN 2016-3,
  EltonTraits 1.0 and PHYLACINE 1.0 and 1.1 (553 differ) and the 48 names
  from those lists it rejects.  It is a crosswalk to PHYLACINE's own
  binomials, for joining names to its traits and phylogenies, not a
  synonymy of mammals; "mdd" remains the source for that.  Orders are
  carried, so "within" and a leading order both work with it.  Two
  megabytes to download.

* "TNRS_local" now recognises a name that opens with an order, class,
  phylum or kingdom the source knows, such as "Carnivora Vulpes vulpes" or
  "Plantae Oenanthe": the prefix is taken off and the search for that name
  confined to it, as "within" does for a whole call.  Before, and in the web
  service still, the prefix was read as the genus and matched to whatever
  resembled it.  A leading family the source knows is treated as a family
  prefix whatever its ending, so a zoological family is recognised without
  setting "nomenclature".

* "TNRS_local" now recognises an abbreviated author.  A surname written
  "Theob.", "Edw." or "Dön" is expanded against the authority it is being
  compared with when it is a prefix of exactly one surname there, so
  "Edw." against "(Edwards, 1914)" scores as the same author rather than
  0.16.  The web service intended an abbreviation table for this and ships
  with the lookup disabled; this needs no table.

* "TNRS_local" sets a new warning, "[Author]" (bit 16 of "Warnings"), when a
  name matched only approximately and the authority it was submitted with
  contradicts the authority of the match.  Against a large source, a name
  the source lacks is often matched confidently to a near neighbour, and the
  author is what gives it away: "Ixodes barkeri Barker 2019" lands on
  "Ixodes bakeri Arthur & Clifford, 1961" at 0.8.  The web service's four
  flags keep their values.

* "TNRS_local" now applies an accuracy threshold by default, 0.53, the web
  service's default, and applies it by the web service's rule: a match is
  dropped only when its overall score and every component score fall below
  the threshold.  Earlier versions applied "accuracy" to the overall score
  alone and applied nothing unless asked, so a name whose only candidate
  was a poor fuzzy match to an unrelated genus came back with that match at
  a score of 0.2 or 0.3.  Pass accuracy = NULL to keep every match.

* "TNRS_local_build" allows an hour for each download rather than R's default
  60 seconds, which the 488 MB Catalogue of Life archive could not meet.

* Names can now be resolved offline, against a locally cached copy of the
  taxonomic sources, using the same algorithm as the web service.  Four new
  functions:

  - "TNRS_local_build" downloads and prepares the taxonomic sources.  This is a
    one-off step of about 116 MB and a few minutes for the default source,
    leaving about 120 MB on disk; afterwards no internet connection is needed.
    The downloaded archive is deleted once the data has been prepared, since
    it is not needed for matching.  Pass keep_archive = TRUE to keep it, which
    roughly doubles the space used but means a rebuild needs no second
    download; either way its size and checksum stay on record.
  - "TNRS_local" resolves names against that local copy.  It returns the same
    columns as "TNRS", so existing code that reads the output keeps working.
  - "TNRS_local_status" lists every source the package can use: which are
    built for offline use, which version each is and their DOIs so that local
    results can be cited, and which are merely available, with what
    downloading them would cost.
  - "TNRS_local_remove" deletes the cache.
  - "TNRS_local_citations" gives everything a local result should be cited
    with: the TNRS publication, this package, and each source resolved
    against, at the version that was actually built rather than the one
    currently published.  Optionally written out as BibTeX.

* "TNRS_local_add_source" registers a checklist you supply as a local source,
  on the same footing as the ones this package downloads.  Use it for an
  authority the TNRS does not distribute or for an in-house list.  Only the name
  itself is required: anything the checklist does not carry, such as the genus
  and epithet, is derived from the name using the same parser the resolver uses.
  Columns are looked for under their Darwin Core names by default, and named
  explicitly with the "columns" argument otherwise. A source added this way can
  be used on its own or blended with "wfo" and "wcvp", with "Source_conflict"
  marking the names they disagree about.

  The Cactaceae checklist at caryophyllales.org, which the web service offers
  as source "cact", is not distributed publicly, so this package cannot
  download it.  If you obtain a copy you can register it with
  "TNRS_local_add_source".

* "TNRS_cdm_to_dwc" converts a taxonomic export from the EDIT platform into
  Darwin Core, ready to register.  That is the form the Caryophyllales.org
  checklists are supplied in, Cactaceae among them: one sheet of accepted taxa
  and one of synonyms.  It stacks the two, folds the platform's own rank
  spellings onto botanical ones, and reads each name's status off the tree, so
  that names the authors could not place are not promoted to accepted taxa.

* The local functions are additive: "TNRS" and the other API functions are
  unchanged, and continue to query the web service.

* See vignette("TNRS_offline") for the workflow and for how local results
  differ from the web service.

* Animal names can now be resolved.  Two sources cover them:

  - "mdd", the Mammal Diversity Database, 6,836 accepted species and about
    56,000 synonyms, 68 MB to download.  Published under CC BY 4.0 with a DOI
    per release.
  - "col", the Catalogue of Life, 5.4 million names covering all life.  It is
    a 488 MB download and by far the largest source offered, so it is worth
    building only if you need its breadth. Note, however, this breadth allows
    better handling of cross-code homonyms.

  Neither is built by default.  "TNRS_local_status" lists them, and
  "TNRS_local_build" fetches them on request like any other source.

* Sources now record which nomenclatural code they follow, and
  "TNRS_local_status" reports it.  It decides how a family prefix in a
  submitted name is read: botanical families end in -aceae, zoological ones in
  -idae, so "Felidae Panthera leo" resolves against an animal source where it
  would not against a plant one.  "TNRS_local" takes the code from the sources
  requested, and the new "nomenclature" argument overrides that.  Asking for a
  plant and an animal source together resolves under both, which is what makes
  a name shared between the two codes visible rather than silently resolved
  under one of them.

* "TNRS_local_add_source" gained a "nomenclature" argument, so a checklist you
  supply can say which code it follows.

## NOTES ON THE LOCAL VERSION

* "TNRS_local" defaults to a single source ("wfo"), where "TNRS" consults
  "wcvp" and "wfo" together.  One source means the answer comes from one
  authority.  Pass sources = c("wcvp", "wfo") for the web service's behaviour;
  where the two disagree, the new "Source_conflict" column marks the name.

* "TNRS_local_build" builds "wfo" alone by default, matching "TNRS_local", so
  the one-off setup is about 116 MB rather than 201 MB.  Build "wcvp" as well
  if you want both.

* Asking "TNRS_local" for a source that has not been built reports what is
  missing and the exact "TNRS_local_build" call that would fix it, rather than
  starting a large download unannounced.  The new "build_missing" argument
  controls this: it defaults to interactive(), which asks first and reports the
  download and disk cost, and can be set to TRUE so that an unattended script
  builds what it needs.

* The local backbone is downloaded from the publishers and is usually newer
  than the copy the web service is running, so some names legitimately resolve
  differently.  Report the versions from "TNRS_local_status" alongside results.

* The "accuracy" argument filters on the overall score. The web service
  discards a match only when every component score is below the threshold, so
  it is more permissive than its own documentation describes.

## Bug Fixes

* Names are now matched back to the rows they were submitted on.  Each distinct
  name is sent once, under an identifier of this package's own, and the answer
  is rebuilt against the request.  This fixes three reported problems at once:

  - A blank name is no longer dropped by the service, which used to shift every
    later name onto the identifier before it, silently and with no error.  That
    was the most damaging of the three, since the identifier column exists to
    join results back to the caller's data (#16).
  - A name given more than once now returns a row per identifier, rather than
    one row whose ID was the identifiers pasted together, "test-2,test-1" (#15).
  - A name that is missing, empty or only whitespace is returned unmatched
    rather than disappearing from the output (#14).

  Sending each distinct name once also means a list with repeats is smaller to
  resolve.

* The package now carries a CITATION file, so citation("TNRS") gives the
  software with its DOI and the TNRS publication together, rather than a
  citation generated from the description.

* "TNRS_status" reports whether the web service is reachable, how quickly it
  answered and which version it is running, so that a failing call can be told
  apart from a failing connection without guesswork (#20).

* A request that fails now says why.  Every failure used to be reported as
  "There appears to be a problem reaching the API", which discarded the reason,
  so a timeout, a refused connection, a name that would not resolve and a
  certificate problem were indistinguishable.  Each is now named, along with
  what to do about it.

* "TNRS" and the other API functions take a "timeout" argument, in seconds.  A
  large batch of names legitimately takes minutes, and there was previously no
  way to wait longer than the default.

* Submitted names are checked before anything is sent, so a malformed request
  is explained rather than answered with "HTTP Status 400" (#13, #19).  The
  checks cover the number of columns, the type supplied, empty input, and
  missing or duplicated identifiers; the last of these would silently corrupt
  any join back to the caller's data.  "TNRS" and "TNRS_local" share the checks
  and report the same problems in the same words.

* A family prefix that agreed with the matched name was penalised as though it
  had been wrong.  The submitted family was left unscored while still counting
  as a component of the name, so the score was divided by a part that
  contributed nothing: "Fagaceae Quercus alba" scored 0.5996 where the web
  service gives 1.  A family the backbone does not recognise still scores
  0.5996, as it does there.

===============
# TNRS 0.3.8
===============

## NEW FEATURES

## Bug Fixes

* Fixed the retry loop in "TNRS_robust", which used the wrong comparison and so
  never executed.  Suspicious results are now actually re-queried.
* "TNRS_robust" now tracks suspicious results by name ID rather than by row, so
  that all matches for a name are re-done together when matches = "all", and
  re-done names no longer appear twice in the output.  Results are returned in
  the order the names were submitted, and names that remain suspicious after the
  final attempt are retained rather than dropped.
* "TNRS_robust" now passes additional arguments (e.g. url) through to retries.
* "TNRS_base" no longer errors when the API is unavailable; it returns NULL, as
  the other functions do.
* "TNRS" now warns, rather than silently dropping names, when one batch of a
  multi-batch query fails.
* Argument checking for "accuracy" now uses inherits() rather than class(),
  matching the check already used in "TNRS_base".

===============
# TNRS 0.3.7
===============

## NEW FEATURES

* Added experimental new function "TNRS_robust" which functions identically to "TNRS", but checks for some very rare errors that can occur sporadically.

## Bug Fixes

* Added code to remove pipes from species names, as these cause errors.

===============
# TNRS 0.3.6
===============

## NEW FEATURES

* A new functions, TNRS_synonyms, has been added. This returns all synonyms for a single species from a single source.

* Added tests for TNRS_synonyms

## Bug Fixes


===============
# TNRS 0.3.5
===============

## NEW FEATURES

* A new source has been added, "cact", which contains only species within the Cactaceae.

## Bug Fixes

* Updated tests per new API options
* removed documentation for '...' from TNRS_core, as we aren't using this anymore.
* Updating to reflect new sources
* Fixed bug in TNRS.R where additional parameters weren't being passed to internal functions
* Updated documentation

===============
# TNRS 0.3.4
===============

## NEW FEATURES

* Updated default settings per a pending TNRS API update.

## Bug Fixes

===============
# TNRS 0.3.3
===============

## NEW FEATURES

## Bug Fixes

* Updated to fix a bug that occurs with TNRS_metadata if there are issues making API connections, but not internet connections

===============
# TNRS 0.3.2
===============

## NEW FEATURES

## Bug Fixes
* Fixed problems with testing causing CRAN checks to fail


===============
# TNRS 0.3.1
===============

## NEW FEATURES

* Added progress bar when downloading multiple  batches of records
* Added an option to skip checks for internet when running functions
* TNRS "..._score" columns now returned as numeric (rather than character)

## Bug Fixes

* Removed a few non-ascii character in the example file which were causing errors in vcr
* Used new "skip_internet_check" option to fix bug with vcr when hitting the internet multiple times (once to check the connection, again to run the query)

## Misc.

* Ran styler to make the code a bit neater

===============
# TNRS 0.3.0
===============

## NEW FEATURES

* Revised code to make maintanence and testing easier
* Testing now makes use of the vcr package where possible
* Added the option to specify a threshold for match scores

## Bug Fixes

* Package should no longer produce errors in the event of API connection issues (e.g. server problems).


===============
# TNRS 0.2.0
===============

## NEW FEATURES

* Added Spanish language vignette
* Multiple sources now specified by character vector.
* Updated taxonomic sources and classifications.
* Added checks of inputs.

## Bug fixes

* Fixed error in naming metadata elements
* A server upgrade rendered RCurl incompatible with our api, so we switched to httr
* Modified functions to fail gracefully if the internet or API are unavailabile
* tests will now be skipped if offline

===============
# TNRS 0.1.0
===============

## NEW FEATURES

* released to CRAN