docs: correct the database connector page and document query_timeout - #509
Open
marevol wants to merge 1 commit into
Open
docs: correct the database connector page and document query_timeout#509marevol wants to merge 1 commit into
marevol wants to merge 1 commit into
Conversation
The 15.9 database connector page described behaviour the connector does
not have, and left out the parts most likely to trip an administrator up.
Applied to all seven languages.
Corrections:
- Installation pointed at Maven Central and at WEB-INF/lib. The plugin is
distributed through maven.codelibs.org, and the admin console installs
into WEB-INF/plugin.
- The JDBC driver section did not say the driver is a separate download,
nor that it has to reach the crawler process classpath. Both
WEB-INF/lib and WEB-INF/env/crawler/lib work; nothing else does.
- "BLOB, CLOB, NCLOB, byte array and binary stream columns are
automatically passed through content extraction" was wrong. Only the
binary forms are extracted; CLOB, NCLOB and character streams are read
as text, and the MIME type hints do not apply to them. Whether a BLOB
arrives as java.sql.Blob or as a byte array is decided by the driver.
- The url=url recipe was presented without its trap: with no column
labelled url, the parameter of the same name wins and the document URL
becomes the JDBC connection URL.
- Scripts see the entire data store parameter map, credentials included,
not only the result columns. Column label casing differs between
databases, and a label that does not resolve leaves the field unset
rather than raising an error.
- "Use environment variables" and "use the encryption features" had no
instructions. Both exist: FESS_ENV_ variables expand as ${...} inside
parameters, and a parameter matching app.encrypt.property.pattern is
encrypted at rest.
- Troubleshooting listed symptoms the connector no longer produces. The
messages now identify which step failed, and sections for missing
parameters, per-row failures and missing search results were added.
- fetch_size did not say that MIN_VALUE is a MySQL idiom that other
drivers reject.
- The incremental crawling section told the reader to narrow the SQL by
hand without saying what that costs. When a crawl finishes, Fess
deletes the documents of that data store configuration that were not
part of the crawl that just ran, so a filtered query leaves only the
matching rows in the index. The section now says so and points at
delete_old_docs=false.
New parameter:
- query_timeout, with a note that stopping a job cannot interrupt a query
blocked inside the driver.
Structure was verified across all seven files: identical note, warning
and literal block counts, list-table rows with a consistent cell count,
and section underlines checked with east-asian width taken into account.
marevol
force-pushed
the
docs/ds-database-accuracy
branch
from
August 23, 2026 08:08
5dfc565 to
f9d6394
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The 15.9 database connector page described behaviour the connector does not have,
and left out the parts most likely to trip an administrator up. This corrects it
and documents
query_timeout, added in thefess-ds-dbstack(codelibs/fess-ds-db#19 … #28).
All seven languages:
ja,en,de,es,fr,ko,zh-cn. 15.9 only.Corrections
WEB-INF/libmaven.codelibs.org, intoWEB-INF/plugin- where the admin console installsWEB-INF/liborWEB-INF/env/crawler/libjava.sql.Blobor as a byte array is the driver's choiceurl=urlpresented as a recipeurlcolumn makes the document URL the JDBC connection URLFESS_ENV_variables expand as${...}(crawler.data.env.param.key.pattern), and a parameter matchingapp.encrypt.property.patternis encrypted at rest. Plus a note that DEBUG logging masks theseClassNotFoundException/Connection refused/SQLExceptionfetch_size: "set toMIN_VALUEfor MySQL streaming"delete_old_docs=falseThe
delete_old_docsbehaviour is inDataIndexHelper#deleteOldDocs(): it returnsearly only when the parameter is literally
false, so a partial crawl sweeps therest away by default.
New parameter
query_timeout, with a note that stopping a job cannot interrupt a queryblocked inside the driver - the stop is only checked between rows.
Verification
.. note::(7),.. warning::(4), literal blocks (17) andlist-tablerows (18); sectionunderlines checked with east-asian width taken into account.
and no file still mentions
repo1.maven.org,Maven Central,mysql-connector-j-8,ClassNotFoundException,No suitable driverorConnection refused.Unknown interpreted text role "doc"from the Sphinx-only:doc:role in the"See Also" list.
conf/conf.pyfails to load under Sphinx 9.1with a
unicodeescapeerror at line 228, which is unrelated to this change.Note on scope
An earlier revision of this branch also documented incremental crawling through
${last_crawl_time}. That connector change (codelibs/fess-ds-db#29) was notmerged, so the parameters and the section describing them have been removed and
the manual-filter section restored - now carrying the
delete_old_docswarningit was missing.
One thing for the reviewer
The Spanish page carries no diacritics (
Descripcion,Metodo,contrasena). That is pre-existing and page-wide; #511, stacked on this branch,fixes it across the seven affected Spanish pages.