[#295] - Search mask, facets and autocomplete for the TEI text edition - #316
Open
sebhofmann wants to merge 3 commits into
Open
[#295] - Search mask, facets and autocomplete for the TEI text edition#316sebhofmann wants to merge 3 commits into
sebhofmann wants to merge 3 commits into
Conversation
CMOTEIFileIndexAccumulator reads the TEI of a text edition and states its content in solr fields of the document of the file: the text of the edition, the lyrics, the apparatus and everything the teiHeader holds about the piece, its poets, its metre and its witnesses. The schema gets those fields, a stemmed copy per project language and an untouched copy for facets and drop downs. CMOTEIFileIndexStrategy keeps those files out of the extracting update handler of solr. A file whose content is sent has to carry every field of its document as a literal parameter of the url, which the text of an edition blows beyond the header limit of the servlet container, so nothing of it was indexed at all. The extraction is not needed either, the accumulator reads the TEI itself and states its text structured and per element.
…text edition
The edition search gets a mask of its own, "Textedition": it searches the
fields of the TEI over a join from the document of the file to the text
edition it belongs to, in the full text, the apparatus, the lyrics, the
incipit, the metrical licences, the poets and composers and the witnesses.
The sidebar of that search gets facets for makam, usul and music genre, which
stand at the expression two steps away over the catalogue, and for text form,
metre and rhyme scheme, which stand at the TEI. Each of them needs a join, so
they are computed by the json facet api: the buckets are counted in the domain
of the joined documents and a nested facet joins back to the hits to state the
number a reader expects. Every filter of the search travels once as its own
parameter, the facets point at it as {"param": "ffN"}, which keeps the url
short. What the sidebar offers as a facet is left out of the mask, those values
are categories and codes and are better picked than typed.
Text fields whose values nobody knows by heart suggest them while the user
types, the transcribed names of the poets and composers, the sigla of the
sources and the type of a reading. A text field is matched by its analyzer, so
a name of the corpus is found without its diacritics as well, and the values
are read out of the highlighting of solr, which returns only the matching ones
of a multi valued field. A string field is matched as it stands, the case
ignored, by a facet which only keeps the values containing what was typed.
Also repairs some characters of the german messages, which were saved into the
ASCII file as single bytes and were shown as replacement characters since.
The url of a single hit is built from the parameters of the search, and the ones which only feed the counters of the facets belong to the sidebar: the json facet, the switch of the classic facets and every filter the counters point at. The last ones were still carried along, so the browse view of a hit got a handful of ff0, ff1 parameters it has no use for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #295, works on #296
Facettierte Suche in der TEI-Textedition, dazu die Indexierung der TEI-Dateien und ein Autocomplete für die Felder, deren Werte man nicht auswendig tippt.
Indexierung (
[#296])CMOTEIFileIndexAccumulatorliest das TEI einer Textedition und schreibt seinen Inhalt in Solr-Felder des Dateidokuments: Text, Liedtext, Apparat und alles, was derteiHeaderüber Stück, Dichter, Versmaß und Textzeugen hält.schema.xmlbekommt diese Felder, je eine gestemmte Kopie pro Projektsprache (tei.lang.de.*usw.) und eine unveränderte Kopie für Facetten und Dropdowns (tei.facet.*), beide per Wildcard, damit ein neues TEI-Feld das Schema nicht anfasst.CMOTEIFileIndexStrategyhält die TEI-Dateien aus/update/extractheraus. Wird der Inhalt einer Datei gesendet, muss jedes Feld ihres Dokuments alsliteral.*-Parameter in der URL reisen, was der Text einer Edition über das Header-Limit des Servlet-Containers treibt: es wurde bisher gar nichts davon indexiert. Die Extraktion ist auch nicht nötig, der Accumulator liest das TEI selbst und stellt den Text strukturiert und pro Element bereit.Die Datei entscheidet sich am Wurzelelement, gelesen per StAX ohne DTD und externe Entities. Die Editions-TEIs sind UTF-16, die Kodierung kommt aus der Byte Order Mark.
Suchmaske und Facetten (
[#295]){"param": "ffN"}. Ein Local-Param-Verweis ({!v=$ff0}) darf hier nicht stehen, das ist eine textuelle Substitution, bei der ein Wert am ersten Leerzeichen endet.fq, alle müssen erfüllt sein. Deshalb dünnen die Facetten sich gegenseitig aus.Autocomplete (
[#295])Felder, deren Werte man nicht auswendig kennt, schlagen sie beim Tippen vor: die transkribierten Namen der Dichter und Komponisten, die Siglen der Quellen und die Art der Lesart.
hafizfindet alsoHâfız-ı Şîrâzî. Die Werte kommen aus dem Highlighting von Solr, das von einem mehrwertigen Feld nur die passenden Werte liefert und nicht die Mitbewohner desselben Dokuments. Gefragt werden Wort und Wortanfang, weil die Textfelder gestemmt sind, eine Präfix-Anfrage aber nicht analysiert wird.facet.containsbehält nur die Werte, die das Getippte enthalten, also auch mitten im Wert (tifindetaddition).combobox/listbox/optionmitaria-expandedundaria-activedescendantverdrahtet.Sonstiges
Ein paar Zeichen in
messages_de.propertieswaren als einzelne Bytes in die ASCII-Datei geschrieben und wurden seitdem als Ersetzungszeichen angezeigt, die sind mitrepariert.Getestet
mvn clean installgrün, 47 Tests, davon 18 für den Accumulator und 7 für die Index-Strategy.hafiz→Hâfız-ı Şîrâzî(Klick und Tastatur),tmk→TMKlii,ti→addition,ozbek→ vier Namensvarianten,dedeals Komponist →Dede Efendimit 504 Treffern,iskundistanbin der Quellenmaske. Keine Konsolenfehler.