Fix Novel543Parser: clean up split-chapter headings, ads, and promo banners - #2942
Open
s4daharu wants to merge 1 commit into
Open
Fix Novel543Parser: clean up split-chapter headings, ads, and promo banners#2942s4daharu wants to merge 1 commit into
s4daharu wants to merge 1 commit into
Conversation
…anners - strip the '(1/2)' page part marker from chapter titles - keep only the first heading when a split chapter is merged from multiple part pages, instead of repeating it for every part - remove ad slots (div.adBlock / div.gadBlock), the '溫馨提示' site notice paragraphs, and the VIP membership promo banner Junk removal runs in preprocessRawDom (whole document, before image collection) so the VIP banner is not packed as an unused image, and is repeated in customRawDomToContentStep when packing. Covers novel543.com and twbook.cc.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
novel543.com (and twbook.cc) split long chapters across multiple part pages (
8096_1.html→8096_1_2.html), whichwalkPagesOfChaptermerges. This produces several content problems in the packed EPUB:<h1>第1章 xxx (1/2)</h1>followed by<h1>第1章 xxx (2/2)</h1>.div.adBlock,div.gadBlock), red "溫馨提示" site-notice paragraphs, and the VIP membership promo banner are kept as book content.Changes
extractTitleImpl: prefer the chapter heading, and strip the trailing(1/2)part marker from titles.customRawDomToContentStep: keep only the first heading of a merged split chapter and strip the part markers from it.removeNovel543Junk(root): removes ad slots, 溫馨提示 notice paragraphs, and the VIP banner.preprocessRawDomon the whole fetched document before image collection, so the promo banner is no longer downloaded/packed as an orphaned image.customRawDomToContentStepwhen packing (idempotent).Applies to both registered sites: novel543.com, twbook.cc.
Testing
unitTest/UtestNovel543Parser.js(7 QUnit tests: heading dedupe/marker strip, title extraction incl. book pages, continuation-link detection formoreChapterTextUrl, and whole-document junk removal viapreprocessRawDom) and registered it inTests.html.eslint --config eslint/.eslintrc.js plugin/js/parsers/Novel543Parser.jsis clean.