Concept - Select caches in vvMixinInit

[Peter-J-Freeman]

Testing a concept SeqFetcher.fetch_seq() LRU cache produced a modest improvement in average runtime during benchmarking, but overall execution times were more consistent. The benchmark is also likely to underestimate the benefit in production because VariantValidator commonly exhibits sequence locality.

During gap mapping of VCF files, the converter repeatedly processes the same transcript/genomic accession combinations while generating the different HGVS representations (genomic, transcript, protein and related formats). This results in repeated requests for the same sequence regions within tight processing loops, allowing an LRU cache to eliminate many duplicate sequence retrievals.

Similar locality is expected in routine clinical and research workflows. Variants are commonly analysed in batches originating from the same sequencing sample, gene panel or disease cohort, where multiple variants frequently occur within the same gene or transcript. Successive variants therefore repeatedly access the same transcript and genomic sequence, allowing cached sequence data to be reused across validations. Although the average benchmark improvement is modest, the practical benefit is expected to be greater for real-world batch processing workloads and for gap mapping, where repeated access to the same sequence regions is common.

The same principle may also be applicable to the HGVS Data Provider (HDP). Rather than introducing caching throughout the codebase, it may be possible to wrap the expensive external resources when they are created in vvMixinInit, allowing repeated lookups to be cached transparently while leaving the remainder of VariantValidator unchanged and not relying on cache all implementation via vvhgvs data providers

Current architecture

                 Validator
                     │
         ┌───────────┴───────────┐
         │                       │
    SeqFetcher                 HDP
         │                       │
    fetch_seq()         get_tx_exons()
                        get_tx_identity_info()
                        get_tx_mapping_options()
                        ...
                              │
                          Database

Proposed architecture

                      Validator
                          │
          ┌───────────────┴───────────────┐
          │                               │
 CachedSeqFetcher                  CachedHDP
          │                               │
     SeqFetcher                         HDP
          │                               │
     fetch_seq()                 get_tx_exons()
                                 get_tx_identity_info()
                                 get_tx_mapping_options()
                                 ...
                                       │
                                   Database

Validator initialisation

Current:

self.sf = SeqFetcher(...)
self.hdp = hdp_connect(...)

Proposed:

self.sf = CachedSeqFetcher(
    SeqFetcher(...)
)

self.hdp = CachedHDP(
    hdp_connect(...)
)

No downstream code would require modification, as the Validator would continue to expose the same self.sf and self.hdp interfaces.

The wrapper classes would transparently cache repeated lookups while delegating to the underlying implementations.

class CachedSeqFetcher:

    def __init__(self, seqfetcher):
        self._seqfetcher = seqfetcher

    @lru_cache(maxsize=...)
    def fetch_seq(self, *args):
        return self._seqfetcher.fetch_seq(*args)

class CachedHDP:

    def __init__(self, hdp):
        self._hdp = hdp

    @lru_cache(maxsize=...)
    def get_tx_exons(self, *args):
        return self._hdp.get_tx_exons(*args)

    @lru_cache(maxsize=...)
    def get_tx_identity_info(self, *args):
        return self._hdp.get_tx_identity_info(*args)

    @lru_cache(maxsize=...)
    def get_tx_mapping_options(self, *args):
        return self._hdp.get_tx_mapping_options(*args)

    ...

This keeps the optimisation isolated to a single layer, avoids modifying call sites throughout VariantValidator, and focuses caching on the two most frequently accessed external resources: sequence retrieval and transcript/database lookups. It also provides a single place to tune cache sizes, collect cache statistics or disable caching if required.

[Peter-J-Freeman]

Here is the list of most common calls to hdp

Most Common HGVS Data Provider (HDP) Calls

Calls	Method	Purpose
19	get_tx_identity_info()	Retrieve transcript identity information (gene symbol, coding status, transcript metadata).
5	get_tx_for_gene()	Retrieve all transcripts associated with a gene.
5	get_pro_ac_for_tx_ac()	Map a transcript accession to its protein accession.
4	get_tx_exons()	Retrieve transcript exon coordinates and structure.
4	get_gene_info()	Retrieve gene information from the database.
3	get_tx_mapping_options()	Retrieve genomic mapping options for a transcript.
3	get_tx_for_region()	Retrieve transcripts overlapping a genomic region.
2	get_tx_limits()	Retrieve transcript coordinate limits.
2	data_version()	Retrieve the VVTA/UTA data version.
1	get_tx_seq_anno()	Retrieve transcript sequence annotation.
1	get_gene_info_by_alias()	Retrieve gene information using an alias or previous symbol.

[Peter-J-Freeman]

Benchmark methodology

To evaluate the impact of introducing in-process LRU caches, the full
VariantValidator test suite was executed three times for each caching
configuration using:

pytest -n 4

Each benchmark consisted of 2,197 passing tests and 6 skipped tests.
The same hardware, Python environment and xdist configuration were used
throughout. Three independent runs were performed for each
configuration, and the mean runtime, standard deviation (SD) and
coefficient of variation (CV) were calculated.

Three configurations were compared:

1. No caching (baseline).

2. SeqFetcher cache only, caching repeated fetch_seq() requests.

3. SeqFetcher + HGVS Data Provider (HDP) cache.

The HDP cache stores the results of repeated calls to:

• get_tx_identity_info()

• get_tx_for_gene()

• get_pro_ac_for_tx_ac()

• get_tx_exons()

• get_gene_info()

• get_tx_mapping_options()

get_tx_for_region() was intentionally excluded because genomic
coordinate queries are expected to have a very low cache hit rate during
normal validation workflows.

Results

Configuration	Run 1 (s)	Run 2 (s)	Run 3 (s)	Mean (s)	SD (s)	CV (%)	Δ Mean	Δ SD
No cache	468.55	450.96	472.48	464.00	11.46	2.47	–	–
SeqFetcher cache	463.84	459.06	459.62	460.84	2.61	0.57	-0.68%	-77.2%
SeqFetcher + HDP cache	412.18	435.23	415.53	420.98	12.56	2.98	-9.27%	+9.6%

Conclusions

Caching repeated sequence retrievals alone provides only a modest
improvement in overall runtime (0.68%), but it substantially reduces
run-to-run variability, decreasing the standard deviation by
approximately 77%. This indicates that sequence retrieval is not a major
runtime bottleneck but that eliminating repeated fetches makes execution
more consistent.

The majority of the performance improvement comes from caching HGVS Data
Provider lookups. When the HDP cache is combined with the SeqFetcher
cache, the mean runtime falls from 464.00 seconds to 420.98
seconds, representing a reduction of approximately 43 seconds
(9.27%).

These results suggest that repeated transcript and annotation lookups
are a significant component of VariantValidator execution time.
Specifically, caching transcript identity, transcript-to-gene
relationships, transcript exon structures, transcript mapping options,
protein accessions and gene information removes a substantial amount of
repeated database work during validation.

Although the combined cache exhibits similar run-to-run variability to
the uncached implementation, the overall reduction in execution time is
large enough to clearly demonstrate that the HGVS Data Provider cache is
responsible for the vast majority of the observed performance gains,
with the SeqFetcher cache providing a smaller complementary benefit by
avoiding repeated sequence retrievals.