question about inference, when Killed in Parsing reads

[Henry-Ding]

Hi,

I successfully ran the configured model using the sample data.

However, when I used my own data, the 11-12 minutes it took to run in the "Parsing reads" stage were "Killed".

There were no error messages, regardless of whether I added the "--read-alns" parameter or not.

My ONT fastq file contains 2,215,080 reads, after preprocessed contains 2,496,551 reads.

My task is running on the 2T memory and 80G GPU memory.

Sincerely.

[Henry-Ding]

herro/scripts/preprocess.sh ont.fq.gz ${sample}ont_preprocess 104 1

seqkit seq -ni ont.fq.gz > ${sample}ont_readIDs.txt
/herro/scripts/create_batched_alignments.sh
${sample}ont_preprocess.fastq.gz
${sample}ont_readIDs.txt
104
${sample}ont_batched_alignments

This is the preprocessed code of mine.

[dominikstanojevic]

Hi @Henry-Ding,

The bare Killed with no error message is almost certainly not a true out-of-memory condition — on a 2 TB machine that would be very surprising. Are you by any chance running the job through a scheduler like Slurm or PBS? Even on a large server, jobs are typically submitted with a memory allocation well below the physical RAM, and the scheduler will silently kill the process the moment it exceeds that limit.

If so, could you check what memory limit was assigned to your job? Let us know and we can help you figure out the right allocation for your dataset size.

Best,
Dominik