Usermin mail access very slow and odd behaviours

[ksihota]

SYSTEM INFORMATION
OS type and version	Ubuntu Linux 24.04.4
Webmin version	2.641
Usermin version	2.540
Virtualmin version	8.1.0 GPL
Theme version	26.41
Apache version	2.4.63
Package updates	All installed packages are up to date

I have been noticing a slowdown of the Usermin mail display. The right panel takes forever to display the messages. I see the moving barber pole and often the red moving indicator line at the top but the messages do not display. Sometimes I can select another folder and its messages will display and if I click back on Inbox the messages might come up. I have tried a variety of way to refresh the page or panels and they sometimes work but not consistently. It basically makes using the mail client useless.

I also see issues with deleted messages still being displayed but with the dates changed to 1969

Do I have a configuration issue on my server or is something else going on?

Here are the contents of my etc/usermin/mailbox/config file

no_mailer=0
sendmail_path=/usr/lib/sendmail
smtp_ssl=0
mail_style=0
folder_types=local,ext,pop3,imap
nologout=1
from_map=/etc/postfix/virtual
shortindex=0
folder_virts=virt,comp
html_base64=0
mail_file=mbox
mail_dir=/var/spool/mail
noindex_hostname=1
view_images=3
pop_locks=1
top_buttons=1
edit_from=0
server_attach=0
mail_dir_qmail=Maildir
spam_report=
show_to=0
no_crlf=0
view_html=2
spam_always=0
wrap_width=80
no_orig_ip=1
from_format=1
perpage=20
mail_system=4
html_edit_mode=simple
pop3_ssl=0
send_mode=localhost
pop3_server=*
mail_qmail=

[jcameron]

How large is the mailbox you are trying to list?

Also, where are the mail files stored on your system?

[ksihota]

maildir

The complete maildir directory that holds all mail folders and files for my email address. Generally its the Inbox that is the slowest and other folders usually display much faster (or at least appear to)
Type: directory
Size: 912.33 MiB (4096 bytes) Blocks: 8 IO Block: 4096 directory
Device: 8,0 Inode: 1186435 Links: 55

This is the inbox showing a few of those odd (deleted) messages in Usermin. They do not show up in Thunderbird.

Is there something I can run to go through and repair the mail folders if that is an issue?

[ksihota]

Interesting, it may have to do with VPN. I usually don't have VPN on for that but I noticed that it had turned on automatically (maybe after a reboot) that I hadn't noticed. I am testing that theory out to see if it doesn't work when I am using VPN and if that is the only issue. That doesn't explain the strange behaviours with deleted messages though (or could it? Some kind of cache behaviour?)

[ksihota]

Okay, I don't think it was VPN. The slowness may be due to file size. I removed all messages with attachments and then all large messages (usually embedded images) and the loading appears to be quite a bit faster. The odd None (no subject) and 12/31/1969 still display in the lists and they appear to hold their original information (they are all in strange locations when I sort) as they do not sort with the displayed information. I tried ctrl-F5 and ctrl-shift-R to clear the cache but that does not clean up the display.
Using Thunderbird everything loads quickly, and sorts properly. I also do not see those strange entries after messages have been moved or deleted. Also Usermin display a count of 208 messages whereas Thunderbird displays a count of 149. When I view the number of files in my Maildir/cur directory it displays 149

[ksihota]

Left the thing alone for a couple of hours. When I came back and checked the message count was corrected and the deleted messages were all gone. There was still one of those 1969 messages at the top of the page (marked as unread) and when I viewed it it said it no longer existed. I refreshed the page using the refresh button at the top of the message page and that message cleared. I believe that message was one of my mail filters that moved the incoming message into a different folder so it no longer existed in the Inbox but really I believe that the interface should refresh itself after those types of actions. It should not rely on a user having to do the refresh. I know I have used that menu button before to refresh a page and try and remove those non-existent messages and it has not always worked. I have a lot of mail filters that organize my incoming messages into different folders. Maybe the issue relates to how the client interacts with the mail filter system?

[ksihota]

I noted that the count had corrected itself to match the number of messages in the cur directory but that is not quite right. I deleted 1 message and the cur directory (and thunderbird) both report 150 messages whereas Usermin displays 151 messages so it is still counting the deleted message. I tried refreshing and the count does not correct. I expect that there is some job that is happening at a set time interval that is correcting these numbers but I do not appear to have control of? Does Usermin have some kind of internal refresh that it performs that might need to be set to occur more often or after certain tasks are completed?

Edit: The message I thought was gone (the one I deleted and was then renamed as none and the date set to 1969) resorted to the bottom of the list so I didn't notice it there. Refreshing the page does not remove it.

[ksihota]

As an aside, but related to the Usermin interface, I noticed that the interface in Virtualmin and Webmin appear to have a large blue button to return you to a previous page when a procedure redirects you to a results page. In Usermin after doing a search or process that uses the search, such as doing a search in email filters and then moving or deleting all the files, you are left on the same empty page. I look for a return button but the only choice I have is a greyed out Home Icon on the top right. This appears to be an interface inconsistency. When you select an email to read that blue return button is available so I always expect to see it in order to get back. I know I can use the Home Icon but was wondering why the same Return interface is not used in the Search page.

I have to use the search procedure in Filters to move or delete messages because the move procedure (as I pointed out in another thread) does not work on its own. It always shows that no messages meet the specified criteria, whereas the Search procedure finds the specified messages as set in the Filter.

[iliaross]

This PR webmin/webmin#2744 should fix it.

@ksihota In the meantime, could you try this PR? Run the command below to apply it and see if it helps:

 webmin patch https://patch-diff.githubusercontent.com/raw/webmin/webmin/pull/2744.patch

[ksihota]

@iliaross thanks for looking into this. I ran the command in webmin of the main server and got this return
root@server:~# webmin patch https://patch-diff.githubusercontent.com/raw/webmin/webmin/pull/2744.patch
Patch applied successfully to:
mailboxes/boxes-lib.pl
mailboxes/folders-lib.pl
mailboxes/mailboxes-lib.pl
I then checked the Usermin account and after waiting some time for it to load (still extremely slow loading/display of the message list for Inbox) got exactly the same info returned. The None message is still there and both Thunderbird and file manager still reports 1 fewer messages (they do not list the NONE message.)
Should I reboot my server to see if it makes any difference?
The interesting thing is that the interface reports the extra deleted message but when you select it to view it knows it does not exist. That leads me to believe that there are two different processes, one that is reading some old information to display the list data and a different one that views the actual data that exists.

[iliaross]

Should I reboot my server to see if it makes any difference?

Ah, you should also copy these patched files to Usermin:

sudo cp /usr/share/webmin/mailboxes/boxes-lib.pl /usr/share/usermin/mailbox/boxes-lib.pl
sudo cp /usr/share/webmin/mailboxes/folders-lib.pl /usr/share/usermin/mailbox/folders-lib.pl
sudo systemctl restart usermin webmin

Do that, and give it another try.

[ksihota]

Ran the 3 commands and the Deleted message still displays as none. I am running the commands using the Webmin Terminal

[iliaross]

I'm not sure what’s wrong then, sorry.

Jamie, do you have any idea?

[ksihota]

Here is an odd behaviour that might be helpful. When I sort the messages by date I get Messages 1-154 of 154 and the deleted message is displayed as none. However, if I sort by any other column, subject, size or sender I get Messages 1-153 of 153 and the deleted message is gone.

[iliaross]

Thanks for the heads-up! Got it.

Please now also apply this patch webmin/webmin@034d0a0 — it should fix the issue for you.

After applying it, run a terminal tab with tail -f /var/usermin/miniserv.error and share any errors that appear after reading mail with the new patch.