Backup script halts at rsyncing

[kdeldycke]

As reported by Anže Jarni:

Sorry to bother you on your email regarding a probably obsolete project of yours, but I could not refuse to ask you this because I've spent a few days trying to fix this with no result...

It is about your system-backup.py script which does rsync and rdiff-backup over SSH. It has been working fine on a number of machines, but on the new hosting server, it just halts at rsyncing and stays there forever. It creates the directories, but fails to transfer any files. When I run the actual commands directly from bash, it completes normally without any stalling so I guess it's a python process management thing.

I would be glad if you could give me a few pointers or ideas from your experience, what the problem might be.

Thank you for your effort!

[kdeldycke]

It's hard to debug without proper trace. The script is old and really need to be rewritten.

It use deprecated Python API to execute all external commands. I suspect this old API to wait for the command to finish before returning any status or message. Thus giving you the feeling that nothing happens.

And I think it's not only the API that is blocking: the whole script does.

I'm sorry but I don't have time or interest now to rewrite it. But if you fix some bugs or add new features, feel free to submit a pull request. I'll be happy to merge it.

[kdeldycke]

My suspicions were true. Here is an answer from Anže Jarni:

Thank you for your response. I have found out that reducing verbosity of
rsync and rdiff-backup solved my problem. It seems that the output od rsync
and rdiff-backup goes over buffer on the python run() command and halts any
further execution. waitpid() then just waits indefinetly.

I will experiment a bit more and try to prepare a fix.

Have a great day!

[kdeldycke]

Thanks Anže for your tenacity ! Your did a great job tracking the root cause ! I'm sure you're not far from a definitive solution.