Skip to content

Delete temp files and use system temp dir - #1

Open
Imanm02 wants to merge 2 commits into
Mutahar789:mainfrom
MohammadiIman:temp-cleanup
Open

Delete temp files and use system temp dir#1
Imanm02 wants to merge 2 commits into
Mutahar789:mainfrom
MohammadiIman:temp-cleanup

Conversation

@Imanm02

@Imanm02 Imanm02 commented Jun 15, 2026

Copy link
Copy Markdown

The reference checker wrote every uploaded PDF to /tmp, and the LaTeX diff left its whole job tree (extracted sources, build dir, output PDFs) under /tmp/slate-diff, and nothing ever deleted either one. On the hosted instance that means disk keeps growing and every visitor's uploaded paper and source stays on the box.

What I changed:

  • run_diff reads diff.pdf and diff.tex into memory and removes the entire job tree in a finally, so cleanup runs on success and on failure. It returns pdf_bytes and tex_bytes instead of file paths.
  • Job dirs orphaned by a killed run are reaped on the next call, so disk stays bounded.
  • The reference checker unlinks its temp PDF in a finally after the check.
  • Both paths use tempfile.gettempdir() instead of a hardcoded /tmp. That also makes local runs work on Windows, where /tmp resolved to C:\tmp and the reference check crashed on a clean machine.
  • app.py serves the diff download buttons and the preview from the returned bytes.

I verified locally: an end to end diff produces a valid PDF and leaves no temp directories behind.

Copilot AI review requested due to automatic review settings June 15, 2026 07:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR shifts diff/refcheck temp artifacts to the system temp directory, improves cleanup of generated working directories/files, and changes the diff pipeline to return in-memory bytes for downstream UI download/preview.

Changes:

  • Use tempfile.gettempdir() and rename temp prefixes from slate-* to sift-*.
  • Add stale job reaping and ensure diff workdirs are deleted via try/finally.
  • Return pdf_bytes/tex_bytes from run_diff() and update Streamlit download/preview to use bytes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
diff.py Switch temp root handling, add stale-job cleanup, and return PDF/TEX content as bytes while always deleting job workdirs.
app.py Update refcheck/diff flows to use tempdir + cleanup, and consume diff results as bytes for download/preview.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diff.py
from typing import Callable

ROOT = Path("/tmp") / "slate-diff"
ROOT = Path(tempfile.gettempdir()) / "sift-diff"
Comment thread diff.py
Comment on lines +455 to +457
return {"pdf_bytes": diff_pdf.read_bytes(),
"tex_bytes": diff_tex.read_bytes(),
"strikethrough_used": strikethrough}
Comment thread diff.py
Comment on lines +340 to +347
def _reap_stale_jobs(max_age_secs: int = _JOB_MAX_AGE_SECS) -> None:
if not ROOT.exists():
return
cutoff = time.time() - max_age_secs
for job in ROOT.glob("job-*"):
try:
if job.is_dir() and job.stat().st_mtime < cutoff:
shutil.rmtree(job, ignore_errors=True)
Comment thread app.py
Comment on lines +329 to +338
pdf_bytes = result["pdf_bytes"]
tex_bytes = result["tex_bytes"]

st.success(f"diff.pdf ready ({pdf_path.stat().st_size/1024:.0f} KB).")
st.success(f"diff.pdf ready ({len(pdf_bytes)/1024:.0f} KB).")

c1, c2 = st.columns(2)
with open(pdf_path, "rb") as f:
c1.download_button("Download diff.pdf", f.read(),
file_name="diff.pdf", mime="application/pdf",
use_container_width=True)
with open(tex_path, "rb") as f:
c2.download_button("Download diff.tex", f.read(),
file_name="diff.tex", mime="text/x-tex",
use_container_width=True)
c1.download_button("Download diff.pdf", pdf_bytes,
file_name="diff.pdf", mime="application/pdf",
use_container_width=True)
c2.download_button("Download diff.tex", tex_bytes,
Comment thread app.py

st.markdown("---")
_preview(pdf_path)
_preview(pdf_bytes)
@Imanm02

Imanm02 commented Jun 15, 2026

Copy link
Copy Markdown
Author

I pushed d5c88a2 with the temp-dir and reaper changes; notes on each point below.

ROOT temp dir (1)
Each job now gets a private directory via tempfile.mkdtemp(prefix="job-", dir=ROOT). The per-job path is random and created with mode 0700, and mkdtemp creates it atomically and fails rather than reusing an existing path, so jobs no longer write into a predictable shared location. The base directory is also created with mode 0700.

Return contract (2)
run_diff has a single caller in this repo (app.py) and is not a public API, so there is no external integration to keep compatible. Returning the old file paths would also require keeping the job files on disk, which is the leak this PR removes, so I kept the bytes-only return.

Reaper mtime (3)
Addressed in d5c88a2. The reaper now bases staleness on the newest mtime found anywhere in the job tree (job.rglob("*")), not just the top-directory mtime, so a job that is still writing files is never treated as stale. The per-job dir is also removed in a finally on every run, so the reaper only handles orphans left by a hard kill, and the compile timeouts keep a job well under the 1 hour cutoff.

Memory from bytes in session_state (4 and 5)
The app already held both full uploaded zip blobs in session_state, and diff PDFs are usually small, so this does not add a new class of memory use. Keeping files on disk and streaming on download would reintroduce the disk leak this PR fixes. To lower the footprint I now drop the two input zip blobs from session_state as soon as the diff finishes (d5c88a2). If output size becomes a concern we can add a max-size cap separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants