Skip to content

Filing: Zotero, papis & more

papio's deep integration is zotio → Zotero: ownership deduplication, idempotent plan/apply, import retry, enrichment, and collection filing. If you file papers somewhere else — papis, Calibre, a plain folder, your own script — use the generic on_ready hook instead.

What on_ready does

When a job reaches ready (its PDF passed structural and identity validation), the daemon runs your configured shell command once, with the job's metadata in PAPIO_* environment variables:

[hooks]
on_ready = 'papis add --from doi "$PAPIO_DOI" "$PAPIO_PDF"'
timeout_seconds = 120   # optional; default 120, range 5..600

Contract:

  • Fires once per ready transition. Import retries and daemon restarts do not re-fire on_ready.
  • Fire-and-forget, best effort. A slow or failing hook never blocks or fails the job, but filing is not guaranteed. papio does not retry hooks.
  • Audited. Each run records a durable hook.on_ready job event with status, exit_code, duration_ms, and trigger (ready or manual). Hook stdout/stderr is not recorded (it could carry secrets from your environment) — have your command do its own logging if you need output.
  • Shell semantics. The command runs via /bin/sh -c (on Windows, cmd /d /s /c "<command>", which keeps your quotes as written). The recipes below are POSIX-shell.
  • Concurrency. Concurrent ready jobs may run hooks concurrently; if your command needs serialization, own it (e.g. flock).

Managed folder filing

Set an absolute destination to enable durable folder filing:

[filing]
folder = "/absolute/path/to/papers"

The daemon records a destination-specific receipt for each validated job. It retries transient folder failures after restart without repeating arbitrary on_ready scripts. The final filename derives from the filing receipt key. Existing files must match the expected artifact hash; a different file or a symbolic link causes a conflict instead of an overwrite.

papio jobs filing list --json
papio jobs filing inspect <job-id> --json
papio jobs filing retry <job-id>

Automatic retries use only the current configured destination. Historical receipts remain inspectable after a configuration change. Use retry --destination <absolute-path> to select a journaled destination explicitly. Empty filing.folder disables managed filing.

Archive and hook execution

Archiving an acquisition prevents new hook execution. A running hook holds a durable reservation until its command returns. The diagnostic events are hook.filing_reserved and hook.filing_released, joined by reservation_id. A crash or a failed release leaves the reservation unresolved, so archive fails closed. There is no automatic reconciliation for that reservation. The reservation covers command execution, not background descendants that a best-effort script leaves running after its shell exits.

Environment variables

Variable Value Always set?
PAPIO_JOB_ID job id yes
PAPIO_REQUEST_ID originating work-request id yes
PAPIO_DOI DOI (10.…) empty when the work has no DOI
PAPIO_ARXIV arXiv id empty when absent
PAPIO_PMID PubMed ID empty when absent
PAPIO_TITLE requested title empty when absent
PAPIO_SHA256 artifact content hash yes
PAPIO_PDF absolute path to the validated PDF yes
PAPIO_STATE ready yes

Treat PAPIO_PDF as read-only. It points into papio's immutable content-addressed artifact store. Copy the file if you need to move or rename it — papis add and cp both copy by default.

Recipes

papis

[hooks]
on_ready = 'papis add --from doi "$PAPIO_DOI" "$PAPIO_PDF"'

Works acquired without a DOI need a fallback; point the hook at a small wrapper script instead:

#!/bin/sh
if [ -n "$PAPIO_DOI" ]; then
    exec papis add --batch --from doi "$PAPIO_DOI" "$PAPIO_PDF"
fi
exec papis add --batch --set title "$PAPIO_TITLE" "$PAPIO_PDF"

Plain folder

[hooks]
on_ready = 'cp "$PAPIO_PDF" "$HOME/Papers/"'

Running without zotio

zotio.executable is optional — answer none at the zotio prompt in papio init, run papio init --zotio-path "", or clear the key in config.toml. With it empty, the deep Zotero integration (auto-import, papio zotio … commands) is disabled, ownership lookup treats every work as new unless you configure a library source (below), and papio doctor reports zotio as not configured (optional) instead of failing. Without it, two automatic hand-offs remain, and they differ. The opt-in managed folder destination keeps a durable receipt and retries after restart. A hook is a best-effort script that you write, and papio does not retry it as filing. papio acquires and validates; the folder or your hook files.

Finding and repairing failed filings

papio doctor includes a filing check when on_ready is your library hand-off. It warns when a ready or imported job has no successful filing. Both commands in this section require a configured [hooks] on_ready command.

List those jobs:

papio jobs unfiled

Use --filter failed or --filter missing to show one class. After you fix the hook or its destination, file one job again:

papio jobs refile <job-id>

The command runs the same hook with the same PAPIO_* environment. It waits for that bounded run and records a manual event. It refuses a job whose newest filing succeeded, and it refuses a second run while one is active. When the hook fails or times out, the command prints the result and exits nonzero, so a script can tell the paper is still unfiled. papio never retries a filing automatically.

De-duplicating against a non-Zotero library

Point papio at an export of what you already hold and it stops re-acquiring it. papio search marks those results [in library], and papio acquire --batch skips them.

[[library.sources]]
name   = "owned-pdfs"
kind   = "file"
path   = "~/library/with-pdfs.bib"
format = "bibtex"
claim  = "pdf_present"

claim is the whole contract, and it has no default:

  • pdf_present — every entry is something whose full text you hold. A match skips acquisition.
  • record_present — citations only. A match annotates papio search but never skips, because a citation without full text is exactly what you would want acquired.

papio does not read BibTeX file or papis files to guess which one applies: that is per-manager convention, and a wrong guess would skip a paper you asked for. Export the subset you mean, or declare record_present.

Ownership matches normalized DOI and PMID identifiers. It also collapses arXiv version suffixes, so v2 matches v1 as the same work. BibTeX supports DOI, arXiv, and PMID; CSL-JSON and NBIB support DOI and PMID; RIS supports DOI only. No format supports every identifier, and titles are never matched. ISBN is excluded — an edited volume shares one ISBN with every chapter in it, so one match would suppress twenty distinct requests. A .bib also cannot say which manifestation it holds, so a source never satisfies an explicit --desired-version published request; conversely, a request without an explicit desired version matches any manifestation, so an exported v1 can suppress a requested v2.

Any tool that exports RIS, BibTeX, CSL-JSON, or MEDLINE/NBIB works — papis, JabRef, Calibre, Mendeley, EndNote, a hand-kept .bib. There is no supported-app list because there is nothing app-specific to support.

A live library: command sources

An export file goes stale as soon as your library changes. A command source runs a program instead and reads its standard output as the export, so papio sees your library as it is now:

[[library.sources]]
name            = "owned-pdfs-live"
kind            = "command"
argv            = ["~/bin/export-owned-pdfs"]
format          = "bibtex"
claim           = "pdf_present"
refresh_seconds = 300

argv is the program and its arguments. papio runs it directly, with no shell. The program must be an absolute path (~/ is allowed); papio does not look it up on PATH, because the daemon's PATH is often not your shell's. To use a pipeline, name the shell yourself, for example argv = ["/bin/sh", "-c", "papis export --all --format bibtex"]. When the command exits, papio stops any process it left running in the background. The same claim contract applies: for pdf_present, the command must print only the entries whose PDF you hold. papio does not filter the output for you.

papio runs the command at most once every refresh_seconds (default 300), not once for each paper, and lookups during a run share it. A run that exits non-zero, passes timeout_seconds (default 30), prints more than max_output_bytes (default 32 MiB), or prints output that does not parse makes the source unreadable, exactly like a missing export file. The configuration reference lists every limit.

An unreadable source is not an empty library

If zotio is configured, generic library.sources are ignored. Otherwise, if a configured source cannot be read, papio refuses to guess. --batch creates no jobs and tells you which source failed. Discovery watches fail the run the same way, on the first page or on any deeper page: an acquire watch queues nothing, an alert watch records no digest entries, and papio acquire --from-digest creates no jobs. The watch also keeps each source's stored scan position, so the next run reads the same pages again. Before the fifth consecutive failure, each cadence attempts another run; a successful run resets the failure count. The fifth consecutive failure disables the watch. After fixing the source, force a run with papio watch run <id>; when it succeeds, papio watch resume <id> puts the same watch back on its schedule, with its id, query, and digest history intact. Resume refuses until a run has succeeded since the watch was disabled. --include-owned is available only for papio acquire --batch, meaning "proceed despite ownership uncertainty". papio doctor performs a fresh one-shot probe of each source and reports that read's record count and outcome; it does not report daemon cached age, count-collapse detection, or retained failure state.