I’m looking for a service I could install to archive a huge pile of letters, preferably in PDF form, to a database. I’m living in a country where paper is still king, and digital services are either non-existent, or loathed (Germany). My current situation is that I have a mailbox with lots of PDFs all over the place, but also many folders of paper sent in 2007 etc. that I have to keep, but I also have to find them every five years or so.
So what I’d like to have is a service to my homelab, where I could scan these and copy these, that would index them, clean them, OCR them and all that good stuff. It should have really good metadata abilities, because my files are usually named in a very random way, so if I could copy these, and quickly categorize them, that would be really awesome.
There is one service called Papermerge, that kind of fits to my use-case. I spent one afternoon with it, and there were a few issues:
- crashes quite often
- when sending a large folder of PDFs, uses all the CPU and crashes again
- categorizing functions are not very good, it takes time to get everything together and clean when organizing files
This might not be very interesting if your country has digital services for everything, but for us needing to suffer this paper madness, a service to do so would be great.
I second Paperless NGX. I have been using it for a few years, and it has been working great!
This is the correct and only answer :)
This ticks all your boxes. It’s really good.
The killer feature for me is my networked scanner scanning directly to the paperless consume samba share and the documents just popping up in the inbox fully OCRd and pre-categorised. Pretty magical.
NB, the docs make it sound like a proper DB is optional, but it’s really not. Performance was iffy for me with sqlite but is rock solid with Postgres.
This was it for me now, installed paperless-xng, set it up to scan my email folders, copied all random PDFs from my “organized” tax folder and scanned the rest.
Too bad I just happen to have that Brother printer/scanner without SMB or FTP support. So I need to go through the process of scanning on my computer first, then uploading.
If it can email you can send it to an email address and paperless can automatically grab it and archive it.
I’ve been digging into the settings of this printer and, sadly the only send it can do is as a fax… It’s the entry model, been serving us for years very nicely. It even connects to the internet, but misses features such as email, smb or ftp. For me this looks like something an open source firmware could fix. It has enough processing power to possibly run a lightweight Linux distribution, so installing one that would enable modern communication protocols doesn’t seem impossible.
I’ve had excellent luck with Docspell. https://github.com/eikek/docspell
I love docspell.
Note that just because everything is digital doesn’t mean something like that isn’t necessary: If you depend on your service provider to keep all of your records then you will be out of luck once they … stop liking you, go out of business, have a technical malfunction, decide they no longer want to keep any records older than X years, …
So even in a all-digital world I’d still keep all the PDF artifacts in something like that.
And I also second the suggestion of paperless-ngx (even though I’m not using it for very long yet, but it’s working great so far).
Of course. My setup now is a Proxmox server + a NAS. What I’m planning to do is to install a service for this to Proxmox, then have the files synced over NFS to the NAS, which then backs them up every night to Backblaze. And of course I need to have the paper copies too, but to be able to search, tag and archive the documents is great when you need to remember a thing X that was mentioned in a paper I got back in 2014.
I’ve been pretty happy with paperless-ngx, it should tick all your boxes
Just explain me how simple folders by date or/and whoever send the letters in PDF format aren’t enough for a regular person?
The three letters OCR, tagging, fuzzy search and ease of use are the ones for me.
I never needed the date for a letter but quite often its context for example.
Your suggestion just digitalizes physical folders. If that’s enough for you ok - but you’re missing out.
Well I do see the advantages of what your suggesting, no depute there. Searching for a specific tag would make my life easier but at what cost?
As I was saying a person - not a company - won’t likely be receiving that much important letters to the point you can’t simply go through a couple of folders and find out what you’re looking for. Paperless-ngx could indeed make me save a few minutes while searching for documents but then, what about the amount of time and effort it would be spending keeping the software running, up to date, backups etc? More importantly, what about longevity? Those kinds of archives are something you may want to get into 10 or 20 years to look for a file and then software you chose might not be around or working anymore.
The extra minutes wasted while searching and having the piece of mind provided by simple folders and PDF files seem to be a good tradeoff as it eliminates the need for databases, upgrades, special servers, formats and whatnot.
For me it was a few hours wrapping my head around how paperless ngx works and its setup. I had a folder structure as you described already on my Nextcloud so I just configured paperless to observe it for new files.
Where I spent more time then reasonable with was the tagging - you can automate it based on… Well everything.
Now I just let it suggest me tags based on my existing documents plus add a NEW tag to the ones I’ve never reviewed. That’s just a reminder for me though to review tags when searching, I don’t actively re tag new uploads.
If you have a docker environment I suggest just pulling a container up3, throwing all your documents in it and see if it would save you time or cost you time. Would be an hour well spent!personally the OCR alone is it worth it for me - my country still loves paper letters and being able to copy text out of that is awesome (IBAN, account numbers, etc - all the stuff that’s suspectible to typos).
If you have a docker environment I suggest just pulling a container up3, throwing all your documents in it and see if it would save you time or cost you time. Would be an hour well spent!personally the OCR alone is it worth it for me - my country still loves paper letters and being able to copy text out of that is awesome (IBAN, account numbers, etc - all the stuff that’s suspectible to typos).
Yes I understand the pain and I usually go with Acrobat to do OCR of scanned documents. Now tell me something, are you sure docker and paperless will be around in 10 or 20 years? How are you planning to deal with that long term? I’ve documents from the 90’s copied over from floppy disks and whatnot a simple flash drive or hard drive plugged into my computer works as a quick backup for everything. Extra layers of protection can be added, but generally speaking files are easier to copy and checksum across time and media than some software with hundreds of dependencies, a webserver and whatnot.
Worst case I have all my OCRed documents as raw files which I can migrate to whereever.
Files still exist. For my case encrypted as well. My backups roll on the data, not the container.
But I’m not trying to convince you, I tried answering the questions :)
And two answer your last question clearly: I survived before paperless, I’d get along without it. I find a new tool to mitigate my manual labor as good as possible - if that’s not possible then jo harm done. I know I’m flexible, I can learn new tools and I’m never vendor or tool locked-in. I have a high level of self confidence when it comes to my tool chain and how I’d adapt any part of it - from password manager to cloud storage and my mail flow.
To be honest I couldn’t self host anything if I’d had the fear of being lost if a tool is discontinued.
But I’m not trying to convince you, I tried answering the questions :)
I was just trying to see how you’re thinking about the possible lock-in and dependency on those platforms… also exposing my real concerns with them.
To be honest I couldn’t self host anything if I’d had the fear of being lost if a tool is discontinued.
Yeah but most thing we self host are more “fungible” be it a torrent client, RSS aggregator etc. can be quickly replaced by another alternative as they hold little to no data and even sometimes the data they hold doesn’t even have any value. A document management solution however is a long term thing that holds important documents.
My way of using paperless-ngx includes an automatic export to plain pdf-files which are synced via syncthing.
Everything is accessible with a normal filesystem and over the keepass-gui…
Ahh g I don’t use paperless as an exclusive document storage but as a pure manager. It searches and tags but doesn’t have exclusivity over any files but it’s own indices!
It doesn’t provide more value than jellyfin in that regard - make it visible and accessible.
Good luck finding stuff when you have 1000+ files and you only search in the file name
