Hot on the heels of LCS (https://lemmy.world/post/557346), I bring you Lemmy Post Purger (LPP).
Instances can grow fast, and that can be a problem if you have limited disk space. If you don’t mind losing history that your users have not interacted with, LPP can help. It will purge posts along with their comments and media before a certain time period if no comments, likes, or saves exist for users on your instance.
I feel like this ruins the aspect of an archive of information where users can go back through and find useful info similar to SO in a way. Maybe there will be a meta search engine for looking through all of the popular instances?
Generally I agree. But I think there are divergent interests:
- Sustainable decentralisation
- Archiving as much useful information as possible behind a good global search
At some point these conflict and it makes sense to direct efforts specifically toward one if you want it realised.
Purging to lower costs makes sense for decentralisation.
To have a great archive of human knowledge, at some point it would make sense to have instances or services dedicated to that.
We need a sort of Pushshift for Lemmy. That way Lemmy instances can purge posts for their own survival, while a full archive exists and can be accessed if needed. Best of both worlds. Perhaps the Lemmy project should be working with the Pushshift people to integrate that functionality.
I think it depends on the implementation of this idea, given the nature of federation. It seems like a good place to start.
It’s a balance between losing information due to purging old stuff, and losing information due to an instance shutting down entirely from costs being too high.
so it will only clear ‘history’ of other instance’s post and media saved in my own instance? am i getting this right?
Where can we suggest improvements?
This could be a very powerful tool if the right options are added. One of the things I’d like to see is the ability to set an upvote or comment threshold. If posts were popular, I would like to keep them. If, however, a post received very little upvotes and had little interaction, there’s not much value in keeping it. And I’m talking about all interactions here, not just from members of our own instances.
This doesn’t seem like a very hard thing to implement and would be well received.
On the GitHub issues page would be a good place. That particular option would just be added to the query.
Starred this to take a better look later. Thank you for making this!
This looks great! I’d want more customization though. I’d want to see
- community / instance filter so it doesn’t delete everything
- purge based off of disk usage, such as 90% full, then purge.
This would allow the instance to still have content to interact with for a longer period of time.
2 hour turn around from your other post, not too bad
Yea I was on a roll. Set up my insurance and made both that afternoon. Productive day ;)
Hey there @[email protected] does this script still work?
I did a fresh git clone today, and when I try to npm start I am getting:
[email protected] start node index.js
undefined:1 undefined ^
SyntaxError: Unexpected token u in JSON at position 0 at JSON.parse (<anonymous>) at Object.<anonymous> (/root/lcs/src/index.js:8:30) at Module._compile (node:internal/modules/cjs/loader:1191:14) at Object.Module._extensions…js (node:internal/modules/cjs/loader:1245:10) at Module.load (node:internal/modules/cjs/loader:1069:32) at Function.Module._load (node:internal/modules/cjs/loader:904:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) at node:internal/main/run_main_module:22:47
This is what I ran, I am using the Ansible install of the latest version of Lemmy.
cd src export LOCAL_URL=https://my.full.TLD export LOCAL_USERNAME=myadmin export LOCAL_PASSWORD=myadminpass export PURGE_OLDER_THAN_DAYS=14 export HOURS_BETWEEN_PURGES=4 export PG_HOST=172.18.0.3 #this is the local IP on my box that answers for the docker container that has postgres export PG_PORT=5432 export PG_USERNAME=lemmy export PG_PASSWORD=thelemmydb-password npm install npm start
Hmm, I should probably check the issue tracker… I haven’t checked in a while, perhaps Lemmy has changed something.
If there is anything I can do to help test, let me know. I just reinstalled my server because it had 100 percent filled up and manually deleting files was not releasing the disk space.
