My apologies for the long title.

I’m looking for something to organise the materials I collect on various topics. As mentioned, these usually include archived webpages, documents/manuals, media (pictures, videos, audio) etc.

I know I could just create a directory system for this (and will likely do so if I can’t find anything like what I describe), but just wanted to ask if anything like what I want exists.

Thanks!

  • @vegetaaaaaaaEnglish
    5•1 year ago

    Use files when possible - I use a combination of:

    • filesystem hierarchy (max 3-4 levels deep, I can share the general structure if needed)
    • markdown notes for wiki-like content or notes (either versioned in git - so also accessible from my Gitea instance, or under the Nextcloud Notes/ directory, so also accessible from the Nextcloud Notes app)
    • software mirrors either through a mirroring script or using Gitea’s mirroring feature
    • Shaarli for bookmarks and wiki-like content, which get processed every day by a script that archives content to local files (mostly audio/video for now, I’m still writing the page archiving part, archivebox is too bloated for my needs and is missing critical features such as ad blocking)

    All these components are linked in some way or another (e.g. all media automatically goes to the media directory of a jellyfin instance)

    • @MigratingtoLemmyOPEnglish
      4•1 year ago

      Thanks, I do plan to use a organisation structure with plain directories. Please do share, it would help me greatly!

      Indeed, I too plan to use markdown for any notes on what I’m looking at/is relevant to the object of interest.

      Thank you for the note about archivebox, I didn’t think about this before! Indeed, that would be very important: are you writing an alternative tool? Would you like to share the repo? I was considering just using wget to pull down pages but that might not work all the time.

      Thanks a bunch!

      • @vegetaaaaaaaEnglish
        2•1 year ago

        organisation structure with plain directories

        Slightly edited so you get the idea:

        ├── ARCHIVE
│   ├── DOCUMENTS
│   │   ├── 2018
│   │   ├── 2019
│   │   └── 2020
│   ├── WORK
│   │   ├── PROJECT1
│   │   └── PROJECT2
│   ├── DATA-MISC
│   ├── NOTES
│   ├── GAMES
│   ├── IMAGES
│   │   ├── MISC
│   │   ├── 2018
│   │   ├── 2019
│   │   └── 2020
│   ├── BOOKS
│   │   ├── IT
│   │   ├── DIY
│   │   └── NOVELS
│   ├── MUSIC
│   │   └── ARTIST - ALBUM
│   ├── SOFTWARE
│   │   ├── LINUX
│   │   ├── WINDOWS
│   │   └── ANDROID
│   └── VIDEO
│       ├── MOVIES
│       ├── MUSICVIDEOS
│       └── DOCUMENTARIES
├── DOWNLOADS
│   ├── DOCUMENTS
│   ├── WORK
│   ├── BOOKS
│   ├── GAMES
│   ├── MUSIC
│   ├── SOFTWARE
│   └── VIDEO
└── TMP

        I use UPPERCASE for my base directory structure, so I know if a directory is uppercase it’s probably part of the fixed structure. The key is to keep it max 2-3 levels deep.

        Level 1:

        • ARCHIVE: stuff I want to keep, gets backed up
        • DOWNLOAD: stuff I did not have time to listen/look at/process yet. Not backed up (but I do backup a list of the files in this hierarchy).
        • TMP: stuff I use regularly but does not deserve to be archived/backed up (working copies of projects, random scripts/programs, VM disks…). Temporary, expendable.

        Level 2: Broad topic or media type. Max 5-8.

        Level 3: Finer-grained topic/media type. Only the ARCHIVE tree has this level of organization. There may be directories deeper than that but I don’t actively manage them, they just… exist (extracted archives, etc.). One exception are subdirectories named NOBACKUP which are always excluded from automatic backups.

        archivebox […] are you writing an alternative tool? Would you like to share the repo? I was considering just using wget to pull down pages but that might not work all the time.

        I am working on this tool which is a generic data manipulation/workflow tool. The shaarli workflow already works to grab bookmarks from Shaarli and download audio and video files. The webpage archiving module is still not written, it’s the early design stage (issue), it will probably use wget in the backend, the alternative would be running a full headless browser and I don’t want to get into that. This is my first medium-sized python project and I try to keep it clean, so it will take some time. Currently I’m more focused on other workflows/parts of the software.

        As for file organization inside the directories, I try to maintain consistent/useful file naming including (depending on directory) date in YYYYMMDD format, author/parties involved, subject.