I installed Jerboa and noticed that it grayed the titles of posts that I had already viewed even though I had viewed them on the web. That told me (unless I am somehow confused) that the server side tracks what posts you have read.
From my perspective that seems like a terrible invasion. I can understand some benefit to showing the post status in the UI, but if it is stored at all, the storage should be exclusively on the client side. I mentioned this also in the “issues” thread and got no reaction, so maybe I’m missing something or in error.
Thanks, that is good to know, but that is a type of evil where I would hope Lemmy doesn’t follow Reddit. I sometimes posted to Reddit but I more often read passively without logging in, partly to avoid some of the tracking.
Sure. I think it’s good to be aware of for sure and I agree that it would be nice if Lemmy isn’t tracking as much. I also recognize that I’ve accepted a certain amount of tracking in my life over the years at this point.
I don’t disagree really… Just pointing out that as long as you’re logged in Reddit has always tracked posts viewed as far as I’m aware… Facebook similarly tracks all activities and always has.
These are obviously not models to aspire to but I think that it’s helpful to be aware of what we’ve dealt with up until this point.
Facebook was notoriously evil and I actuallly have all their domains that I know of blocked from my computer at the DNS level in order to avoid their spying. That Reddit tracks posts viewed is new to me but I guess not that surprising. Usenet never tracked posts viewed and basically couldn’t. Wikipedia emphatically doesn’t track that, though it doesn’t track view counts. Arxiv.org doesn’t track (or at least publish) view counts for individual papers (see here) though they do publish stats about the entire site. A real privacy focused site would avoid publishing any about what viewers are doing. There is a whole topic in cryptography called private information retrieval about how to run a server in which the clients can verify that the server can’t know what they are reading. This is what Lemmy should aspire to, imho. (Aspirations aren’t meant to be achieved literally, but only to provide guidance).
I may open a thread in /c/[email protected] asking about this, but the answer might be to launch my own Lemmy instance and retrieve all of the Lemmy posts so I can browse the ones that interest me without leaking any info. I’m sort of in a position to do that, but most people unfortunately aren’t.
What do you mean with “tracking” exactly? The way I understand it, tracking is analysing and using user data, for example for marketing purposes.
Posts and content need to be saved on the instance as far as I understand, I don’t see any other way. And posts and comments are essentially public information, anyone can see the posts that your username posts and comments, that’s kinda the entire point of posting and commenting.
Up and down votes too, otherwise I don’t see how the concept of up and down votes could work. The server needs to know which comments or posts you upvote, otherwise it doesn’t register it. And theoretically, the server admin could track that information and make statistics based on it, although this is potentially where legal issues come in if it’s not properly explained what is done with your data.
Same with metadata stuff and data such as which posts you access/read. The server has to know that information, when you click on a post you want to read, you are essentially asking the server to provide you with that post, so the server has to know which post you want to read and this is generally logged on the server for a certain time.
The question is does the server keep and archive this information and/or is this information used and analysed by somebody.
According to the admin, data is not sold or used for marketing purposes.
Tracking of reads = when you read someone’s post, there is a permanent record made, e.g. in a db row associated with the user, that @aski3252 read that post. That is somewhat different from normal httpd access logs that associate only with IP addresses and which typically get distilled down to aggregate data, and prefeferably discarded after a short period. Where I worked, we kept logs around for 30 days for stuff like abuse investigations but deleted them after that. In fact with a little careful design of the log data, or if the query is sent by HTTP POST instead of GET, the parameters that identify what you were reading will usually not be logged at all.
It’s not mostly an issue of selling data for marketing purposes. The data could also be extracted by cyber attackers, seized by law enforcement, subpoenad in a lawsuit, or whatever. The only way to stop that from happening is to not retain the data in the first place. “Marketing purposes” is a smoke screen anyway. E.g. if you are a regular lurker on a community about workplace organizing or job hunting, that info will be more valuable to your boss than it will be to some advertiser or marketer. So the real customers of internet usage data (and phone records etc.) are far less benign than “marketing” organizations.
It is not necessary to record voting data except to prevent you from voting twice on a particular topic. So if voting closes (say on a poll), all the data about who voted in it can be deleted. There is also no need to remember HOW anyone voted. It’s enough to remember that you voted on a particular topic, and increment the relevant vote counter. That is also how real-world elections work. See also the topic of “receipt-free voting” in cryptography.
I agree with you that if you actually publish something on the site, there is a certain amount of disclosure unavoidably associated with that.
First of all, just to be clear, I’m not at all an expert on this topic for those who haven’t noticed. My questions are mostly because I want to learn how it works, not because I want to tell you that you are wrong or anything like that. You seem to know a lot more than me anyway.
Tracking of reads = when you read someone’s post, there is a permanent log record made
When you read someone’s post, you first need to access that information from the server. In order to do that, your client tells the server which post you want to see and the server sends you that post. Those interactions are most likely logged on the server as well as which IP address has requested that information, etc. There is no absolute sure way to make sure that the admin does not use those logs to extract that information, at the end of the day, it comes down if you trust the admin.
But there is also a “show read posts” option which seems to hide read posts overall, which does indeed suggest that read posts are saved and used and which seems to work independent of client.
It’s not mostly an issue of selling data for marketing purposes. The data could also be extracted by cyber attackers, seized by law enforcement, subpoenad in a lawsuit, or whatever.
Sure, I do get the issue to some extend, but I don’t see how it is quite as bad as you seem to imply. For example, I worry more about personal data, such as my e-mail address being leaked, which is why I generally use a throw away email. I don’t really see why I, or some attacker, should care about which posts I have “read”, but maybe I don’t understand the full implications getting this information means.
“Marketing purposes” is a smoke screen anyway.
Of course it is, but I don’t think there are any lemmy instances that use lemmy data for marketing purposes. Data seems to be used only to improve the user experience, at least that’s how it’s intended.
It is not necessary to record voting data except to prevent you from voting twice on a particular topic.
If it wasn’t logged or only logged client side you could upvote/downvote infinitely, no?
There is also no need to remember HOW anyone voted. It’s enough to remember that you voted on a particular topic, and increment the relevant vote counter. That is also how real-world elections work. See also the topic of “receipt-free voting” in cryptography.
Yes, I understand how web servers work (I have implemented them) ;-). I’ve also been involved in abuse investigations that involved crunching of 100s of GB of raw logs. If I wanted to figure out what posts you had read based on raw http logs, it would be a big pain in the neck involving matching your user ID with IP addresses, and trying to match HTTP queries with posts, where the relevant log entries were scattered through billions of similar entries from other people. Last time I did something like that, the analysis took about 15 hours on a quite big server, though that particular task also had to find groups of queries corresponding to login sessions. While if there’s a database table that identifies every post that has been read by every user, all I have to do is type some SQL and the info comes up immediately.
As for the invasiveness of that info, don’t you have any private life at all? Are you pro-XYZ about some political question while your boss is rabidly anti-XYZ? You probably don’t want him to know what you’re reading. Same if you’re getting sued by someone trying to dig up dirt on you, or if you are running for some kind of office (look at all the NSFW content aski3252 reads on Lemmy! Sinner!!!, etc). Or say you are in a country where some dictator gets into power and decides to round up all the Star Trek fans. You suspected something like that was coming, so you carefully avoided posting in the Star Trek communities, but unfortunately you were reading them and now you have been found out. Just use your imagination ;).
Re voting, let’s say there is a poll “Is Spez an idiot? Vote yes/no, poll closes on July 1”, and you vote in it. To stop you from voting twice, the server must remember until july 1 that you voted, but not how you voted. After July 1, it is impossible to vote again, so the info that you voted at all can be deleted. What currently happens instead seems to be that “aski3252 voted yes” is retained forever. There are some minor UI benefits to that, so I described it as iffy rather than outright evil. If it were up to me though, I would minimize the amount of info kept.
I noticed previously that stuff I read in the browser does not show up as read in the mobile app. I also just tested it with different browsers and as far as I can see, read posts are marked as unread when I use another browser.
So are you actually sure about your claim? This is very easily testable, so I hope you have actually confirmed this before you accuse lemmy of participating in a “terrible invasion” of privacy…
I have said several times that I am not completely sure. I will see if I can do some better tests. It is something that I noticed when I installed Jerboa, so I asked about it, and people seemed to confirm that there was server side tracking.
Anyway, even if it’s confirmed, unless there is deception involved (which I have no reason to suspect), there’s not much of an “accusation” to be made. I would say, in the event that individual post views really are saved on the server, that Lemmy’s designers made a policy choice that I don’t agree with. I’d call that a description rather than an accusation. I’d try to open a discussion about getting the decision changed. If that didn’t succeed, I’d look for technical workarounds and/or limit my reading on the site.
No prob, and I’ll go a little further, from having seen this kind of thing many times before. Lots of times these info leaks happen because it was technically convenient or somehow useful to do X, without thinking through the privacy implications. Security vulnerabilities happen the same way. People just want to get their thing done with minimum fuss, rather than dithering around weighing complicated tradeoffs. So X is not explicitly a policy decision at all, but instead is a technical decision that turns out to have policy implications.
I’m a security developer so I have to be attuned to this kind of thing, but I miss stuff too, as does everyone. Most of the time nobody is being “bad”. They are just trying to ship product in a complicated environment full of subtle interactions, and it is easy to overlook stuff, especially if you haven’t already spent a lot of time dealing with those same issues.
I can’t say that the backends don’t track that for sure because I haven’t looked at the source or anything. But keeping a history is something very commonly done in the client. Just like Web browsers.
Right, what I saw (unless I’m mistaken which is possible) was reading posts on one client (Firefox browser on my laptop computer) and then seeing the read posts marked on a completely different client (Jerboa on my phone). That means the info must have somehow been communicated between the two clients. Suspicion points to the server. I will ask on /c/[email protected] about this and/or look at the code base.
I browsed and posted on Lemmy for a while through a desktop browser on my laptop, then installed Jerboa on my phone and started playing with it, and immediately noticed that posts I had previously read through the browser were marked in Jerboa. The only ways Jerboa could have gotten that info are: 1) the server recorded the info from the browser and relayed it to Jerboa, or 2) I was confused somehow and had also read those posts through Jerboa.
#2 above is something of a possibility but that leaves #1 as still not dispelled suspicion. I was hoping that someone familiar with the implementation would comment.
I installed Jerboa and noticed that it grayed the titles of posts that I had already viewed even though I had viewed them on the web. That told me (unless I am somehow confused) that the server side tracks what posts you have read.
From my perspective that seems like a terrible invasion. I can understand some benefit to showing the post status in the UI, but if it is stored at all, the storage should be exclusively on the client side. I mentioned this also in the “issues” thread and got no reaction, so maybe I’m missing something or in error.
Reddit tracked posts viewed as well.
Thanks, that is good to know, but that is a type of evil where I would hope Lemmy doesn’t follow Reddit. I sometimes posted to Reddit but I more often read passively without logging in, partly to avoid some of the tracking.
Sure. I think it’s good to be aware of for sure and I agree that it would be nice if Lemmy isn’t tracking as much. I also recognize that I’ve accepted a certain amount of tracking in my life over the years at this point.
Tracking posts is understandable. Tracking up and down votes is iffy. Tracking reading is inappropriate and invasive.
I don’t disagree really… Just pointing out that as long as you’re logged in Reddit has always tracked posts viewed as far as I’m aware… Facebook similarly tracks all activities and always has.
These are obviously not models to aspire to but I think that it’s helpful to be aware of what we’ve dealt with up until this point.
Facebook was notoriously evil and I actuallly have all their domains that I know of blocked from my computer at the DNS level in order to avoid their spying. That Reddit tracks posts viewed is new to me but I guess not that surprising. Usenet never tracked posts viewed and basically couldn’t. Wikipedia emphatically doesn’t track that, though it doesn’t track view counts. Arxiv.org doesn’t track (or at least publish) view counts for individual papers (see here) though they do publish stats about the entire site. A real privacy focused site would avoid publishing any about what viewers are doing. There is a whole topic in cryptography called private information retrieval about how to run a server in which the clients can verify that the server can’t know what they are reading. This is what Lemmy should aspire to, imho. (Aspirations aren’t meant to be achieved literally, but only to provide guidance).
I may open a thread in /c/[email protected] asking about this, but the answer might be to launch my own Lemmy instance and retrieve all of the Lemmy posts so I can browse the ones that interest me without leaking any info. I’m sort of in a position to do that, but most people unfortunately aren’t.
What do you mean with “tracking” exactly? The way I understand it, tracking is analysing and using user data, for example for marketing purposes.
Posts and content need to be saved on the instance as far as I understand, I don’t see any other way. And posts and comments are essentially public information, anyone can see the posts that your username posts and comments, that’s kinda the entire point of posting and commenting.
Up and down votes too, otherwise I don’t see how the concept of up and down votes could work. The server needs to know which comments or posts you upvote, otherwise it doesn’t register it. And theoretically, the server admin could track that information and make statistics based on it, although this is potentially where legal issues come in if it’s not properly explained what is done with your data.
Same with metadata stuff and data such as which posts you access/read. The server has to know that information, when you click on a post you want to read, you are essentially asking the server to provide you with that post, so the server has to know which post you want to read and this is generally logged on the server for a certain time.
The question is does the server keep and archive this information and/or is this information used and analysed by somebody.
According to the admin, data is not sold or used for marketing purposes.
Tracking of reads = when you read someone’s post, there is a permanent record made, e.g. in a db row associated with the user, that @aski3252 read that post. That is somewhat different from normal httpd access logs that associate only with IP addresses and which typically get distilled down to aggregate data, and prefeferably discarded after a short period. Where I worked, we kept logs around for 30 days for stuff like abuse investigations but deleted them after that. In fact with a little careful design of the log data, or if the query is sent by HTTP POST instead of GET, the parameters that identify what you were reading will usually not be logged at all.
It’s not mostly an issue of selling data for marketing purposes. The data could also be extracted by cyber attackers, seized by law enforcement, subpoenad in a lawsuit, or whatever. The only way to stop that from happening is to not retain the data in the first place. “Marketing purposes” is a smoke screen anyway. E.g. if you are a regular lurker on a community about workplace organizing or job hunting, that info will be more valuable to your boss than it will be to some advertiser or marketer. So the real customers of internet usage data (and phone records etc.) are far less benign than “marketing” organizations.
It is not necessary to record voting data except to prevent you from voting twice on a particular topic. So if voting closes (say on a poll), all the data about who voted in it can be deleted. There is also no need to remember HOW anyone voted. It’s enough to remember that you voted on a particular topic, and increment the relevant vote counter. That is also how real-world elections work. See also the topic of “receipt-free voting” in cryptography.
I agree with you that if you actually publish something on the site, there is a certain amount of disclosure unavoidably associated with that.
First of all, just to be clear, I’m not at all an expert on this topic for those who haven’t noticed. My questions are mostly because I want to learn how it works, not because I want to tell you that you are wrong or anything like that. You seem to know a lot more than me anyway.
When you read someone’s post, you first need to access that information from the server. In order to do that, your client tells the server which post you want to see and the server sends you that post. Those interactions are most likely logged on the server as well as which IP address has requested that information, etc. There is no absolute sure way to make sure that the admin does not use those logs to extract that information, at the end of the day, it comes down if you trust the admin.
But there is also a “show read posts” option which seems to hide read posts overall, which does indeed suggest that read posts are saved and used and which seems to work independent of client.
Sure, I do get the issue to some extend, but I don’t see how it is quite as bad as you seem to imply. For example, I worry more about personal data, such as my e-mail address being leaked, which is why I generally use a throw away email. I don’t really see why I, or some attacker, should care about which posts I have “read”, but maybe I don’t understand the full implications getting this information means.
Of course it is, but I don’t think there are any lemmy instances that use lemmy data for marketing purposes. Data seems to be used only to improve the user experience, at least that’s how it’s intended.
If it wasn’t logged or only logged client side you could upvote/downvote infinitely, no?
That does seem to be a good point.
Yes, I understand how web servers work (I have implemented them) ;-). I’ve also been involved in abuse investigations that involved crunching of 100s of GB of raw logs. If I wanted to figure out what posts you had read based on raw http logs, it would be a big pain in the neck involving matching your user ID with IP addresses, and trying to match HTTP queries with posts, where the relevant log entries were scattered through billions of similar entries from other people. Last time I did something like that, the analysis took about 15 hours on a quite big server, though that particular task also had to find groups of queries corresponding to login sessions. While if there’s a database table that identifies every post that has been read by every user, all I have to do is type some SQL and the info comes up immediately.
As for the invasiveness of that info, don’t you have any private life at all? Are you pro-XYZ about some political question while your boss is rabidly anti-XYZ? You probably don’t want him to know what you’re reading. Same if you’re getting sued by someone trying to dig up dirt on you, or if you are running for some kind of office (look at all the NSFW content aski3252 reads on Lemmy! Sinner!!!, etc). Or say you are in a country where some dictator gets into power and decides to round up all the Star Trek fans. You suspected something like that was coming, so you carefully avoided posting in the Star Trek communities, but unfortunately you were reading them and now you have been found out. Just use your imagination ;).
Re voting, let’s say there is a poll “Is Spez an idiot? Vote yes/no, poll closes on July 1”, and you vote in it. To stop you from voting twice, the server must remember until july 1 that you voted, but not how you voted. After July 1, it is impossible to vote again, so the info that you voted at all can be deleted. What currently happens instead seems to be that “aski3252 voted yes” is retained forever. There are some minor UI benefits to that, so I described it as iffy rather than outright evil. If it were up to me though, I would minimize the amount of info kept.
I noticed previously that stuff I read in the browser does not show up as read in the mobile app. I also just tested it with different browsers and as far as I can see, read posts are marked as unread when I use another browser.
So are you actually sure about your claim? This is very easily testable, so I hope you have actually confirmed this before you accuse lemmy of participating in a “terrible invasion” of privacy…
I have said several times that I am not completely sure. I will see if I can do some better tests. It is something that I noticed when I installed Jerboa, so I asked about it, and people seemed to confirm that there was server side tracking.
Anyway, even if it’s confirmed, unless there is deception involved (which I have no reason to suspect), there’s not much of an “accusation” to be made. I would say, in the event that individual post views really are saved on the server, that Lemmy’s designers made a policy choice that I don’t agree with. I’d call that a description rather than an accusation. I’d try to open a discussion about getting the decision changed. If that didn’t succeed, I’d look for technical workarounds and/or limit my reading on the site.
Thanks for your clarification.
No prob, and I’ll go a little further, from having seen this kind of thing many times before. Lots of times these info leaks happen because it was technically convenient or somehow useful to do X, without thinking through the privacy implications. Security vulnerabilities happen the same way. People just want to get their thing done with minimum fuss, rather than dithering around weighing complicated tradeoffs. So X is not explicitly a policy decision at all, but instead is a technical decision that turns out to have policy implications.
I’m a security developer so I have to be attuned to this kind of thing, but I miss stuff too, as does everyone. Most of the time nobody is being “bad”. They are just trying to ship product in a complicated environment full of subtle interactions, and it is easy to overlook stuff, especially if you haven’t already spent a lot of time dealing with those same issues.
Since there’s a setting in your user profile to show or not sure read posts, it’s clearly something tracked.
I can’t say that the backends don’t track that for sure because I haven’t looked at the source or anything. But keeping a history is something very commonly done in the client. Just like Web browsers.
Right, what I saw (unless I’m mistaken which is possible) was reading posts on one client (Firefox browser on my laptop computer) and then seeing the read posts marked on a completely different client (Jerboa on my phone). That means the info must have somehow been communicated between the two clients. Suspicion points to the server. I will ask on /c/[email protected] about this and/or look at the code base.
Isn’t that a browser thing, not a lemmy thing? Iirc, your jerboa history shows up in your default browser
I browsed and posted on Lemmy for a while through a desktop browser on my laptop, then installed Jerboa on my phone and started playing with it, and immediately noticed that posts I had previously read through the browser were marked in Jerboa. The only ways Jerboa could have gotten that info are: 1) the server recorded the info from the browser and relayed it to Jerboa, or 2) I was confused somehow and had also read those posts through Jerboa.
#2 above is something of a possibility but that leaves #1 as still not dispelled suspicion. I was hoping that someone familiar with the implementation would comment.
Interesting! I’ve only used lemmy on mobile, so haven’t had that experience.