I gather that it had a use unrelated to Lemmy but for Lemmy posts I make a case here that it is more misleading than helpful. For a moment, please ignore the underlying reasons why things are the way they are and focus on how the issue presents to the end-users.
(1) By pulling in solely the post, but not any of the comments, it at best provides only partial information - which if all you wanted to read was the post, then why bother pulling it here at all? (as opposed to retrieving from its original location - I mean, to do it you already need the full URL…) While if instead you wanted all of the comments… - e.g. to be able to reply to - then too bad, b/c it won’t do that?
(2) It also does not pull in any of the old vote counts. So if hypothetically a post had 1000 upvotes, and then after pulling it here it received adjustments +2 from upvotes and -4 from downvotes, then its total would then be 998, right? Except PieFed would instead display “-2”, a qualitatively different score for a highly popular post that is a terrible misrepresentation of the actual facts about it.
(3) It conveys a distorted view of things to the end-users. e.g. see [email protected] where there are 6 posts from the last 2 months, right? Right?! No, there is actually only a single post there in its entirety, then a few more that I and what I assume was Blaze pulled in - note how those other 5 have zero comments, and total scores near zero, due to the aforementioned issues. Really the “earliest” post that PieFed.social reliably has from that community is from 4 days ago, and then beyond that is a scattered, partial mess. There are actually MANY more posts from the last two months, which are not represented here. Ergo, the initial impression that a quick glance at this community offers turns out to be false, due to these federation issues.
(4) showing only partial information is often called a “false positive” or type I style of error, whereas showing nothing at all for those posts that are not fully here avoids that pitfall. If certain content is not here then… well it is not here, and that’s that, but for only some of it to be here leads to much confusion, imho.
Almost entirely distinct from this issue, the ability to find an existing post given its URL should be added to the search menu, b/c that is where people will go to find it. But ofc all the more so if the retrieval button is removed or made less prominent, so that that find ability is not lost along with that.
I understand that there are hard limitations of the federated model itself. So if e.g. older comments and votes cannot retroactively be pulled in - or possibly even if so - then maybe this function should just be abolished? Or perhaps a couple more layers of “are you sure you want to do this?” added, or better yet moving it from its prominent place showing up to everyone on almost every page to a more subdued location where only those who know what it is and what will happen if it is used are likely to access it? I now feel that I actively made the situation in [email protected] worse by pulling in those posts, and wished now that I hadn’t done so, as it could lead people astray into thinking “this is all the posts that the community has to offer from this time-period” (NO, it actually has MANY MANY more than that, on the original server!?!!). Now that I know this I can refrain from using it, but it would be nice to help others who climb this ladder after me as well:-). So I am sharing my thoughts with you in case that helps.
PieFed is freaking awesome and you all who work on it are magnificently extraordinary to share your knowledge with the world:-).
It’s difficult. Lemmy doesn’t supply a list of replies when we retrieve the post over ActivityPub, nor votes. We could probably get that data by calling their API but that feels wrong.
The fediverse has developed a convention of pasting urls into the search function in order to retrieve a post but I don’t think it’s intuitive for new users to do that; a search tool is for searching, not importing. It’d be great to find a better way to integrate this feature.
It feels like it should be OK for piefed to implement an extra “proper import” feature somewhere, which actually goes and retrieves post with full comments and votes. I don’t think any instance admin would have an issue with it. I certainly don’t care unless someone finds a way to turn this into a DDOS (which can be dealt with rate limits)
(& [email protected] - we’ll see if notifications like this work here as well!:-P)
Yes [email protected] your idea would be better than it is now - the benefit would be to allow someone to reply to comments on that post, with an account based on PieFed rather than needing to create one on a remote instance. Although even Lemmy does not have this capability (afaik?) to retrieve posts from a remote instance, so there is no “expectation” that it be provided.
Just to reiterate: if the retrieval function was to either be enhanced to work fully & properly, or else to go away entirely - or at least be de-emphasized (right now it seems available from almost every page across the entire piefed.social site?) so that common users (like me!:-P) don’t just play around with it willy-nilly and generate such partial (mis-)information postings - then either way that would solve the problem.
The problem, imho anyway, is the presence of the partial information, which is so confusing to people. But again: if the retrieval function was simply not present at all, that does not seem all that “bad” to me? (especially since, again afaict, Lemmy lacks it too, at least for non-admins)
And I can’t really speak to Rimu’s comment about whether it’s “cheating” to use the API, except naively to say that doing something all of once upon a special request each time might not be all that bad? Though that’s still code to have to write, test, deploy, investigate for potential exploits, etc., and I am not sure how commonly people would really want to make use of it? Although if they did, I could see someone wanting to get e.g. all of the posts for the past several months for a community that nobody has previously subscribed to on a PieFed instance. But that’s a lot of effort for seemingly little gain? Purely from an end-user perspective, I would rather see things like adding a Preview option to comment replies and/or making the replies to comments be in-line in the sam epage (especially since after you do something, like perhaps editing a reply, the web UI deposits you to a different page than where you started, and you have to go hunt for the reply you just worked on, possibly having to delve deeper into the comment chain i.e. so that a browser “find” may not work, just to see how your comment ended up being rendered; e.g. I’ve made comments where the image was not rendered correctly, or just this morning I noticed that my mobile browser had put a space between the [url display text] (https://actualurlhere.com) - so this is a common item that I end up having to deal with multiple times a day and sometimes multiple times per hour; while in contrast, I virtually never need to see an old post and reply to it; so while the presence of the misinformation is confusing and I hope can be dealt with sooner, the expansion of the retrieval function is a significantly lower priority issue - again, just imho, in case it helps to hear from a fresh POV:-D)
I tried replying to you a bit lower down in the thread - see where I tried to mention you in my response to [email protected].
It did not work well at all:-(. I see this error when I click on the resulting link (which PieFed seemed to render as https://piefed.social/community/lookup/rimu/piefed.social):
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
The processing for Mentions hasn’t been added yet. But when it is, it’ll likely follow the convention of letting users add them beginning with an ‘@’. Beginning stuff with an ‘!’ is the convention for communities, so when you tried to Mention Rimu using one, it tried to look it up as a community.
Thanks! So like @[email protected] then… eventually:-).
I feel it’s the other way around - there’s no way to get comments from federation but perhaps screenscraping could be added for that, so that’d be a plus.
I like this feature (and I extended it on my local instance so it can refresh and pull down all the latest posts from a community, in case federation is delayed or something). Grabbing comments would be nice too.
(Votes would be tougher and probably not worth it. I guess it could screenscrap a total count and then create dummy actors for the individual vote counts or something…)
Oh I never replied to this - I must have gotten distracted, sorry for ignoring you (unintentionally I assure you:-).
I agree that it is nice to “view” a post in its entirety from another server, but what is the benefit to pulling the post into here? Lemmy solves the former issue by adding the rainbow-colored federation icon to everything - posts, comments, users, all of it will go to the original instance. PieFed makes it quite difficult imho to do all of that. There is an EXTREMELY buried “View community on original server”, which actually looks like it is somehow tied to the list of moderators, and only found on a community page. There is no way afaict to go directly to an original actual post though, or comment. There is also a way to go to a users’ home instance, but not to the actual user account itself, again afaict.
I would love to see a retrieval function that puts the post into a temporary location and allows someone to “view” it for awhile, before discarding it later (a week? a month?). But there isn’t much benefit to this, since even without an account you can view any OP from its home instance already - so it’s a lot of work for very little gain.
But to have it “here” - I think the benefit would be that you could respond to the comments within it? Yes, that may be worth the disadvantages that I mentioned. Perhaps then it could be indicated with an icon, or a horizontal separator between it and the other more recent ones, to distinguish it as something pulled in specially rather than having arrived the usual way - e.g. if you look through a community and see posts from, let’s use a hypothetical scenario, “the last 2 months”, and then all of a sudden there is a jump and a single post from “2 years ago” appears, it may help to show that discontinuity? But anyway yes, being able to respond to comments in it does add functionality, so that I like.:-)
(Edit: except if the vote counts are off - that could be quite damaging? e.g. a post with -2 total score is VERY different than one with +998, so it tells an entirely different story in terms of community receptivity to that idea. I would rather see no score at all, like literally perhaps an “n/a”, rather than see incorrect values there. Lack of information is fine - so long as it is labelled - it is misinformation that is what causes the damage in terms of how a message is received, differently than what was intended originally to have been conveyed.)
No worries about the delay. Once a reply to me from someone didn’t federate and I didn’t see it by chance until a month later - see https://lemmy.world/comment/12146899
I think you are asking for something different actually. This feature that I like and use is different from what you’re asking - in some ways they’re the exact opposite. But at the same time they aren’t mutually exclusive - in fact I’d say they are complimentary.
But if I’m understanding correctly, it’s easy enough to add into piefed the "See original post at " thing. For a rough job, I just put this line below
[HTML_REMOVED]See on original instance at [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
above this bit, https://codeberg.org/rimu/pyfedi/src/branch/main/app/templates/post/_post_full.html#L2
And then every post on my instance will take me to the original author’s instance. (There’s probably a bit of tweaking to do here - for example I put it above the “Home > Communites…” line as a quick hack which probably isn’t the most user friendly UX.)
I put a small diff file at the end of this post as well.
But - back to the original feature - I think you’ve got exactly what the benefit is. Here’s a real life example of how it was useful:
For a while, I had issues with federation that was causing posts from lemmy.world not to federate to my instance for three whole days. But with this option, I could manually load in posts into my instance and add comments (and for some reason the replies to those comments tended to come in right away). Without this option I wouldn’t have been able to comment at all unless I waited, or used an account on a difference instance.
I’m fine with storing extra metadata so the post original (typical federation vs being manually pulled in - and perhaps a third for posts that get backfilled when a new community is first added?). In fact your suggestions on displaying this to the end user, like a special icon, are good ones.
But keep in mind that the vote discrepancy already exists on backfilled posts, and note that it could also happen with something that arrives through normal federation.
(E.g. Beehaw defederated from lemmy.world at some point but AFAIK piefed.social can still talk to Beehaw. Then up/down votes from users on Beehaw won’t make it into the vote count on the version of the post on lemmy.world but would make it to piefed.social - which might not be that big of a deal if it’s just a couple of users, but if the difference was massive then we’d have exactly the problem you state.)
We wouldn’t be the first to notice this either. See for example https://discuss.tchncs.de/post/1907501
diff --git a/app/templates/post/_post_full.html b/app/templates/post/_post_full.h
tml
index 3467929…df06190 100644
-– a/app/templates/post/_post_full.html
+++ b/app/templates/post/_post_full.html
@@ -1,4 +1,5 @@
[HTML_REMOVED]
+ [HTML_REMOVED]See original on [HTML_REMOVED]{{ post.ap_id }}[HTML_REMOVED][HTML_REMOVED]
{% if post.type == POST_TYPE_IMAGE -%}
[HTML_REMOVED]
[HTML_REMOVED]Part 1 of 2.
In the past I have used the rainbow-colored Fediverse icon for things like a quick short-cut to the original instance, to check things like the community or instance rules, but mostly ever since Lemmy started being able to read the static.wikia types of images, I haven’t needed it anymore, now that all instances display more or less identical content for a given post.
With the exception of that known Lemmy.World federation issue. What happened there, from what I hear, is that the Fediverse was never meant to have a singular server holding 80% of the entire userbase, and to have every community be joined by someone on every instance (which Blaze has been doing with his army of alts, to aid newcomers who couldn’t figure out how to join a community that nobody else on the instance had already joined yet). Read more here: https://aussie.zone/post/13429731 (github issue linked from there). TLDR: the issue is already solved (probably) but is in the 0.19.6 version, which Lemmy.ml is testing but until Lemmy.World upgrades to it this issue will continue. I’ve seen it happen on piefed.social, but also startrek.website, discuss.online, and other servers as well.
But in general, I was not asking to be able to view every single post from its original server (though again, that would be somewhat useful, occasionally), nor even being able to pull in a post when it is not there, but to make the pulling in of a post be either full or else none. Here is a simple mathematical equation that may perhaps help: 1 + 1 = 2 is true, and 1 + 1 = ? could also be true, but 1 + 1 = 1 is not. The first one is fully and precisely true, the second is vague and unsatisfying that someone doesn’t know the answer… but at least it is still technically correct. Whereas the third answer is just flat false.
It sounds like from what you and others are saying that it is literally impossible to be able to get the answer of “2” via the current methods of Federation. Okay then, well that seems to inform our answer: if we absolutely cannot get the proper answer of “2”, then we should put “?”, but in no case should we (imho at least) show “1”, just b/c after we pull in a post it receives +1 upvote. “x+1=1” is invalid for nearly all values of x, except the one that happens to be true, and yet ALL of the others are false, hence we should not shorten x+1 to merely say “1” as if 1 + 1 = 1 were a correct statement.
Alternatively, which depending on interrelation with other reasons might be the better way even, perhaps we should simply disallow the pulling in of posts altogether - if they cannot be done “properly” then perhaps to avoid misleading aka false statements, it should not be done at all?
Still further alternatives could be to (1) pull in “ghost” or “shortcut” versions of the post, which are displayed VERY noticably different than “real” ones, and which when clicked go to the original server, and which also have something like “?” or “n/a” rather than a vote count; or (2) similar to 1 but which when clicked show all of the comments, those having been pulled in via some other means (API?) but there too the vote count(s?) should not be displayed, unless they are known to be accurate.
Other matters also interrelate with the above though. e.g. I notice that piefed.social has defederated from hexbear.net (https://piefed.social/instance/hexbear.net does not resolve and https://piefed.social/instances?search=hex&submit=Search looks empty) - so if any votes can from that instance, those should not be counted. Web scraping would include them though, and others e.g. those from people who piefed.social (or any PieFed instance, just picking on that one:-) has banned. Perhaps this is a minor point, but it does touch on how those vote counts are interrelated to other matters, which if brought in via “alternative” means will make it harder to figure out what to display for them. And… oops, I see that you already got to the reverse of that point, where Beehaw would add vote counts, beyond what e.g. web scraping would be able to pick up. Well, nothing is fully perfect, but I did want to suggest that we avoid any KNOWN sources of misinformation. Some corner cases like that… can be saved for perhaps another day when further developing this ActivityPub implementation makes more sense, compared to other more urgent things that need to be worked on in the meantime. i.e., especially if something is also a problem for Lemmy, then it seems understandable to me that if it is likewise a problem for PieFed, then there is little “expectation” that we would do better than them, in every single possible way?
Damn, I’ve hit a space limit.
Part 2 of 2.
That said: I thought that we have a problem that Lemmy does not: we allow users to pull in older posts. But your linked example seems to disprove that - do you know how they got those posts then, even though they are “old”? Or perhaps I am misunderstanding something: we CAN get older posts, but we usually choose NOT to, UNLESS someone specifically requests the pulling in of such a post. Damn it does get complicated:-). In general though, I hope you see what I meant to convey: if we can only get new posts and cannot get old ones accurately, then perhaps it is better not to try to get those old ones (except as ghosts/shortcut links)?
As for existing backfilled posts - yeah, it is unfortunate, but it is what it is and I don’t think “backwards compatability” to existing entries is our primary concern right now? If it were, they could perhaps be deleted - if we could even figure out which ones they are? - or left alone. A year from now they will confuse the heck out of people… but that can be an adventure for them to puzzle over, so long as things work properly from here forward - do you agree? :-D
Anyway it’s just something fun to think about:-).
Hey, thank you for this engaging set of posts - and the entire post and thread, really. It really opened my eyes up and made me think hard - exploring a new mental space, in Star Trek speak ( see https://memory-alpha.fandom.com/wiki/All_Good_Things…_(episode)#Act_Ten for the reference - it happens near the end of the episode)
I think I prefer 1 or the ? version. Going with 2 means I can’t interact with a post that fails to federate (or is older than my instance) at all. With 1 or ? I can. Being able to backfill comments would be icing on the cake.
From the linked post of https://discuss.tchncs.de/post/1907501 it seems the standard bearer is just what the home instance of the community has for that post. So comments and votes made to a post after defederation do not count. This makes sense - if you can’t trust the other instance, probably you can’t trust their admins either - and it’s easy for an admin with database access to add fake comments and up/downvotes, etc. So we just pick the obvious source of truth and roll with that.
One small issue here - from screenscraping we can get the vote total count but maybe not the individual votes. We wouldn’t know who voted which way. That’s not a big deal - we can just make up a “[email protected]” ap id to own the vote in the backfilled cases, but I think it makes the case for ? rather than 1.
(As an aside I don’t think Lemmy does backfilling, only pyfedi does it - as the top comment says, “just… it does not backfill previous content and votes.” But the post itself I think must get picked up when a new comment or up/downvote gets seen.)
Damn I loved that Star Trek episode - it was so great! It’s so easy to just keep doing the same old thing - e.g. to explore the stars, but to join a card game - THAT would be a NEW adventure, surely.
But as for the possible, one idea would be to make a hard separation: let’s say a PieFed user joins a new community XYZ. As of that very moment, all newly made posts along with all votes come in, and everything that can make it there through any federation networking issues are reflected in that posts, BUT all old posts do not show up - even if new votes or comments are sent, those are simply discarded - and perhaps a message is displayed “there are no more posts available from this community”. So everything NEW is fully there (to the extent possible/feasible), while everything OLD is fully missing (it would have been only partial/misleading information anyway), and this way there is nothing in-between.
At least, it’s a thought:-). Surely it would not please everyone, but it might be a least worst option?
Err - so basically do what Lemmy does?
Like I said, that wouldn’t work for me. But then again, I’m an admin of my single user instance. And I can see the needs of “real” instances - ones trying to build a community of many users - being different from mine.
So maybe allow it as an admin toggle, default to Lemmy behaviour. That would seemingly cover everyone - and the admins on piefed.social can manually pull in an older post/comment if there’s a compelling reason to make an exception (and add the cautions about possibly inaccurate vote counts and such).
No (I think?) - I thought your link showed that Lemmy was showing inaccurate information as well. So what I was proposing was perhaps better than what Lemmy does, in the sense that Lemmy would pull in the additional posts upon a new voting occurrence, but PieFed could block that and either show only posts for which fully accurate information is available, and then discard those where the extra (voting) information is not available, or else it could go a step further and go ahead and show those posts that only have partial information but make sure to label it specially to indicate that. Lemmy does neither of those.
Second issue: right - an admin should be able to, and maybe there’s no real reason to prevent regular users either, but it shouldn’t be so common, so it would be nice to move the link to do that somewhere else so that new users (like me!) don’t get confused about what it means and is supposed to do and pull things in on a whim without knowing the factors involved.
