Forum "Spring" Cleaning - Operation Broomstick

All this terminology I dont understand made me think I was in the FFXIII thread for a second.

4 Likes

Nah, those terms make sense!

2 Likes

Small update for everyone, I have worked out how to get the text to appear as a quote… kinda! It won’t link to the original post but it does format a LOT nicer so that helps.

This is the post as is:

If you remove the highlighted text only here

then the first quotes before the username, then everything after the second quote to the </p

You get this:

It’s not perfect, but it’s a LOT nicer. So if anyone is doing this, please follow this guide and we can get some quotes back :slight_smile:

Ah that’s cool I was also looking at the quote stuff in the Tampermonkey script, I’ve got an updated version up now that fixes quotes.

I have also added a thing that just strips <p></p> tags from the post, as I found that if you remove those all the italics and stuff seem to work. I’m just experimenting here but in all my posts, the original line feeds are still present so removing the p tags doesn’t appear to mess up formatting at all.

1 Like

Well I’m not techinically inclined at all, but I was able to install your script @kez and just did a successful test run. Works great so far! Thanks!

2 Likes

Glad it’s working for you! Let me know if there’s anything else you’d like to make it do. :)

2 Likes

I’ll run a bit more today to see what I can find that can be removed. One thing in the future I’d like to do is clean up image files that are stored on imgur, but that’s proving quite tough to confirm exactly what lines relate to the image and are not from the formatting. Still bet you can do it :wink:

2 Likes

I’m guessing you mean the ones like this?

I have added something to the script that converts those into working images (though I have a hazy memory of looking into this in the past and hitting roadblocks…)

But I don’t think any of my posts have that issue so I can’t see what it looks like with all the html tags etc. Could you provide an example?

1 Like

More like this post: Show me things you made. - #97 by jaws

When editing it, you can see the HTML that was used and the results

But when you update it it appears as above with a broken link:

When you view the post to edit, the html coding from Flarum had different actions, and when you delete everything except the URL it then breaks the link to the image. Now on Discourse images are pasted and stored on the server, so no external hosted is needed but to do that it’s very manual.

I’ve found it in a few posts when editing the @ issue. If you can see exactly what can be removed to make it work please let me know. I’ve tried a few options and revert back to copy/pasting the image. There’s a lot of fine tuning issues within the text, I’m making notes of things I see but working through one step at a time, save breaking one thing by fixing another.

1 Like

Hmm I see! Well let’s see if I can get that image to display first of all, gonna try a couple variations!

Tough one huh!

Yeah indeed, this is all coming back to me now. It works fine in the preview but once you submit, it breaks.

Exactly! There’s one specific line that works. Except when it doesn’t work.

This is know in the scientific world as bollocks haha

2 Likes

Here’s a perfect example of a post which needs a LOT of editing:

And while I am focusing on the @ issue I want to do what I can while I’m there, as it’s only a few seconds more now to get it done. I can’t think of a way of doing it automatically as it’d cause errors somewhere else.

Okay, so I did look into this previously. This issue is like a combination discourse/imgur problem. What’s happening is, when you add a link to an image, the IC discourse server is accessing imgur and downloading/verifying the image somehow. Since imgur is getting spammed with requests from the IC server, it is applying a limit and returning an error. Discourse then assumes that the image is broken for everyone and refuses to show it.

I think this can be resolved one of two ways:

  1. Edit the offending posts directly in the DB and sidestep discourse’s parsing step. I assume this is why existing images somewhat work and subsequently break if you try to edit them. The downside to this is that they will probably break if you try to edit the post in discourse in the future.
  2. Change the forum config to prevent spamming the imgur servers. It seems there is a setting in discourse called “download remote images to local” somewhere - is that enabled currently? If so, could be worth turning it off and see what happens.
1 Like

Once we get going on the @ issue I will definitely look into that, thank you. I’m not sure imgur is being hit repeatedly with requests, but something in the way it shows images from a url is the issue - could be why it works in the edit of a post but not when published, but that’s just an idea.

I’m thinking more your second idea there as I don’t know exactly what the behaviour is at this point. The option is set to enabled right now, and I’d prefer to keep it that way right now while we’re cleaning one thing.

1 Like

Well, if you try to link to an imgur page directly the preview text says “the web server returned an error code of 429”:

If you google imgur 429, it happens when the server is being rate limited by imgur.

1 Like

I found the same thing on the Discourse server.

One thing I did check is using markdown text to host an image and so far seems to work like this:

Edit - worked in the preview but not when published

This worked again in preview so you’re right, it’s not allowing access to imgur as is. The only thing that doesn’t make sense in older posts with the html code in the post it does work, but also has horrible formatting around the image, such as on this post:

I checked the URL the image is for and it still works fine ‘’‘https://i.imgur.com/dPbVqhG.png’‘’

Yeah so my theory is this:

  • The editing preview is just local to your machine, so when you try to view the image the request is coming from your PC and imgur is happy to oblige.
  • When you hit “Reply” on discourse, it sends the message to the IC server, which subsequently attempts to download the image from imgur and imgur says “Stop calling me!!”. Then discourse is like “well this clearly isn’t a working image” and replaces it with a broken image link in the final post.
  • When the initial discourse migration happened, I assume all the posts were just copied into the database - so this discourse middleman step never happened. On those old posts, the final output simply contains a link to the image that the viewer’s PC then fetches from imgur… BUT if you go in and try to edit them, discourse jumps in the middle again and breaks the image.

EDIT: So this means any time you try to fix the @ stuff in a message with an imgur image that currently displays but with a bunch of jank around it, it will break the images even if you don’t change any of that text.