have had 38 âsystem messagesâ - all telling me I am a cunt.
1 Like
Nothing to worry about. Discourse is cleaning house.
New YouTube links will be processed when Discourse has got through the 500k things it has in the queue.
As ever, itâs heartening to see the forgiving and understanding nature of the average Sotonian come to the fore in moments of stress.
Cunts.
4 Likes
Iâm losing the will to live having to constantly open new tabs just to follow The Butlerâs ecletic tastes.
Should you be drinking while cleaning?
Iâm in London helping my son and his girlfriend move from their 3rd and second floor flats into their new 3rd floor flat.
Good job I havenât just had âthe cock opâ otherwise Iâd have tender undercarriage right now.
Also good job I didnât get letdown by Zipcar so had to use my estate instead of a transit otherwise Iâd not have had the pleasure of multiple, multiple, multiple car trips.
3 Likes
âŚand Iâve just furloughed Chambers, you should have said. 
2 Likes
Is there any way we can turn this system notifications off?
I know that @saintbletch says this is Discord doing its thing, but itâs close to unusable atm. Like SaintsWeb.
1 Like
Serve you right for posting so much shite over the years 
2 Likes
Iâve made a change that should quieten things down a bit. Let me know if it has / hasnât worked.
1 Like
No idea why but copying a tweet or uploading a photo, THIS is becoming a nuisance after I type any character.
Android Samsung A7 latest software update. Chrome
OK, that message means that Discourse thinks you have already started replying to that topic in another window.
I donât think itâs a bug, I think it telling you exactly what it saysâŚ
It suggests that you have that topic open, and a reply drafted in another tab of the browser or on another device - computer, tablet, other phone, etc.
It doesnât know which one of the replies you want to keep so asks you to make the decision.
Try shutting down closing all the tabs in the browser and reloading the page in a single tab.
If not then shut down the browser completely on the phone and re-load and try again.
It it persists, check any other devices that you might have used and close the tabs and re-load.
If itâs not that then log out and log back in again.
If it still persists I can log you out in a âspecialâ way but that can be a bit nuclear.
Update
tl;dr the long-running job has finished and Iâve (hopefully) cleared any @system notifications youâve been sent telling you that your post was edited.
I hope itâs solved the issue of apparently non-secure topics.
Time and @tigger will tell.
Anyone interested in the details⌠(Just me and @BTripz, I guess)
The very long collapsed section below will answer questions likeâŚ
- What happened over the weekend to the server?
- Why did the things I post not expand links to YouTube, Twitter, etc.?
- Why did things slow Sunday night into the early hours of this morning?
- Why did I get sent hundreds of messages from @system telling me my posts had been edited?
All this and more will be answeredâŚbut if you donât have the patience or inclination then know that in trying to fix something I caused problems.
Bletch
I need to send some of this in a message to Bob and it's quite a cathartic process for me to write it so you might as well get it too. You might find it interesting or it might help you sleep or you might choose to ignore it...
Why did you break stuff, @saintbletch, you fucking twat?
Valid question.
Iâll tell you why but thereâs a bunch of context and technical mumbo-jumbo you will need to fully understand how deliciously stupid technical things can become.
Background and Technical Stuff
Posters were reporting that their browser was complaining that parts of Sotonians were not secure.
This is a small problem for our users as we can pretty much assure them that the risk is very, very low, but if Google and other search engines think we are serving insecure content then it will reduce our search engine rankings, so I had to try to get it fixed.
What was causing the ânot secureâ problem?
Not 100% sure, butâŚfirstly I should say that all of the content that gets typed into Sotonians or that is uploaded to Sotonians is sent out over secure channels to your browsers (via HTTPS).
I think the Chrome message came from when users post images or media and, instead of uploading it to our servers, they simply point to those images or media items on other servers somewhere on the Internet where they are provided over a non-secure channel (HTTP instead of HTTPS).
Virtual Hosting
To run the forum I rent space on a server in London (was previously the US). It has 2 processors and 4GB of RAM.
This server is âvirtualâ in as much as it isnât a physical machine with our forum sitting on it in some corner of a data centre. In reality, the virtual server is a bit of software sitting inside a massively powerful physical machine with (probably) 32 physical processors and 100s of GB of RAM. We rent a little bit of all that power.
Using virtual hardware like this allows me to keep the cost down and, if we need it, increase (or decrease) the size of our virtual server really easily. I simply rent a bit more of that power.
I also rent space on Amazonâs S3 (not S4 @Fatso Deputy Dumb) where all of our big images and videos are stored. This is the least expensive way of storing the GBs of shit weâve collected over the years.
Background tasks and queues
One of the reasons that Discourse can be full of features and yet perform as well as it does on relatively modest and inexpensive hardware is that it divides the processing of tasks between the client (your browser) and the server (our virtual box in London ).
So the power of your phone/tablet/PC contributes to the processing of the forum. Not only that but any of the server tasks that can be delayed - i.e. itâs not critical that they run immediately, get put into a queue and are executed in the background when the server has some spare capacity
This means that the server concentrates on the jobs that have to happen right at that moment in time and will get to the other jobs as and when it has time. It continually makes the most optimum use of the hardware available and this results in really snappy performance (most of the time).
Messages
One of the issues of dividing the processing of stuff between the client (your browser) and the server (our machine in London) is that of coordination.
How does your browser client know that someone has posted a new post to the topic that it is currently displaying?
How does the server know that someone just liked a post, etc.?
To do this Discourse uses a messaging system where hundreds of thousands of messages are sent from our server to clients and back again, all over the world.
âNew post in topic 412342â
âNew like for post 2343 in topic 12343â
âSomeone just flagged @Barry-Sanchezâs post 23 in topic 91234â
etc.
Single page application
You may or may not have appreciated that you seldom have to refresh a Sotonians page (unless something has gone wrong).
Once the initial Sotonains page has been sent out, the server never sends the whole web page to you again. Youâre probably aware that the top of the Sotonians page stays where it is and other bits of the page update below. This technique makes Discourse a Single Page Application (SPA).
This dramatically reduces the amount of âstuffâ that our server has to send to your browser each time something changes when compared to other sites. Remember the days of F5? We donât need to send massive chunks of pages out to refresh content. We simply send the small bits that have changed.
Messaging - the exchanging of millions of notifications between server / client / server is the thing that makes that possible. And again, it helps Discourse to perform really quickly.
If you keep your web page open then it can be days/weeks before Discourse gets to send you a full web page.
This is very efficient BUT if things start to get out of sync and bits on the server are updated ahead of where your browser was when it last did a full re-load then weird stuff can (and will) happen.
Baking of posts
When you create a post on Discourse, it stores the text of your post and then, in the background, converts it to HTML so it can be displayed in a browser.
It sends a message to tell any clients (membersâ browsers), that are displaying that topic that it has a new post.
If the post has an embedded image then Discourse will a) create a background job to create different sized thumbnails of that image and b) uploaded it to Amazon S3 (a relatively cheap place to store lots of big files).
When both of these things are done, the post gets âbakedâ into HTML using the new thumbnails and the URL of the image on Amazon S3. Finally, any membersâ browsers that have that topic open will get a background message to tell them to display the new âbakedâ post.
Secondly, if the post text contains links to certain sites or social media platforms Discourse will turn the link into a OneBox representation. It makes it pretty like thisâŚ
Once this is done, a message is sent to anyoneâs browser that has that topic open to tell them that the post has been updated with the OneBox and that it should be re-displayed.
This process of taking the text and making it presentable is called baking and it relies on background processes, messaging and the power of our server.
OK Bletch, so lots of moving parts - but how did things get bent out of shape?
So there are indeed lots of moving parts.
To fix the non-secure warning I needed to do two things.
Firstly, I needed to tell Discourse that if it finds an image or other media item that originates from a site other than Sotonians (letâs say scammersRus.com), then it should download that image and store it on our server - as if it had been uploaded by the lazy twat that posted it in the first place.
This means that whenever that image or media is sent from our servers in future it will be sent from sotonians.com over a secure channel (HTTPS) rather than us telling your browser to open a non-secure connection to scammersRus.com (HTTP).
All good.
But that setting only affects what Discourse does moving forward. We had 390K+posts that could have errant images and media in them.
So I needed to tell Discourse to RE-BAKE all of those posts, which I did using a bit of code.
All good.
This takes about 3 hours to run but after it had finished, the problem with images was still there.
Not good.
So I decided to run the command again, which I did using a bit of code that I double-checked - again taking 3 hours to run. This time over night.
All good.
But after that code finished it still hadnât worked.
Not good.
So I investigated and only then did I remember that when I told Discourse to RE-BAKE all of our 390K posts, it created a background job that could be processed whenever the server has some capacity.
Or rather, it created 390K jobs. Twat.
Twice (remember I ran it twice) - 780K jobs. Double twat.
So @saintbletch had given the server a bunch of things to do that would take it 8+ days to finish at the rate it was going.
Not good.
Also, any posts you were making in the interim that had OneBoxes or images would create new background jobs that went to the back of the queue with 780K messages in front of them.
Your YouTube links would get expanded - but not until next Monday week. LOL.
I had a choice to either cancel the jobs - knowing that Iâd only have to start it again to fix our non-secure issue or I needed to find a way to get that 8 day run of jobs down to something more manageable.
The first thing I did was to tell Discourse to focus on background jobs more than it was. This increased the speed at which our background jobs were being handled and dropped the 8 days to 6 days.
All good.
However, this meant that the performance of the site tanked as our precious resources were being deployed processing background jobs instead of letting us moan about Covid.
Not good.
But I knew that I could change settings to tune and balance the amount of resources Discourse placed on background tasks as opposed to storing away @Polski_Filipâs latest golf gossip. So I set about finding a balance.
All good.
But each time I made a change I needed to watch it for 30 mins to see how quickly the background tasks were now being processed and then each time I changed the configuration I would need to take the box offline to change the config which takes 10-15 mins of downtime.
Not good.
Also, each change I made updated Discourse to the latest version which meant, due to the beuty of Single Page Applications, that anybody on an older version of the client might have experienced weird stuff.
Not good.
So, one of the benefits of running on a virtual server instead of a physical one, is that you can increase its power relatively simply.
So I commissioned a box with 16 processors and 32GB of RAM (which impressively only took about 10 mins to get our system running on). I told it to spend most of its massive capacity running the background tasks, tuned it a few times to get the optimum performance and our jobs finished yesterday morning (36 hours instead of 192).
All good.
However, each time the @system user edits your posts - which it does when it finds something in your posts that needs re-baking, you get a helpful message telling you so.
But many of you had hundreds or thousands of such edits and therefore hundreds or thousands of such messages.
Not good.
So I told Discourse that when weâre downloading images when re-baking posts to not send out messages.
All good.
But youâd already had loads of messages and it seemed the site was âunusableâ.
Not good.
So I wrote some code to mark all of your @system notifications as read.
All good. I think. Not heard anything for 24 hours.
Weâre back to where we were with all media that was previously on external servers (hopefully) uploaded to our servers. Any future media link from external servers will automatically be uploaded.
So there you have it - the technical equivalent of a man in a pub holding a pint of Punk IPA being asked the time, and pouring every fluid ounce of it into his own lap as he tilts his wrist. Bletch trading knowing the time for a wet crotch.
Cunts.
2 Likes
TBH, i have never been more popular⌠made nice change to get some messages⌠;-(
1 Like
Apparently the bulk of the work was editing âtehâ to âtheâ 
5 Likes
I could have told you all that right at the start 
1 Like
I tried Belch, I really tried.
Just woke up and found England had lost 2 wickets.
1 Like
Well, the padlock icon in Firefox on my laptop is now a dull-looking grey, which means that neither Firefox nor Eset has detected any issues with the certificate, the site itself, or any links contained therein.
In other words, Bletch me old beauty, I reckon youâve fixed it.
4 Likes
Fixed that for you. A lot of fixing going on at the moment 
Any idea why some of the recent YouTube links havenât been embedding in âThe Chainâ topic?