Database error messages

[todd1561]

Is anyone else getting pretty constant database error messages like below? I'm not too active on the forums but even with my limited use it's been pretty prohibitive to navigate around. Most times I need to reload a page several times before it'll show and even then it's often broken/half rendered. This seems to have started a few weeks ago.

https://toddnelson.net/misc/dmctalkdberror.jpg

[ssdelorean]

Yes.
Noticed it last week. Would click "reload" and go do something else while it was thinking. Most of time it would load after a bit.
I have not noticed yet this week.

[iflights]

Errors occurring daily for me. Seems to be better early, really early in the morning.

[Bitsyncmaster]

Getting about 90% of the time. I just exit and try again later.

[seventy4burban]

I have been able to get on for days but it appears to be fixed now

Sent from my Pixel 9 Pro XL using Tapatalk

[Aaron Cake]

From an IT perspective, best guess is that it is because the forum is at extremely high load due to all the "AI" bots scraping it for information. Visitor stats usually show 5 - 10 registered members and over 4,500 guests! All those guests are probably bots. I see the same thing on my forum which has been getting hammered with this BS for months. MySQL (the database) exhausts the amount of connections available. For anyone running a website, this has become a significant issue. We are paying for the bandwidth and cycles for these "AI" companies to steal our content then re-format it and sell it to their customers as a "training model".

Right now as I type this reply, there are "only" 2100 guests and no DB errors. Once it gets over 4000, the DB starts to grind.

[seventy4burban]

Sounds about right. I'm working on a possible alternative but not sure if there is demand for an alternative platform at this point

Sent from my Pixel 9 Pro XL using Tapatalk

[JBaker4981]

From another IT Administrator perspective, if the forum was locked down to where you have to log in in order to view content, that would fix the scalping bots running up resources on the Database.

[iflights]

I's like to see it locked down. But it does increase Administrator workload, and it will increase posts on the facebook site with people complaining about why they have not yet been approved.

[RIUM+]

I notice that this forum doesn't have a robots.txt file. Uploading a robots.txt file that blocks AI scrapers that declare themselves as such is a relatively easy thing to do that would at least reduce the problem. Something like ai.robots.txt (https://github.com/ai-robots-txt/ai.robots.txt) is already pre-formatted for easy uploading.

For more effective solutions there's Dark Visitors or CloudFlare, both of which require a lot more knowledge than simply uploading a file, but correspondingly block more bad actors.

[JBaker4981]

I's like to see it locked down. But it does increase Administrator workload, and it will increase posts on the facebook site with people complaining about why they have not yet been approved.
I believe that there is a select few here that should be promoted to Moderators, especially with the loss of Ron (RIP)


I notice that this forum doesn't have a robots.txt file. Uploading a robots.txt file that blocks AI scrapers that declare themselves as such is a relatively easy thing to do that would at least reduce the problem. Something like ai.robots.txt (https://github.com/ai-robots-txt/ai.robots.txt) is already pre-formatted for easy uploading.

For more effective solutions there's Dark Visitors or CloudFlare, both of which require a lot more knowledge than simply uploading a file, but correspondingly block more bad actors.

That's a damn good one - also putting a CloudFlare Proxy in front of DMCTalk would also be a good action item. You'd have to click 'not a robot' in order to be redirected to the site.

Edit: It also seems like the forum is back to normal in the last 12-18 hours

[bueller]

I'd bet it is an issue with bots (AI most likely) scraping too much data. A well formed robots.txt file would be a basic first step that could help dramatically (but it has been noted that many of the AI bots aren't respecting the robots.txt file either). There are a number of other options available too depending on how bad it gets.

I've offered to assist with site duties before also but haven't gotten anywhere. Happy to help and have a strong IT and cybersecurity background like others here.

Without some sort of intervention here soon by the admins this site will be completely unusable -- it errors out at least half the time currently Hope it doesn't come to that as I've learned a boatload from y'all.

[Aaron Cake]

Problem is that the "AI" bots generally ignore robots.txt unless they are from "legitimate" companies. As if there is a "legitimate" "AI" company. My experience with robots.txt is that it didn't decrease this traffic at all.

Something like Anubis (https://anubis.techaro.lol/) may help but requires installation and configuration on the host.

[RIUM+]

So this isn't a cause of the database errors, but I thought I might as well put it here instead of starting a new thread - I noticed the header images on this forum have very poor compression on them. Running them through their paces, I can shrink them by 33% completely losslessly, or by 90% lossy. It's a small improvement that doesn't matter much, but image optimisation is a hobby of mine so I thought I might as well share the results, in case the forum wanted to change over to using them to speed up page loads & reduce bandwidth used a little bit. :)

https://www.riumplus.com/delorean/dmctalk_header_lossless.png
https://www.riumplus.com/delorean/dmctalk_header_lossy.png

https://www.riumplus.com/delorean/dmctalk_header2_lossless.png
https://www.riumplus.com/delorean/dmctalk_header2_lossy.png

[Mark D]

https://dmctalk.org/attachment.php?attachmentid=71721&d=1753883508

[Timeless]

Nice Mark! One of favorites - watched it recently.

[Mark D]

Nice Mark! One of favorites - watched it recently.

:biggrin: I'm surprised anyone got the reference. It's a classic, I need rewatch it again sometime too.

In all seriousness, this place has been extra quiet lately and the database errors have not helped. It took me like 15 min and a dozen tries to even be able to upload that image. And then I got another couple of errors when trying to post.

There has to be something that can be done about all the bots bogging down the forum.

[iflights]

Just got a database error trying to access the database error thread! Ugh!

[ssdelorean]

https://dmctalk.org/attachment.php?attachmentid=71721&d=1753883508

I prefer this dead place over the black hole/time suck/non-searchable FacePlaceGroups. Yuck. Get off my lawn. :-)

[Bitsyncmaster]

I prefer this dead place over the black hole/time suck/non-searchable FacePlaceGroups. Yuck. Get off my lawn. :-)

Yea, Can you see me posting any of my technical information on FaceBook.

[Timeless]

I prefer this dead place over the black hole/time suck/non-searchable FacePlaceGroups. Yuck. Get off my lawn. :-)
Agreed 100%

[skipwater]

Just got a database error trying to access the database error thread! Ugh!

Question is anyone working on a fix?

vBulletin is not the only issue the server need's some love to.

[Helirich]

Getting about 90% of the time. I just exit and try again later.

This what I do. It just did it to me 3 times.

[todd1561]

Question is anyone working on a fix?


I think Tamir is the only one who could do anything about it but I don't think he's on here too often so I PMd him this thread.

[mark w]

this issue is constant....every day. multiple attempts to load the pages fail.

Come on moderators....lets fix this.

also make this site HTTPS....

[todd1561]

The site is already HTTPS.

[JBaker4981]

As a member of the community from ~2002, I am more than willing to assist with stabilizing this forum. We must fix these issues and retain the knowledge of this forum for the community as a whole.

@Tamir - Please, let's talk. Feel free to reach out to me on www.TN-DMC.com!

[Tamir A.]

Hey guys.

Apologies, have seen that things are a bit unstable. I'll get this sorted, been a bit inundated on my end the last month. Appreciate your patience.

Best,
T.

[steve]

:thankyou: It will make working on the car much easier.
Steve

[Bitsyncmaster]

Looks like it got fixed for now. I don't see the 4,000 users on line now.

[Rich]

DMCTalk is again fast and loads pages as reliably as it ever did.

Thanks to the mods and the community for sorting it out.

[ssdelorean]

Huzzah!

[dn010]

I'm not so sure it is fixed. I am still getting database errors. But there is also over 5,400 "guests" visiting right at the moment.

[Timeless]

I'm not so sure it is fixed. I am still getting database errors.
Ditto.

[Helirich]

Ditto.

For me also.

[iflights]

I continue to receive database error messages and encounter slowness. While it did show signs of improvement, those were short-lived, and I appreciate the engagement of the admin.

[JRNY13]

This is how message boards die. This needs to be fixed soon.

[bytes311]

It's probably a bunch of competing AI bots scraping everything they can get.

[JBaker4981]

I noticed DMCTalk redirected through a Proxy and theres less than 65 guests online.... THANK YOU!!!!!!!!!!

[Mark D]

Thanks for the fix! I got a popup check to verify that I wasn't a bot, and the forum has been running much smoother without 5k guests.

[Helirich]

Thanks for the fix! I got a popup check to verify that I wasn't a bot, and the forum has been running much smoother without 5k guests.

I didn?t get one. I hope I?m not a bot.

[Bitsyncmaster]

I didn?t get one. I hope I?m not a bot.

I got one a few days ago but have not seen any since.

[ramirezbradley212]

I got one a few days ago but have not seen any since.Same here that's what bought me here.

[glockworks21]

The bots are back Admins! 5500 guests (no chance)
Site is running like shit, time to reset ;O)