• Welcome to Peterborough Linux User Group (Canada) Forum.
 

Thanks Jason, for getting the PLUG website back up and running

Started by ssfc72, May 24, 2024, 06:41:52 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ssfc72

Mint 20.3 on a Dell 14" Inspiron notebook, HP Pavilion X360, 11" k120ca notebook (Linux Lubuntu), Dell 13" XPS notebook computer (MXLinux)
Cellphone Samsung A50, Koodo pre paid service

Jason

Thanks, Bill. I appreciate that.

The first thing I noticed was that the server load was heavy. And by heavy, I mean the load average was 40x what it should be AT MAX! The CPU was running at 100% and the RAM was near maxing out. The database, which the forum relies on, was the culprit. Why that was, I wasn't entirely sure. I'd restart the database service, things would improve but only briefly. Then it'd be insane again.

Cod3poet helped me walk through some steps and gave me some suggestions. Where I ended up, I should have started. In the SMF logs, one error came up repeatedly, a total of 7700+ instances over 7 hours. Yes, you read that right, over 1000 database read errors per hour! I believe it was causing the issue.  I've attached it so you can view it. It's tiny unless you click on it.

As you can guess, it checks a viewer's IP address to give a total of the unique users online at any given time. You see that total at the bottom of the main SMF page. So every time someone visited, it was choking on this issue.

Fun, huh? I still don't understand the first cause but it works. And that's more than we can say about most things today.



* Zorin OS 17.1 Core and Windows 11 Pro on a Dell Precision 3630 Tower with an
i5-8600 3.1 GHz 6-core processor, dual 22" displays, 16 GB of RAM, 512 GB Nvme and a Geforce 1060 6 GB card
* Motorola Edge (2022) phone with Android 13

cod3poet

NECRO BUMP!

Amazon crawler that went awol and caused a ton of issues, could be for indexing OR it could very well have been an offshoot of the AI training craze where companies are crawling public data to train their AI models.

The IP in question is for an amazon crawler out of Ashburn Virginia. Cool beans.
Arch, Windows, Ubuntu, MacOS. In that order. (Definitely 08/2024)
i9-13900hx/32gb/2tbNVME/4090-Win11-WSL2
Ryzen9 5950x/128gb/2tbNVME/8TBhdd/8TBssd/3080ti-Win11
8gen and 10gen i7/32gb/1tbNVME-Arch(k8s) + m1Mac(work)
Azure Devops Expert / Hacker / Automation Engineer

Jason

The most interesting part of your post was this:

Quote from: cod3poet on August 29, 2024, 11:20:01 AMNECRO BUMP!

Please explain.

The rest of your post is intriguing. I haven't deleted the error logs yet so I'll have to check and see how many IPs were doing this. I know it was a limited number.

I'm eternally thankful it didn't create such a flurry of traffic that I'd be staring down a $2000 bandwidth-exceeded charge.
* Zorin OS 17.1 Core and Windows 11 Pro on a Dell Precision 3630 Tower with an
i5-8600 3.1 GHz 6-core processor, dual 22" displays, 16 GB of RAM, 512 GB Nvme and a Geforce 1060 6 GB card
* Motorola Edge (2022) phone with Android 13

cod3poet

Necro bump, old school forum slang for commenting on an old and generally considered dead post.

Also seemed relevant because it seems I have magically risen from digital silence.

I'd be happy to review the results with you I have been working on some data science(ey) stuff as of late and may start a new post just for that.
Arch, Windows, Ubuntu, MacOS. In that order. (Definitely 08/2024)
i9-13900hx/32gb/2tbNVME/4090-Win11-WSL2
Ryzen9 5950x/128gb/2tbNVME/8TBhdd/8TBssd/3080ti-Win11
8gen and 10gen i7/32gb/1tbNVME-Arch(k8s) + m1Mac(work)
Azure Devops Expert / Hacker / Automation Engineer

Jason

Quote from: cod3poet on September 10, 2024, 11:56:01 AMI'd be happy to review the results with you I have been working on some data science(ey) stuff as of late and may start a new post just for that.

Sure, I'd love to see that. Others might, too. I've been taking a couple of courses related to data analysis. Making dashboards is fun!
* Zorin OS 17.1 Core and Windows 11 Pro on a Dell Precision 3630 Tower with an
i5-8600 3.1 GHz 6-core processor, dual 22" displays, 16 GB of RAM, 512 GB Nvme and a Geforce 1060 6 GB card
* Motorola Edge (2022) phone with Android 13