Page 1 of 13
Posted: Tue May 07, 2013 8:40 pm
As some of you are probably all too aware, this forum has been experiencing all kinds of latency and downtime issues for the last several months. Personally, as a result, I rarely even sign in or attempt to post here anymore, and every single time I do (including today), I am affected by horrible (5 minute+) lag time when navigating or posting, typically accompanied by several "CloudFlare" error messages that the site is down, no live version is available, server is too busy, etc. I won't belabor the details as there have already been multiple threads (most of them appear to be closed now).
Jacob has said multiple times that one possible solution to the issue would be a server upgrade, but that he is unwilling to foot this expense.
That's fine, but I'm left wondering: what exactly IS the expense, and is everyone else also unwilling? I believe there are ways to set up donations for such things using PayPal. At this point, I for one would gladly donate 5-10 dollars toward a server upgrade or any other solution that allowed me to use this forum reliably. If others are having the same issues, maybe they'd be willing as well?
Posted: Tue May 07, 2013 8:56 pm
Posted: Tue May 07, 2013 9:23 pm
Yes I would definitely donate something
Posted: Tue May 07, 2013 11:13 pm
In another forum that I frequent, you have to be a paying member to post and read the journals. Maybe something like this could help to make the site cashflow positive. If it's so popular it should be possible and right for the owner to make some money instead of just spending his own.
Posted: Tue May 07, 2013 11:24 pm
A dedicated server costs around $50--150/month (compared to the current shared server @ $10/month). The blog/forum makes about $75--105/month in ad revenue, so cost is not an issue.
However, I don't want to spend the time figuring out how to migrate and upgrade everything without breaking it. I figure this could take anywhere from 30 hours to 300 hours to learn and result in downtime of anything from 1 hour to 1000 hours were I to do it myself.
So what's really needed is a webmaster who can be trusted to fix this without screwing it up or running off into the sunset with the keys (passwords). If the forum can come up with and vet such a person and the money to pay their fee if any, I'll pick up the running costs of the new server.
Posted: Wed May 08, 2013 2:37 am
Posted: Wed May 08, 2013 3:31 am
I'll donate for whatever we need.
Posted: Wed May 08, 2013 3:44 am
Im in for donating cash for the forum.
Posted: Wed May 08, 2013 3:53 am
I'm in for a bit
Posted: Wed May 08, 2013 4:04 am
I would contribute to this.
Posted: Wed May 08, 2013 4:19 am
I'll do my bit.
Posted: Wed May 08, 2013 6:31 am
I also am willing to donate.
What I do not understand is why the current forum setup causes these problems. Sometimes days go by without problems.. sometimes it is happy hour all the time. It isn't that the forum is super popular, so there is not a ton of traffic.
@jacob: have you determined what is the cause of the slowdown? Is it memory, cpu, network traffic?
Posted: Wed May 08, 2013 2:11 pm
I could do the transfer for you. I've done freelance web design off and on since high school. My most recent work was an overhaul on the site for a small tv show in Texas http://theemcshow.com/
(the old look was like something from the 90's lol) I'm sure Chris would be glad to vouch for me if need be.
A site transfer should only take a couple hours (really depends on the hosts)... and the existing site can stay online in the meantime (though it'd possibly be even more laggy while the transfer takes place).
Posted: Wed May 08, 2013 2:12 pm
Running a responsive, high traffic forum is a headache. I worked at a company that provided it as a service for our clients. It was always a money loser, and the developers never wanted to be the one stuck dealing with the forums.
Performance problems were typically:
1. Over-zealous search engine spiders
2. Spam bots
1 and 2 can be helped by requiring authentication to read the forum. Hiding threads from search engines may help, but not as much. Both are traffic limiting moves, but serious users would still find their way.
3 could be helped by limiting search to authenticated users, requiring a delay between searches, limiting time duration of search, eliminating wild card searches, etc. Basically reduce who can search and how hard they can search.
Many of these could be configuration options in the forum software and might provide a stop-gap if an admin cannot be found.
Keeping a forum running is a pain. We are lucky Jacob has stuck with this place even after moving on to his next venture.
Edited to add - throwing hardware at the problem might not solve anything. It's possible the shared server is providing as much resources as a dedicated server would make available. A forum admin might have to upgrade the forum software, migrate to different forum software, index / change databases, etc. Performance issues are a bear to solve without limiting the users.
Posted: Wed May 08, 2013 2:50 pm
Hmm, from what I understand of this, it shouldn't be too difficult to bring the forum back in its current state on a new server. The potential difficulty would be if you were to upgrade or change the forum software, which as Scott 2 points out might be the ultimate solution anyway for stopping the primary issue (which I understand to be spammers clogging the registration page).
Is the forum running on a MySQL database? I actually have some experience transferring data from an old Invision PowerBoard to phpBB, though probably not enough experience to volunteer for this. (It was for similar reasons too--script kiddie/spammers hijacked the iPB forum.) It can definitely be time consuming, and not everything transfers 1:1. This forum is also way bigger than the one I ran. I wonder if there aren't scripts for easily translating the essential data forms (thread titles, post content, etc) between different forum software...
I also concur that this might be a decent stop-gap:
"1 and 2 can be helped by requiring authentication to read the forum. Hiding threads from search engines may help, but not as much. Both are traffic limiting moves, but serious users would still find their way."
Though I don't think that would block spammers at the registration page...
Posted: Wed May 08, 2013 3:32 pm
A VPS from Linode.com or DigitalOcean.com might do the trick, it's between the two options, and ranges in prices from $20/mo to $60/mo. However, a dedicated server is typically better bang for your buck. You can find good deals for dedicated hosting from http://www.webhostingtalk.com/forumdisplay.php?f=36
. I used to have a hivelocity dedicated server for $50/mo, which was pretty cheap for one.
Probably need to do some sort of MySQL replication so that you copy the database over while keeping the existing forum up, and keep the copy synchronized.
As for blocking spammers hitting the registration page, well perhaps there's a phpbb plugin to add captchas and similar. I usually find that plugin-wise, it's best to use a weird or rarely used plugin so that spammers haven't automated handling that yet. Or perhaps a really well supported one that connects to a third party, like akismet.
The forum might also have an export option, that exports out everything in XML format or similar, that might be importable into a different version/type of forum. Wordpress has something like that, which can be convenient.
Posted: Wed May 08, 2013 3:45 pm
One other possible culprit could be the database?
Databases performance generally slows when it reaches a certain size and it is not tuned. Especially if it is not a commercial database.
Posted: Wed May 08, 2013 3:59 pm
@JamesR: Yes, the export to XML function is exactly how I did it back in the day, but I then had to do a cross-reference of the data fields since they differ between forum software (e.g. finding and translating the field for "post content" in one forum to another). More tedious than technical. That's why I was wondering if there might be scripts that do that part for you.
Assuming, though, that we're only upgrading the server and NOT changing the forum software, then yeah, it should be pretty simple to export the database, set up the new forum on the new server, and then import the database. Maybe a few hours of down time but not too complex. Whether that solves the problem or not might be another matter though.
@MyBrain: That's another really good point re: database size. Maybe some kind of archiving/culling is in order.
Posted: Thu May 09, 2013 1:34 am
1) If the site is moved to another server and it doesn't solve the problem, it would make me very unhappy to be stuck paying $150/month instead of $10/month.
2) The forum runs bbPress 1.0.2. It's based on an SQL database like wordpress. I have no idea about sql databases.
3) Here are the traffic stats for the forum from cloudflare (for the past 30 days)
492,946 Page views
337,200 regular traffic
The blog traffic (same server) is maybe about 150,000 page views.
4) What we need is not just a site transfer but probably a webmaster/sysop who can do a "deep clean" involving transferring the forums to new software as well. bbpress is old!
Posted: Thu May 09, 2013 2:46 am
If you're interested in upgrading the software, phpBB is a more modern forum package that has lots of support. More importantly, the database design is similar enough to port the most critical data: https://www.phpbb.com/community/viewtop ... 5&t=673815