Donations to Upgrade Forum Server?

Questions and comments
Post Reply
Spartan_Warrior
Posts: 1659
Joined: Fri Dec 02, 2011 1:24 am

Post by Spartan_Warrior »

As some of you are probably all too aware, this forum has been experiencing all kinds of latency and downtime issues for the last several months. Personally, as a result, I rarely even sign in or attempt to post here anymore, and every single time I do (including today), I am affected by horrible (5 minute+) lag time when navigating or posting, typically accompanied by several "CloudFlare" error messages that the site is down, no live version is available, server is too busy, etc. I won't belabor the details as there have already been multiple threads (most of them appear to be closed now).
Jacob has said multiple times that one possible solution to the issue would be a server upgrade, but that he is unwilling to foot this expense.
That's fine, but I'm left wondering: what exactly IS the expense, and is everyone else also unwilling? I believe there are ways to set up donations for such things using PayPal. At this point, I for one would gladly donate 5-10 dollars toward a server upgrade or any other solution that allowed me to use this forum reliably. If others are having the same issues, maybe they'd be willing as well?


mfi
Posts: 93
Joined: Sun Jul 29, 2012 10:27 pm

Post by mfi »

+1


chenda
Posts: 3289
Joined: Wed Jun 29, 2011 1:17 pm
Location: Nether Wallop

Post by chenda »

Yes I would definitely donate something


jacob
Site Admin
Posts: 15906
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Post by jacob »

A dedicated server costs around $50--150/month (compared to the current shared server @ $10/month). The blog/forum makes about $75--105/month in ad revenue, so cost is not an issue.
However, I don't want to spend the time figuring out how to migrate and upgrade everything without breaking it. I figure this could take anywhere from 30 hours to 300 hours to learn and result in downtime of anything from 1 hour to 1000 hours were I to do it myself.
So what's really needed is a webmaster who can be trusted to fix this without screwing it up or running off into the sunset with the keys (passwords). If the forum can come up with and vet such a person and the money to pay their fee if any, I'll pick up the running costs of the new server.


sshawnn
Posts: 458
Joined: Tue Mar 08, 2011 8:17 pm

Post by sshawnn »

Im in for donating cash for the forum.


User avatar
C40
Posts: 2748
Joined: Thu Feb 17, 2011 4:30 am

Post by C40 »

I'm in for a bit


buzz
Posts: 115
Joined: Thu Sep 30, 2010 11:16 pm

Post by buzz »

I would contribute to this.


Dragline
Posts: 4436
Joined: Wed Aug 24, 2011 1:50 am

Post by Dragline »

I'll do my bit.


bulgaria
Posts: 70
Joined: Sat Jan 26, 2013 5:28 pm

Post by bulgaria »

I also am willing to donate.
What I do not understand is why the current forum setup causes these problems. Sometimes days go by without problems.. sometimes it is happy hour all the time. It isn't that the forum is super popular, so there is not a ton of traffic.
@jacob: have you determined what is the cause of the slowdown? Is it memory, cpu, network traffic?


taekvideo
Posts: 12
Joined: Sat May 04, 2013 8:11 pm

Post by taekvideo »

I could do the transfer for you. I've done freelance web design off and on since high school. My most recent work was an overhaul on the site for a small tv show in Texas http://theemcshow.com/ (the old look was like something from the 90's lol) I'm sure Chris would be glad to vouch for me if need be.
A site transfer should only take a couple hours (really depends on the hosts)... and the existing site can stay online in the meantime (though it'd possibly be even more laggy while the transfer takes place).


Scott 2
Posts: 2824
Joined: Sun Feb 12, 2012 10:34 pm

Post by Scott 2 »

Running a responsive, high traffic forum is a headache. I worked at a company that provided it as a service for our clients. It was always a money loser, and the developers never wanted to be the one stuck dealing with the forums.
Performance problems were typically:
1. Over-zealous search engine spiders

2. Spam bots

3. Search
1 and 2 can be helped by requiring authentication to read the forum. Hiding threads from search engines may help, but not as much. Both are traffic limiting moves, but serious users would still find their way.
3 could be helped by limiting search to authenticated users, requiring a delay between searches, limiting time duration of search, eliminating wild card searches, etc. Basically reduce who can search and how hard they can search.
Many of these could be configuration options in the forum software and might provide a stop-gap if an admin cannot be found.
Keeping a forum running is a pain. We are lucky Jacob has stuck with this place even after moving on to his next venture.
Edited to add - throwing hardware at the problem might not solve anything. It's possible the shared server is providing as much resources as a dedicated server would make available. A forum admin might have to upgrade the forum software, migrate to different forum software, index / change databases, etc. Performance issues are a bear to solve without limiting the users.


Spartan_Warrior
Posts: 1659
Joined: Fri Dec 02, 2011 1:24 am

Post by Spartan_Warrior »

Hmm, from what I understand of this, it shouldn't be too difficult to bring the forum back in its current state on a new server. The potential difficulty would be if you were to upgrade or change the forum software, which as Scott 2 points out might be the ultimate solution anyway for stopping the primary issue (which I understand to be spammers clogging the registration page).
Is the forum running on a MySQL database? I actually have some experience transferring data from an old Invision PowerBoard to phpBB, though probably not enough experience to volunteer for this. (It was for similar reasons too--script kiddie/spammers hijacked the iPB forum.) It can definitely be time consuming, and not everything transfers 1:1. This forum is also way bigger than the one I ran. I wonder if there aren't scripts for easily translating the essential data forms (thread titles, post content, etc) between different forum software...
I also concur that this might be a decent stop-gap:
"1 and 2 can be helped by requiring authentication to read the forum. Hiding threads from search engines may help, but not as much. Both are traffic limiting moves, but serious users would still find their way."
Though I don't think that would block spammers at the registration page...


JamesR
Posts: 947
Joined: Sun Apr 21, 2013 9:08 pm

Post by JamesR »

A VPS from Linode.com or DigitalOcean.com might do the trick, it's between the two options, and ranges in prices from $20/mo to $60/mo. However, a dedicated server is typically better bang for your buck. You can find good deals for dedicated hosting from http://www.webhostingtalk.com/forumdisplay.php?f=36 . I used to have a hivelocity dedicated server for $50/mo, which was pretty cheap for one.
Probably need to do some sort of MySQL replication so that you copy the database over while keeping the existing forum up, and keep the copy synchronized.
As for blocking spammers hitting the registration page, well perhaps there's a phpbb plugin to add captchas and similar. I usually find that plugin-wise, it's best to use a weird or rarely used plugin so that spammers haven't automated handling that yet. Or perhaps a really well supported one that connects to a third party, like akismet.
The forum might also have an export option, that exports out everything in XML format or similar, that might be importable into a different version/type of forum. Wordpress has something like that, which can be convenient.


My_Brain_Gets_Itchy
Posts: 267
Joined: Fri Mar 02, 2012 5:29 pm

Post by My_Brain_Gets_Itchy »

One other possible culprit could be the database?
Databases performance generally slows when it reaches a certain size and it is not tuned. Especially if it is not a commercial database.


Spartan_Warrior
Posts: 1659
Joined: Fri Dec 02, 2011 1:24 am

Post by Spartan_Warrior »

@JamesR: Yes, the export to XML function is exactly how I did it back in the day, but I then had to do a cross-reference of the data fields since they differ between forum software (e.g. finding and translating the field for "post content" in one forum to another). More tedious than technical. That's why I was wondering if there might be scripts that do that part for you.
Assuming, though, that we're only upgrading the server and NOT changing the forum software, then yeah, it should be pretty simple to export the database, set up the new forum on the new server, and then import the database. Maybe a few hours of down time but not too complex. Whether that solves the problem or not might be another matter though.
@MyBrain: That's another really good point re: database size. Maybe some kind of archiving/culling is in order.


jacob
Site Admin
Posts: 15906
Joined: Fri Jun 28, 2013 8:38 pm
Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
Contact:

Post by jacob »

1) If the site is moved to another server and it doesn't solve the problem, it would make me very unhappy to be stuck paying $150/month instead of $10/month.
2) The forum runs bbPress 1.0.2. It's based on an SQL database like wordpress. I have no idea about sql databases.
3) Here are the traffic stats for the forum from cloudflare (for the past 30 days)

492,946 Page views

337,200 regular traffic

81,472 crawlers/bots

74,274 threats

The blog traffic (same server) is maybe about 150,000 page views.
4) What we need is not just a site transfer but probably a webmaster/sysop who can do a "deep clean" involving transferring the forums to new software as well. bbpress is old!


buzz
Posts: 115
Joined: Thu Sep 30, 2010 11:16 pm

Post by buzz »

If you're interested in upgrading the software, phpBB is a more modern forum package that has lots of support. More importantly, the database design is similar enough to port the most critical data: https://www.phpbb.com/community/viewtop ... 5&t=673815


KevinW
Posts: 959
Joined: Mon Aug 02, 2010 4:45 am

Post by KevinW »

If we're considering switching forum software, let's also consider Discourse:

http://www.discourse.org/


Spartan_Warrior
Posts: 1659
Joined: Fri Dec 02, 2011 1:24 am

Post by Spartan_Warrior »

My impression (correct me if I'm wrong) is that Jacob isn't interested in researching or implementing new forums himself. He's looking for a webmaster to do this stuff.
After discussing all the options in this thread, I think upgrading the server should actually be a last resort. If upgrading the forum software/cleaning up the database/implementing anti-spam registration plugins are all in the cards anyway, it certainly makes sense to try those things on the current server before throwing money at the problem. So we're looking for someone who's willing and able to do those things.
Ah, hell... I suppose I'll throw my own name into the hat as a potential candidate. My experience with forums/SQL is about ten years old and based only on my own trial-and-error learning, so I doubt I am the best candidate. On the other hand, I'd be willing to see what I could do free of charge.
An ideal candidate IMO would be 1) someone experienced, preferably currently or recently running forums; 2) a long-time, well-established and trustworthy contributor to the ERE forum; and 3) someone willing to do it cheaply/free.
So far it looks like it's me and taekvideo upthread who've volunteered. Taekvideo sounds more experienced with webmastering in general but has only 2 posts here and didn't mention whether he/she wants compensation for the work. The site he/she provided as a reference also doesn't appear to involve forums. Taekvideo, do you have experience implementing forums and working with php/SQL as well?
Anyone else willing to throw their names in? Jacob mentioned that this person should be vetted by the forum so I assume it's up to popular vote.


Scott 2
Posts: 2824
Joined: Sun Feb 12, 2012 10:34 pm

Post by Scott 2 »

It's way easier to talk about a problem than do something about it :)
Spartan_Warrior - you certainly seem motivated to fix it and willing to stick around. Based on the description of the problem and your skills, I think it's a good fit. IMO an issue like this just requires comittment to getting it fixed, no extensive expertise tuning SQL databases or anything like that.
Because it's probably easy, I would try disabling search before going to more extensive options. If it solves the problem, replacing it could be simple. For a smaller website like this, google custom search can be enough and free:
http://www.google.com/cse/
Of course, there's no way to be certain what the problem is ahead of time, without extensive profiling of the site.
Trial and error making changes over a period of time is going to be the easier option (without a team of developers), especially when dealing with a shared server.
When handling a performance problem, traffic volume is not as important as what that traffic is doing. Often it turns out that for everything a site does, there are just one or two features that absolutely bury the database. That cascades through to the rest of the site, creating problems for everyone.
BTW - an assumption being made here, is that the forums are the source of the performance problem. Maybe something on the main site is burying the server and screwing the forum. Alternatively, maybe when the forums go bad, they screw the main site. Traffic logs might show if the two parts of the site are messing with one another. If so, even if nothing else is done, it could make sense to seperate them out to different shared servers.


Post Reply