Donations to Upgrade Forum Server?

taekvideo · Post by **taekvideo** » Thu May 09, 2013 6:12 pm

I have to second phpbb.. I've used it for every forums I've run (or setup for other people) and love it.

It also shouldn't be very hard to migrate, looks like someone wrote the code for it already:

https://www.phpbb.com/community/viewtop ... 5&t=673815

The OP is old but a post later in the thread has some code that's supposed to work with bbpress 1.0.2
I'd be willing to help if needed for free, though it looks like spartan would be able to handle it just fine.

Post by **jacob** » Thu May 09, 2013 11:47 pm

I agree with Spartan's post, namely
"An ideal candidate IMO would be 1) someone experienced, preferably currently or recently running forums; 2) a long-time, well-established and trustworthy contributor to the ERE forum; and 3) someone willing to do it cheaply/free."
I'll add.
4) Someone who is willing to follow through and fix it if something breaks. Given some software skills, being willing to learn and figure this stuff out is probably more important than skills.

Post by **jacob** » Thu May 09, 2013 11:54 pm

Here's some specific actions/suggestions I think could/should be done
1) Put the blog on cloudflare (current only the forum is). The problem is that the domain name is earlyretirementextreme.com (which cloudflare doesn't like). Make it www.earlyretirementextreme.com and it would work. This would involve some apache programming to ensure that "google doesn't break".

2) Exclude robots from searching the forums.

3) The forum spam problem is a DDOS type problem. The server gets dragged down by having to serve the registration page (via sql/php). This could be fixable by excluding php/sql access on the registration page? (I think having to generate the reg page too much is the primary source of the problem.) You need to know how to program php to fix this.

4) Switch to phpbb or some other software.

5) I've paired down the blog as much as I know. I think the problem is the forum, not the blog although lately the blog has received lots more spam than it used to. (However the blog uses caching).
I'll stick this thread to the top.

Spartan_Warrior · Post by **Spartan_Warrior** » Fri May 10, 2013 12:19 am

If I took the lead on this, my first plan would indeed be to limit the search capability to reduce load on the database, since that's the easiest and least disruptive step. If no change, I'd go ahead and instantiate a phpBB on the current server and try to import the data from this forum into the new one using the script taekvideo linked. Assuming that transfer goes according to plan, I'd then take down the current forum and start using the new one for a few days and see if the problem persists. If that doesn't help, I'd try some kind of captcha plug-in on the registration page to try to keep it from getting swamped by spam-bots. If still nothing, then I'd suggest we start looking at a server with more bandwidth.
Possibly as an intermediate step to going to the new server, I might try archiving some of the older posts in some fashion, to reduce the size of the database. Though I'm not sure what the popular opinion is on losing older posts.
That's about the extent of what I'd feel comfortable attempting. I wouldn't know how to do apache programming, modify the php source code, diagnose traffic patterns, sleuth out whether something on the blog is causing issues, etc.
As far as Jacob's requirement 4, I'm not sure I'm a good candidate in that respect either. I certainly would help out if I screwed something up, but I don't really want an ongoing or permanent assignment as a webmaster.
I don't really want to do it, but if no one else does either, I will. It's true I'm unlikely to go anywhere. I've actually been TRYING to stop visiting this forum for a few months now because of these issues, and I still can't. LOL
Ironically, today I experienced only slight lag, no really atrocious performance problems for the first time in months.

Post by **jacob** » Fri May 10, 2013 12:30 am

By (4) I mainly mean: "If you break it, you fix it". I'm not asking for a lifetime commitment. I just don't want to be left in a lurch with a broken website.

Spartan_Warrior · Post by **Spartan_Warrior** » Fri May 10, 2013 1:06 am

Yeah, that's fair. I certainly wouldn't leave things in any worse shape than they are. If all else failed, I should be able to just restore the bbPress forum in its current state, like nothing ever happened.
@bigato: I agree, the DDOS issue is probably the main problem. IMO though, switching the forum software might solve this in and of itself. If bbPress is an older/less supported software, it's probably prone to a lot of different exploits that a better supported software (ala phpBB) might not have. (This was the case when I switched from IPB forums back in the day.)
On the other hand, if you can fix it by modifying the registration page, that'd be awesome. Then I don't have to do anything.

anomie · Post by **anomie** » Fri May 10, 2013 1:52 am

Hello,
SW if you want a second pair of eyes on that registration page, please feel free to send it to me; I'll be glad to take a look.
I have experience with some of / most of this stuff, but lack energy and time. I'd be glad to assist, especially over a few hour blocks on the weekend. And don't really need any $.
Anyone looked at the weblogs to see what's going on?
I hate the idea of not having search engines search the forums since kinda want the ere word to get out, and all..
Like Scott's(?) suggestion for trying out google search as a replacement if the reg page fix does not work.
The below can wait but I found it really interesting.

I ran the forum url through a website optimizer . You can see the result here:

http://gtmetrix.com/reports/forum.early ... m/jlbKQwfp
Looks like there is almost 1MB worth of graphics on each page, which I think is pretty high. Header image alone is 340 kb.
anyhow, glad the issue is being addressed.
lemme know if i can help.

anomie · Post by **anomie** » Fri May 10, 2013 2:33 am

2) The forum runs bbPress 1.0.2. It's based on an SQL database like wordpress. I have no idea about sql databases.

oh wow, what is really needed is to upgrade bbPress to current version. http://bbpress.org/download/ current version is " The latest version of bbPress is 2.3.2, released on May 6, 2013."
These popular opensource forum products like bbPress ( and phpBB ) are real susceptible to vulnerabilities and need to be kept up to date.
This is pretty straightforward fix ---

Back everything up, then upgrade everything to current versions!
Theme might need tweaking afterwords.

anomie · Post by **anomie** » Fri May 10, 2013 3:40 am

ugh by my final comments, I envisioned a Wordpress like simple upgrade (4 seconds in place, seamless).. the bbPress version 1.0.2 predates Wordpress . this upgrade would require a little research and a few dry runs on a test server; and "upgrade" would be to a WordPress site-based forum.
http://bbpress.org/forums/topic/upgrade ... -to-2-1-2/
Here is one WordPress plug-in I found that that does the upgrade for you.. (there are probably others) :
"

bbConverter currently supports converting your vBulletin 3x/4x, Invision Powerboard, and BBPress 1x forums over to the new bbPress 2x plugin setup.
bbPress is forum software with a twist from the creators of WordPress which can be found at: http://wordpress.org/extend/plugins/bbpress/

"
...

anyhow, did not mean to be flippant. best wishes.
(I do still think running current software is imperative)

Scott 2 · Post by **Scott 2** » Fri May 10, 2013 12:55 pm

I hadn't looked at the software version. I just assumed it was the latest version of a no longer supported forums product.
Anomie is right, running the current version of the forum software is important. It's been so long since an upgrade, that the transition to the current version might be as hard as switching products.

taekvideo · Post by **taekvideo** » Fri May 10, 2013 3:34 pm

I don't think making the registration page static would do much to help. Assuming bbpress has some kind of caching, that page would already be getting roughly the same performance as static html. If it doesn't have caching (couldn't find anything about it on google) then you'd be better off switching to phpbb which does, and would improve performance on all the forums dramatically.
The images are stored by most browser caches so only loaded the first time a user visits the site during that session. That being said... it wouldn't hurt to compress them and reconfigure them to stay cached for a ~week rather than 4 hours. Combine that with limiting/replacing the search feature and you could get the performance you need.
Lastly... I highly recommend *not* removing old posts. The first time I came to this site was through an old post showing up on google... and I'm sure it's the same for many others. Large forums with years of posts bring in loads of traffic from search engines, since they tend to cover lots of topics and have a lot of different writing styles.

ICouldBeTheWalrus · Post by **ICouldBeTheWalrus** » Fri May 10, 2013 11:54 pm

To anyone trying to work on this, please let me impart a bit of hard-earned wisdom about performance work in software:
Be careful assuming your past experience with something roughly analogous is applicable -- do careful experiments and measure the results. It is very easy to waste a lot of time, and sometimes make things worse by doing what "makes sense" in a complex system with so many moving 'parts'. It's often more productive to change one thing at a time and accurately measure the effect it has.

On a personal note, as a user, I think phpBB has a horribly cluttered user interface and some of its CAPTCHAs are completely impossible. But maybe that all is configurable and I've just seen the worse configurations. I really miss Usenet. It didn't force a single user interface down everyone's throat.

Post by **jacob** » Sat May 11, 2013 12:19 am

I'll say outright that I'm not willing to hand over the forum to a third party or the cloud. Who's to say that they will around in a few years?
To give an example, the average lifetime of a google service is 1459 days (~4 years). I wouldn't want to have anything critical depend on that
http://www.guardian.co.uk/technology/20 ... ces-closed

anomie · Post by **anomie** » Sat May 11, 2013 12:54 am

Hello again,
(hopefully here to contribute positively)
1. So the site pretty much broadcasts its version to any spambots etc looking to exploit a known vuln via its html source ( this is common, so just fyi) ..

<meta name="generator" content="bbPress 1.0.2" />

Fortunately bbPress appears to not have any serious performance vulnerabilities for version 1.0.2
http://www.cvedetails.com/vulnerability ... press.html
No performance vulnerabilities publicly stated, though have not looked at bbPress upgrade notes for other details.
2. Kudos to anyone stepping up to do this work -- SW and taekvideo . I agree with pretty much everything taekvideo has stated, btw.
3. A fix - upgrade to current or transfer to phpBB - does not have to cost any down time -- you test run the fix / transfer , then when finally satisfied success you make the dns change, or declare downtime and apply the change. So proof-of-concept , downtime wise, can be zero cost. And final implementation downtime can be minimized.
With web applications its vitally important to either secure by obscurity (code your own) or if you are using a popular product that thousands of other people are using, that you keep it up to date with security releases/ current version!
Just some information.

hth.

anomie · Post by **anomie** » Sat May 11, 2013 12:56 am

succintly put:

I think the problem should be fixed. Moving to a new server will not fix anything if the software is out of date.

anomie · Post by **anomie** » Sat May 11, 2013 2:27 am

That's a good point.
I don't know the criteria for getting a cve, but here are 2 more 1.0.2 exploits, maybe too minor to get listed:
bbPress 1.0.2 CSRF Change Admin Password

http://www.exploit-db.com/exploits/14214/
bbPress 1.0.2 <= Cross Site Scripting Vulnerability

http://toexploit.com/exploit/na/bbpress ... -scripting
That's just the first page of google search 'bbPress 1.0.2 vulnerability'
hmm here's top of second page...
71133 : bbPress bb-login.php re Parameter XSS

http://osvdb.org/71133
maybe same as above.. i dunno
Yeah, I dunno. I don't mean to detract from any effort to help the forum.
2. And where are all the techies? They are the dominant readers of this place, yes? Geez does the younger generation not have any LAMP guys? (no smiley face. insert anti-smiley face here.)

anomie · Post by **anomie** » Sat May 11, 2013 5:16 am

The cool thing is that -- hey this forum is not the first to experience spam or lag! -- current Wordpress installations have perfected the update process. (Backup before each upgrade recommended of course, but with WP becomes less & less necessary ..)
They do it analogous to Linux software updates -- inplace, instant, pain-free, modular, reboot not necessary for most cases.
This auto-update feature may exist in phpBB and other's.

I'm not claiming to be an expert, just a grunt.
http://en.wordpress.com/stats/ claims 65,488,269 Wordpress "sites in the world"
omg. whatever. Matt Mullenweg or whatever the founder of WP has a really nice and huge situation -- and he has created some nice open source products including spam protection for WP sites (Akismet ).... and sounds like they took bbPress under the WP wing ...
anyhow... WordPress is pretty fkin huge and amazing now in 2013.

4 second upgrade . i sht you not.
(and how did i become the shill for WP? ) ...

skinnyninja · Post by **skinnyninja** » Sat May 11, 2013 5:19 am

I will donate 50 bucks so long as:
1) At least someone else donates as well.

2) It helps to keep the forum going (whether by server upgrades or hiring someone to optimize things).

Post by **jacob** » Sat May 11, 2013 3:06 pm

I think option 4 or 5 are viable. The journals are the biggest loss in terms of not continuing threads. It wouldn't be too hard to start a new journal and link back to the old one though. All other threads don't last that long anyway.
The reason 1.0.2 didn't get upgraded was because it wasn't trivial. At some point there was a restructuring, as in, say, 1.1 is not just an advanced version of 1.0.02 but something different. As far as I know.
Wordpress is easy to upgrade because I can simply click a button and everything happens automatically. I can certainly do that

Where do we go now?
From what it looks like, I'd say Spartan takes the lead as the designated "general contractor"; gets the passwords (but don't share them); and can then semi-count on assistance/help/email with anomie, bigato, taekvideo, and others.
Any objections to that?
To me it sounds like migrating the database to new forum software (I prefer a simple interface with PM option) is the best option. I'm fairly open-minded to what you guys prefer, but I do reserve veto-rights

jennypenny · Post by **jennypenny** » Sat May 11, 2013 3:19 pm

Would we lose old PMs during the process? I have info in them that I don't want to lose (mostly people's personal info like phone numbers, email addresses, etc). I'll copy what I need today, but you might want to let others know if PMs might be lost.