If you want donations, could you use use the "tip" buttons jacob already has set up? Or are you going to set up a paypal account? I would think tip-the-web is easier, but more likely to disappear at some point.
@jacob--will this affect the wiki?
Donations to Upgrade Forum Server?
-
jacob
- Site Admin
- Posts: 15979
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
@jennypenny -
No. The wiki, the blog, and the forum all run independently on separate software but on the same server.
I suspect there's a higher chance of loss for the PMs since PM is facilitated by a plugin. But I don't know. (I strongly recommend not keep anything important in there anyway.)
Don't use the tip buttons---they go to my tip account. (I presume we're talking tips to those who are actually going to do the work, not me.)
No. The wiki, the blog, and the forum all run independently on separate software but on the same server.
I suspect there's a higher chance of loss for the PMs since PM is facilitated by a plugin. But I don't know. (I strongly recommend not keep anything important in there anyway.)
Don't use the tip buttons---they go to my tip account. (I presume we're talking tips to those who are actually going to do the work, not me.)
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
Thanks for summarizing, bigato.
So, it sounds like we've decided on upgrading the forum to new version/software as the most efficient first line of attack against the performance issues. I agree. This will also have benefits overall in reducing the forum's vulnerability.
I should be able to do bigato's Option 4 (new forum from scratch with current one available as archive) with no or negligible downtime. From what I recall instantiating a new forum on a server is pretty plug-and-play. As long as Jacob's server allows multiple SQL databases(?), both forums should be able to run simultaneously. At that point, registration, posting, and maybe even logging in on the old (bbPress 1.0.2) forum can be disabled and it can serve as an archive.
Advantages of this method are: Simplicity and ease of implementation; no messy database exporting/translating; current forum lives on in search engines and link references--no risk of dead links, etc.
Disadvantages are: bbPress forum would still exist on the server, so if anything intrinsic to its mere existence/continued access is slowing down the server, the problem may not be solved; people would have to start new accounts and make new journals.
Option 5 is more complicated and is basically what I've been assuming throughout my posts--updating the forum, but then trying to migrate the old data into the new forum. I'm not as confident that this effort will be successful (at least in my hands), and I'm also not aware of a way of doing it that wouldn't require at least a few hours downtime (so no new posts are made in the old forum while exporting that data and importing it to the new one).
If folks would be happy enough with Option 4, I'm confident in my ability to do it. That said, I notice bigato seems very knowledgeable about this stuff (and a little more interested in it than I am) and has also volunteered. I am quite happy to pass the torch to him if he and others don't mind. I only volunteered because no one else was speaking up! I consider myself more of a candidate of last resort...
Also, I second this:
"2. And where are all the techies? They are the dominant readers of this place, yes? Geez does the younger generation not have any LAMP guys? (no smiley face. insert anti-smiley face here.)"
I'm a technical writer with a Philosophy major, for crying out loud! (anti-smiley face)
Bigato, if you do want to take the lead role, I am happy to offer support in terms of how to back things up and export the database, or anything else you're unsure on that I might know.
On the other hand, if I am to be the lead/general contractor, does anyone have a good FTP client they can recommend for uploading files to the server? I seem to recall I used WS_FTP back in the day, but it doesn't seem to be freeware any longer.
Also, after finalizing who is taking the lead, the final decision, and perhaps more salient to the community at large, is exactly which software to use. I am also still in favor of phpBB mostly because of its broad support and my brief experience with it in the past. However, perhaps we should narrow down the candidates and take votes?
So, it sounds like we've decided on upgrading the forum to new version/software as the most efficient first line of attack against the performance issues. I agree. This will also have benefits overall in reducing the forum's vulnerability.
I should be able to do bigato's Option 4 (new forum from scratch with current one available as archive) with no or negligible downtime. From what I recall instantiating a new forum on a server is pretty plug-and-play. As long as Jacob's server allows multiple SQL databases(?), both forums should be able to run simultaneously. At that point, registration, posting, and maybe even logging in on the old (bbPress 1.0.2) forum can be disabled and it can serve as an archive.
Advantages of this method are: Simplicity and ease of implementation; no messy database exporting/translating; current forum lives on in search engines and link references--no risk of dead links, etc.
Disadvantages are: bbPress forum would still exist on the server, so if anything intrinsic to its mere existence/continued access is slowing down the server, the problem may not be solved; people would have to start new accounts and make new journals.
Option 5 is more complicated and is basically what I've been assuming throughout my posts--updating the forum, but then trying to migrate the old data into the new forum. I'm not as confident that this effort will be successful (at least in my hands), and I'm also not aware of a way of doing it that wouldn't require at least a few hours downtime (so no new posts are made in the old forum while exporting that data and importing it to the new one).
If folks would be happy enough with Option 4, I'm confident in my ability to do it. That said, I notice bigato seems very knowledgeable about this stuff (and a little more interested in it than I am) and has also volunteered. I am quite happy to pass the torch to him if he and others don't mind. I only volunteered because no one else was speaking up! I consider myself more of a candidate of last resort...
Also, I second this:
"2. And where are all the techies? They are the dominant readers of this place, yes? Geez does the younger generation not have any LAMP guys? (no smiley face. insert anti-smiley face here.)"
I'm a technical writer with a Philosophy major, for crying out loud! (anti-smiley face)
Bigato, if you do want to take the lead role, I am happy to offer support in terms of how to back things up and export the database, or anything else you're unsure on that I might know.
On the other hand, if I am to be the lead/general contractor, does anyone have a good FTP client they can recommend for uploading files to the server? I seem to recall I used WS_FTP back in the day, but it doesn't seem to be freeware any longer.
Also, after finalizing who is taking the lead, the final decision, and perhaps more salient to the community at large, is exactly which software to use. I am also still in favor of phpBB mostly because of its broad support and my brief experience with it in the past. However, perhaps we should narrow down the candidates and take votes?
-
jacob
- Site Admin
- Posts: 15979
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
Question: If you simply start a new forum next to the old forum, how would that solve the traffic problem to all the posts in the old forum?
The solution should certainly aim to migrate the posts to the new forum. It's okay if intra-links are lost (I don't think there are that many). Archiving should be the last resort.
The solution should certainly aim to migrate the posts to the new forum. It's okay if intra-links are lost (I don't think there are that many). Archiving should be the last resort.
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
If the issue is spamming of the registration page, disabling the registration page on this (the "old" forum) would theoretically stop the bottlenecking.
The whole solution of upgrading the forums is premised on the idea that there's something about this forum software or a vulnerability in it--not the level of traffic itself--that's causing the performance issue. If it's just the level of traffic, we're back to square one of requiring a server upgrade with more bandwidth, as far as I know.
Migrating the data to the new forum is really just one extra step, though, since there appears to be a bbPress 1.0.2 --> phpBB X.X script. I can try that full migration (option 5) as the goal, as originally intended. But I know I can do Option 4, as a fall-back plan.
The whole solution of upgrading the forums is premised on the idea that there's something about this forum software or a vulnerability in it--not the level of traffic itself--that's causing the performance issue. If it's just the level of traffic, we're back to square one of requiring a server upgrade with more bandwidth, as far as I know.
Migrating the data to the new forum is really just one extra step, though, since there appears to be a bbPress 1.0.2 --> phpBB X.X script. I can try that full migration (option 5) as the goal, as originally intended. But I know I can do Option 4, as a fall-back plan.
-
jacob
- Site Admin
- Posts: 15979
- Joined: Fri Jun 28, 2013 8:38 pm
- Location: USA, Zone 5b, Koppen Dfa, Elev. 620ft, Walkscore 77
- Contact:
I tried blocking the reg page at the server level (using httpaccess). That didn't solve the problem.
I'm in favor of bbphp with migrated posts. Translating the internal hyperlinks, but I don't think it's that important. There can't be that many of them.
I think Spartan gets the keys then. (Do back up the site and have a plan to restore ASAP if something goes wrong. We are talking live data here.)
Bigato, did you get the PM I just sent?
I'm in favor of bbphp with migrated posts. Translating the internal hyperlinks, but I don't think it's that important. There can't be that many of them.
I think Spartan gets the keys then. (Do back up the site and have a plan to restore ASAP if something goes wrong. We are talking live data here.)
Bigato, did you get the PM I just sent?
I strongly recommend using the SSH interface of the webhost, if one is provided, instead of FTP. Generally, FTP is outdated (unsecure), and usually is disabled these days in favor of SSH (which provides SFTP & SCP too). So SFTP or just SSH directly into the user account and use the command line..
An example of backing up the bbpress database:
I would typically take a peek at the configuration files of bbpress to determine the mysql server address/login/pass, assuming it's in plaintext like in wordpress installations. I think shared webhosts usually keep MySQL on a different server from the regular user account. Then I would run a command like:
mysqldump -h actual-mysql-server.com -u login -p bbpresstablename > bbpress-dump.sql
-p option will cause it to ask for a password
The login/pass won't necessarily be the same as the actual webhost/ssh account.
Once it's been dumped, then it's easy to import that into another database if necessary.
--
I'm somewhat more in favor of upgrading bbPress first and seeing how that goes.. maybe try phpbb later?
An example of backing up the bbpress database:
I would typically take a peek at the configuration files of bbpress to determine the mysql server address/login/pass, assuming it's in plaintext like in wordpress installations. I think shared webhosts usually keep MySQL on a different server from the regular user account. Then I would run a command like:
mysqldump -h actual-mysql-server.com -u login -p bbpresstablename > bbpress-dump.sql
-p option will cause it to ask for a password
The login/pass won't necessarily be the same as the actual webhost/ssh account.
Once it's been dumped, then it's easy to import that into another database if necessary.
--
I'm somewhat more in favor of upgrading bbPress first and seeing how that goes.. maybe try phpbb later?
Hello,
Filezilla is great free FTP / SFTP client:
https://filezilla-project.org/download.php?type=client
Filezilla is great free FTP / SFTP client:
https://filezilla-project.org/download.php?type=client
I had a busy week and am catching up on this conversation. My $.02 FWIW:
- I agree, with reservation, that options (4) or (5) are the proper way to go. We are caught in the crossfire of two best practices that are contradictory in our case. The first best practice is to run up to date, actively maintained software so that security patches are available and can be applied in a timely manner. However another best practice is to diagnose problems and determine their root cause rather than sweeping them under the rug. I agree vociferously with
Be careful assuming your past experience with something roughly analogous is applicable -- do careful experiments and measure the results. It is very easy to waste a lot of time, and sometimes make things worse by doing what "makes sense" in a complex system with so many moving 'parts'.
It makes me uneasy to abandon the current software without ever determining why it was slow. IMO if we start having similar problems with the new software we really need to diagnose the problem. That will be easier with a software package that is modern and has a sizable community.
- Personally I refuse to maintain anything PHP-related, which is part of why I suggested Discourse. But if someone else is volunteering to be an ongoing sysadmin and is more comfortable with a different package, I will defer to them. Another best practice is that the person(s) doing the work should have the strongest veto power.
- I have periods of intense business (e.g. last week) which could make me a negligent sysadmin so I'm not throwing my hat in that ring at this time. However I am willing to help with the one-time transition. In particular I am willing to help write a migration script.
- I agree, with reservation, that options (4) or (5) are the proper way to go. We are caught in the crossfire of two best practices that are contradictory in our case. The first best practice is to run up to date, actively maintained software so that security patches are available and can be applied in a timely manner. However another best practice is to diagnose problems and determine their root cause rather than sweeping them under the rug. I agree vociferously with
Be careful assuming your past experience with something roughly analogous is applicable -- do careful experiments and measure the results. It is very easy to waste a lot of time, and sometimes make things worse by doing what "makes sense" in a complex system with so many moving 'parts'.
It makes me uneasy to abandon the current software without ever determining why it was slow. IMO if we start having similar problems with the new software we really need to diagnose the problem. That will be easier with a software package that is modern and has a sizable community.
- Personally I refuse to maintain anything PHP-related, which is part of why I suggested Discourse. But if someone else is volunteering to be an ongoing sysadmin and is more comfortable with a different package, I will defer to them. Another best practice is that the person(s) doing the work should have the strongest veto power.
- I have periods of intense business (e.g. last week) which could make me a negligent sysadmin so I'm not throwing my hat in that ring at this time. However I am willing to help with the one-time transition. In particular I am willing to help write a migration script.
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
Just an update: I've been given the keys to the forum. My goal is to export the SQL data from the forum and test whether it can be converted to phpBB on a local server. If that's successful, I'll move forward with implementing the update to phpBB, and I'll support technical issues on that installation for the foreseeable future.
The question is whether that move will solve the server latency issues. If it doesn't, or if I can't convert the SQL data in the first place, I'll reach out to you guys for help. So to everyone who's been offering advice, thank you and please continue to do so.
However, I do agree with the school of thought that whether the server latency issue is solved or not, it's a good idea to get the forum on more updated software regardless. That's my primary objective for now, and probably my main expertise to offer here. If we eventually do need to investigate the server latency issue more closely and "diagnostically", someone else will probably need to step in.
Due to surprise personal and business issues myself, I haven't been able to start looking at any of this yet, but I intend to do so this week. I'll keep you all informed.
The question is whether that move will solve the server latency issues. If it doesn't, or if I can't convert the SQL data in the first place, I'll reach out to you guys for help. So to everyone who's been offering advice, thank you and please continue to do so.
However, I do agree with the school of thought that whether the server latency issue is solved or not, it's a good idea to get the forum on more updated software regardless. That's my primary objective for now, and probably my main expertise to offer here. If we eventually do need to investigate the server latency issue more closely and "diagnostically", someone else will probably need to step in.
Due to surprise personal and business issues myself, I haven't been able to start looking at any of this yet, but I intend to do so this week. I'll keep you all informed.
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
Actually, I've been away from home with GF/family for most of the past week and (still) haven't gotten a chance to look at this forum, but rest assured it's on my to-do list. Right now it's looking like the soonest I'll get time will be tomorrow or Friday. Bigato, I'll let you know if I run into anything you could help with.
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
Well, I cleared a few hours this afternoon to start looking at this, but lo and behold, the forum was down almost all day, and I forgot to copy some of the access credentials that I had here in PM. So I only ended up having about an hour to work on it. So far I've exported the forum database to my hard drive in both XML and SQL, downloaded WampServer as a local test server, and grabbed the latest installation of phpBB3. I think I still need to get some files off the web server so I can replicate the current forum installation locally. Then I just have to figure out how to use WampServer (never used a local server before) and start seeing about converting the data. So far, so good.
-
Spartan_Warrior
- Posts: 1659
- Joined: Fri Dec 02, 2011 1:24 am
$2k sounds pretty exorbitant. I'm still hoping we can avoid such an expense. Unforunately I volunteered for this project at a bad time, in so far as lately I've had a lot of paid overtime, potentially profitable side projects, random house maintenance issues, and time with girlfriend and family, all of which take priority over a project like this.
I promise I will look into this when I have some free time this Friday and hopefully run the conversion on the test server. If that works, it's just a matter of declaring downtime so I can set up the phpBB and convert the database. I estimate there are about four to six hours of work remaining--far from 3 days!--but hopefully I'm not underestimating...
(The point about plug-ins is interesting. Not sure if anything extra is running on this bbPress installation that might complicate things.)
That said, if someone else wants to take over, and Jacob's okay with that, I will gladly turn over the mantle.
Note: If the regular old SQL conversion scripts don't work, I do have a back-up plan, as I *believe* there is a Wordpress plugin that automagically converts "bbPress 1.x" to phpBB or bbPress 2.0. (It might have even been mentioned earlier in this thread.) If possible I want to avoid mucking around with that though, as I'd have to get into Jacob's wordpress account.
I promise I will look into this when I have some free time this Friday and hopefully run the conversion on the test server. If that works, it's just a matter of declaring downtime so I can set up the phpBB and convert the database. I estimate there are about four to six hours of work remaining--far from 3 days!--but hopefully I'm not underestimating...
(The point about plug-ins is interesting. Not sure if anything extra is running on this bbPress installation that might complicate things.)
That said, if someone else wants to take over, and Jacob's okay with that, I will gladly turn over the mantle.
Note: If the regular old SQL conversion scripts don't work, I do have a back-up plan, as I *believe* there is a Wordpress plugin that automagically converts "bbPress 1.x" to phpBB or bbPress 2.0. (It might have even been mentioned earlier in this thread.) If possible I want to avoid mucking around with that though, as I'd have to get into Jacob's wordpress account.