39 & Looking to retire this year. Advice appreciated

[Caniretirenow]

39 years old. Virginia Resident. Wife and 3 young kids in first and second grade. Single income family. $175k salary.
I'd like to retire this year...one thing money can't buy is time!

Constructive ideas/thoughts welcome & very much appreciated

Assets:
$1M in brokerage accounts
$1.6M in Roll-over IRAs & 401k
$700k in Roth-IRA
$165k in 529 accounts
$30k HSA
100k cars + small brokerage

Only Debt is mortgage
$1.05M home with $380k mortgage - $2450 a month (2.6% rate)

NW is about $4.3M as of 1/25/2021


Expenses:
$5500-$6000 (Includes everything - Vacations - mortgage - utilities - gifts - misc - emergencies - taxes - insurance - etc. etc.)
Upper limit is about 7000 a month assuming 1000 a month for ACA keeping income below threshold.


My plan is to use the brokerage account to withdraw from, until fully depleted, and then start tapping into Roth contributions before i'm 59.5 to avoid penalties, and then tap into 401k/roll-overs as late as possible. Social security would be a nice bonus, but is not counted in my calculations.

- I'm assuming a very modest 8-9% withdrawal rate from the brokerage accounts until depleted.

- I'm assuming a modest 5% ROI on my brokerage account annually. My ROI over the past 10 years has averaged 33% annually. So i think 5% is reasonable.

- I'm assuming a very modest 5% growth on my IRA accounts until i withdraw from them.

- i plan to move out of stocks into mutual funds before pulling trigger...but can't do that in my brokerage account due to taxes.

I was going to quit at once...but my plan now is to ease into it is to request to go to part-time in May 2021. Use vacation to take 3 months off to do some of the things i want to do. Go back to work part time after long vacation. Re-evaluate portfolio on jan 1 and decide if i can fully retire depending on portfolio size.

Any thoughts?

[The Old Man]

You have expenses of ~$84 K and a net worth of $4.4 million. This works out to a withdrawal rate of 1.9%. For my portfolio my dividends are somewhat south of 2%. You could probably retire and your drawdown should come very close to your dividends/interest. The annual drawdown of your principal should be minimal at worst.

Yes, you can retire. You have a lot of planning to do to work out the exact details, but yes you can retire.

I would normally be concerned about the elevated stock market valuation; however, your need to sell assets should be very minimal, so it is not a significant concern.

[unemployable]

Your liquid net worth is a good bit south of 4.3M.

You have to live somewhere, and I presume even with no job you're golden-handcuffed into your kids' school districts, so you'd be reluctant to move. Fun fact, I've seen $70k houses in SW Virginia I'd be happy to live in, and yes I've been there many times and would be OK living there. Bitch of a state to get a speeding ticket in though, really the only thing holding me back. Anyway...

529 money is presumably side-pocketed for kids' college so not relevant to net worth calculation. HSA is similar and really isn't a lot of money in your world anyway.

Cars and non-investments should be written down to zero, shouldn't count in net worth unless they're true collectibles which means they're not daily drivers. Although you can't drive a sofa. But you still have to get around, sit somewhere, keep food cold...

So "actionable net worth" is closer to $3.3m. Still a 2-handle withdrawal rate, so no problem there. BUT! Most of that pelf, roughly two-thirds, is in deferred-tax accounts. You seem to acknowledge this, but a lot can happen in the twenty years until you can touch the money via normal means. One thing that can happen is you get bored. And I suspect you can't just, like, hire a nanny and hang out in Thailand for six months.

Anyway, you have other options, including backdoor Roth laddering, 72t withdrawals and simply pulling the money out of the IRAs and eating the penalty. Regarding the latter, someone in a similar net-worth orbit here decided to do just that a few weeks ago. I'd model various scenarios and the potential tax bite and your level of comfort with each. Personally, I'd be more comfortable with a paid-off house rather than owing the bank money with no job. Might you be forced to move somewhere cheaper right when the kids are applying to college? (Happened to me.) I don't have specific answers here, not for free at least. But my point is your issue is not if but how. And I would answer the how before dropping the keys off with the boss.

[IlliniDave]

I'd guess you've inherited much of what's in those retirement accounts? I say that because the balances are quite large given your age and income. The only reason that stands out is that if it is a very recent thing I'd encourage taking some time prior to making any large life decisions based on a recent sudden change in your financial picture. Seems like you are currently planning to ease into it, but have had other thoughts in the recent past. I'd advise the slower thoughtful approach.

I'm also guessing you live in a HCOL or VHCOL area (Beltway?). If you've been in your house at least 2 years, in your shoes I would strongly consider selling while home prices are high and relocating to a more moderate COL area and pocketing the difference in house prices (less what's due the mortgage company).

That's just a couple things I would be looking into at the outset. You may think them bunk, which is fine, no need for you to have to defend your plan.

I suspect you are well aware that the numbers imply about a 3% annual withdrawal rate (relative to financial assets). I'd be uncomfortable with that at your age. But I'm a fiscal arch-conservative.

You are correct time is a key consideration. You don't have to detail it all here, but I would suggest putting some serious thought into how you will spend your time. I'd also suggest using the transition period to track your spending in gory detail if you are not already doing that. You might be surprised. Lastly, when you have some ideas what you might be doing you can make some educated adjustments to your current actual spending profile to verify the numbers still work. For a while I was looking at options to get at retirement money prior to age 59.5, something that's now OBE, and it can be a little complex with draconian penalties for making errors. Given the amount of money involved a consultation with a tax attorney might be money well spent.

Lastly, I'd run those numbers by the crowd over on bogleheads.org. Most of us here are considerably less wealthy and you might find a few folks over there who can speak to a situation more similar to yours rooted in first-hand experience. Or you could fully embrace ERE and come to the realization you could retire viably with 10%-20% of your present net worth (ironically, perhaps, by spending much of your time in ways to avoid spending your money).

In you situation retirement it is certainly doable so long as manage your life/lifestyle to fit within the available financial resources.

[Caniretirenow]

Thanks for taking the time.
No, none of it has been inheritance. Just investments in solid stocks and being lucky. My account balance was about 150k on jan 2011.
I'm in Virginia so yes, HCOL. But moving is not an alternative at this point. We just bought this house last year and very happy with the area.
I've tried talking to a fee-based financial advisor but all they've tried to do is sell me whole life insurance or annuities lol while trying to charge me 2500 for any advice. Silly
I'll try to find a tax attorney per your suggestion.
Thanks again

[IlliniDave]

Thanks for taking the time.
No, none of it has been inheritance.

That's pretty dang impressive! I've done much better than I deserve (grew ~4X from 2011-2020) but that's an order of magnitude better than me. Congrats.

I've avoided financial advisors, your experience is typical, selling annuities/insurance products. They make sense for some people.

Good luck.

[Hristo Botev]

My thoughts, fwiw, as someone with similarly high income and expenses, who is roughly the same age, with a similar family situation living in a HCOL area. (And as someone who is well aware that my spending, if not my net worth, makes me a better candidate for Bogleheads, as opposed to ERE.)

I've not pulled the trigger on a fee-only adviser, because every time DW and I go down that road I'm quickly reminded of the sticker shock. That said, I could see maybe doing it at some point in the future (very non-ERE of me, I know), if ever I were to actually find myself in a position where I was dependent on my savings to cover all of my expenses, forever. But that's only because I know a couple fee-only advisers who I actually trust to keep my interests paramount. Also, I wouldn't do it until I was a much, much better informed consumer of those services--e.g., I and a handful of others on the forum are currently doing Jacob's financial curriculum (http://earlyretirementextreme.com/start ... sting.html). And it may very well be the case that by the time I finish that curriculum, I will no longer see any real value in hiring an adviser.

Also, lately, I've been much more interested in using my income to reduce my spending in "retirement," by paying off the house and my kids' schooling, and by developing the skills (e.g., "real" cooking competency, gardening, preserving and food storage, homebrewing, home repair, bike repair, and so forth) so that by the time I "retire," the actual money I need to cover my monthly expenses will be so low it'll likely be covered by some other side income, so that I won't actually need to really touch any of the money I've got invested. I.e., rather than the $6-7K in monthly expenses for our family of 4 (which includes mortage et al. and the kids' K-8 school tuition), the expenses will be more like $2-3k/mo.--which is more in line with 1 JAFI, if I'm allowed to divide up that monthly spending by 4 (which, I know, isn't really fair).

ETA: I'm impressed with the NW number, especially for a single income family--though, honestly, if I could do it over again, it'd have been much cheaper to give up one income and avoid the daycare expenses when the kids were younger, once you throw in the tax advantages.

ETAx2: I know housing is always a tough topic, especially with a young family. But, anecdotally, we also love the area we live in, despite super-high property taxes and high median home prices. The compromise we came to was to just put our expectations aside as to what sort of home we thought we should be living in (it helped that DW was watching a lot of House Hunters International on HGTV, and getting very excited about how well Europeans et al. seem to live with much less square footage), and to downsize into a much smaller home (a townhouse) that's not as desirable as our prior home, b/c it was built in the 80s and looks like it, among other things. So we're still in the same area (better located, in fact, from a walkability standpoint), but our house is now more like $325K as opposed to $650K; which means our property taxes are much lower, as are utilities and upkeep and insurance. And the home will be paid off in a couple years, as opposed to more like 25.

[The Old Man]

Second the recommendation for a financial advisor. Go with a fee only advisor. A standard advisor will only want to sell you things or get you into their proprietary actively managed funds. Choose an advisor that specializes in early retirement. Your major challenge will be accessing your retirement funds before age 59.5. A fee-only financial advisor should provide useful advice that doesn't involve selling you products.

Also, second the recommendation for Bogleheads.

[jacob]

I'd third the bogleheads or early-retirement.org. Your numbers are really in another universe. As far as I know the number of active forumites with similar numbers is zero. This is relevant for tax planning and portfolio risk-analysis.

Typical around here is $10-15k spending, $500-750k savings, 33-60 years saved, easy to cover portfolio failures with part time job or second career.

Yours is $70k spending, $4300k saved, 60 years saved, hard to cover spending with part time job or second career. So different concerns. What you basically need is a full-life plan that is tax-optimized (prob a bucket system) and focuses on portfolio protection/downside risk (since it would take a long time/much luck to replicate it).

Also see
https://earlyretirementextreme.com/myfo ... Levels.jpg

[Caniretirenow]

Thanks very much. Great advice. I'll take it.

[Sclass]

Wait! What?

Are we just running this guy off?

This sounds pretty interesting. Your numbers actually tell an interesting story. My thought was the philosophy on this forum was applicable to all levels of wealth. I think you can pick up some ideas here - especially hammering down your expenses.

I haven’t really thought hard about your numbers but it tells an interesting story. You know how to save. You are a rockstar investor. I think we can use a member like you around here. So what if we have to move a decimal point on your sheets. Scalable that’s the key. Your numbers are big and scary to the lentil eaters here...but I think you’re frugal. Ratiometrically speaking. You have saved money fast. You have lived below your means. You grew your savings while young. You totally belong here.

Bogoeheads? Yuck! Fee adviser? Hell no this guy doesn’t need one the way he’s making green.

As far as I know the number of active forumites with similar numbers is zero. This is relevant for tax planning and portfolio risk-analysis.

What? I should just leave. Really Jacob?

Edits - damn autocorrect. And I almost forgot, my advice, retire now. Hang around with us and learn how to cut unnecessary fat out of your budget...if that’s what you’d like.

[Caniretirenow]

Haha too funny.
Appreciate the note.

Yes, we're frugal in the sense that we live below our means (always have) and we cut where we can. For me, it's about value....not the dollar amount. I buy expensive stuff when i can justify their value....and will dread paying $5 for something that lacks value. Hehe
My car is $80k (EV) and i didn't hesitate for a second.... but just the other day, i struggled for 20 minutes pulling a trigger on a funny $15 face mask with a 25% discount and free shipping. I ended up not buying it lol

I find bikes in trash bins and side of the street....fix them up and sell them on Craigslist for 25-200 a piece....we eat at home almost all the time....our 2 trips a year are almost always to Florida...to which we drive over 15 hours (love it) and spend under $100 a night for luxury timeshare properties we rent from "desperate to rent" owners.

My point is that yes....i agree with your assessment. Good eye.

[Frita]

Riffing on the Dali Lama quote, “Do not try to use what you learn from Buddhism [ERE] to be a Buddhist [EREr]; use it to be a better whatever-you-already-are.“

[Sclass]

So I’ve been playing around with the tax deferred account problem in my mind while swapping an alternator in my car this afternoon.

Does it really make a difference if $1M is in a regular investment account and $3M is in IRAs? Let’s turn the problem upside down. What if $4M is in a regular investment account? You withdraw at 2% and have a happy retirement. Most of your expenses are covered by dividends.

Now if $1M is outside and $3M is locked down, the fear is that the $3M is somehow less viable right? But I think you just withdraw your $7000/mo and move forward. Obviously you know how to grow your money faster than that if you’re 39, a millionaire, no inheritance and a $175k/year job. In the meantime you just grow your tax deferred accounts the way they’re supposed to be done and take them when you get old.

Say you only make 7% on your account and you spend your $1M down to zero over 26 years while living on $7000/month. But your $3M in tax deferred accounts has swelled to $18,000,000 at 7% while waiting untouched. Hope I punched that out right.

How about a worse scenario. Say you earn 4% on your $1M and you run out of money in your regular investing account in 16 years while withdrawing $7000. You’re 4.5 years short of 59.5 and you’ll have to pay a penalty and tax to get your IRA early. By now, your IRA is worth $5.7M. You’re 55 with $5.7 minus whatever penalties and tax you have to pay. You only need to take out 4.5 years of money to get to age 59.5.

If you’re still making your paltry 4% you’ll need $350,000 from your retirement account. Tack on a 10% penalty and you have a $385,000 withdrawal. You still have $5.3M waiting for you at age 59 1/2. Plus another $1M or gains if you’re still invested at 4%. $6.3M at 59.5 sounds good.

I still think you’re good. I’m not sure the tax deferred account twist is such a big deal given your requirements. Hope I’m not totally off on the above calculations.

Short version is spend down your regular account and then withdraw some of your IRA early if needed.

And I almost forgot. Welcome!

[unemployable]

If you’re still making your paltry 4% you’ll need $350,000 from your retirement account. Tack on a 10% penalty and you have a $385,000 withdrawal.

You still have to pay income tax on this as if you worked for it -- which of course you did, in a prior year. In 2021 dollars/brackets you'd be in the 32% Federal bracket plus state tax is another few percent on top of that. So more like $600-650K to get $350k spendable. You''d be feeling that bite every year, and that assumes you're still married, haven't moved, tax laws and brackets haven't changed, inflation hasn't gone crazy... Being upper-middle class is a bitch taxwise.

Short version is spend down your regular account and then withdraw some of your IRA early if needed.

I don't have a problem with this strategy in general, busting your IRA that is. You certainly won't want to be forced to do it in a bear market, however, which tends to be when it happens. My point was you want to compare this option to others such as running 72t's from year one and deciding what you're most comfortable with.

[Sclass]

Wow that’s a huge tax bill. I didn’t think of that.

[Qazwer]

Tax optimization for you will be huge. You are clearly smart. It is time you learn the tax code. Play with alternative withdrawal methodologies etc
I am not sure you have checked out different techniques of early access to funds
https://www.madfientist.com/how-to-acce ... nds-early/
Is a good reference.
The next thing is risk mitigation. You might want to aim for safety more than gains at this point. Consider a boring diverse mutual fund/bond portfolio for a good chunk of your money. A loss with stocks might be a bigger deal to your lifestyle than gains.
Good luck

[Sclass]

Yeah that is pretty cool stuff I had no idea something like 72t existed. I can imagine how this got into the tax code. It seems kind of self defeating.

I guess reading back the OP said right up front he was going to access his funds in this order.

[IlliniDave]

So I’ve been playing around with the tax deferred account problem in my mind while swapping an alternator in my car this afternoon.

Does it really make a difference if $1M is in a regular investment account and $3M is in IRAs?

In an absolute sense, no, at Caniretire's (or your, based on your comments above) stash/net worth levels, it's in the noise. But as you said via prior post, part of ERE is avoiding unnecessary spending. There are legitimate ways to get money out of a tax sheltered account "early" without paying the penalty tax. The main one I'm aware of is the so-called 72t (aka SEPP) mechanism. There are multiple ways to do it, but it is rigid, and the penalties are harsh if it is not done correctly (up to 50% of what your withdrawal should have been per your SEPP option done correctly). So the conventional advice for people (like me) who are uncomfortable deciphering IRS code with confidence, is to seek advice from a tax accountant or tax attorney with expertise in the retirement realm when setting up the plan. In the scenario you outlined, maybe something like a $1K up front to save $35K in early withdrawal penalties or in an improbably worst-case situation, ~$180K in messed up SEPP penalties.

The other thing to potentially factor in is that taxable account assets tend to create ongoing tax liability, and often capital gain taxes when sold. The former will tend to decrease as you liquidate over time. The latter would tend to increase as you sell over time (the typical strategy from a tax perspective is to sell assets with the lowest unrealized gains first). Probably somewhat of a second-order effect in most cases.

I agree with your conclusion, at least for myself. To spend non-retirement money first seems to be the most widely accepted first-order prudent layman's approach. That's why the early withdrawal option has been OBE for me. The taxable part of my stash is now more than enough to get me to 59.5, a point in time that is distressingly close. I like the simplicity of it. For people in certain situations that want to game the system to it's fullest, it might make sense to look at look at some of the Roth conversion tactics that are out there, even before age 59.5.

ETA: When I think and type it goes slow, and I see someone beat me to the gist of my post. But I'll leave it around.

[2Birds1Stone]

You still have to pay income tax on this as if you worked for it -- which of course you did, in a prior year. In 2021 dollars/brackets you'd be in the 32% Federal bracket plus state tax is another few percent on top of that. So more like $600-650K to get $350k spendable. You''d be feeling that bite every year, and that assumes you're still married, haven't moved, tax laws and brackets haven't changed, inflation hasn't gone crazy... Being upper-middle class is a bitch taxwise.

How does that work? If he needs to tap $350k of spend from tax deferred accounts spread over 4-4.5 years then he would need to pull $85-90k/yr for expenses. Here in NY that would be 10-10.5% effective Federal Tax and ~5% effective state tax, and there is no FICA. Even adding a 10% penalty doesn't come anywhere close to $650k.