Ever since landing my first job post-college in 1999, I’ve been determined to build enough passive income in order to not have a job. A future that included getting into work by 5:30 a.m. and leaving after 7:30 p.m. each day for decades seemed too brutal to endure.
In 2009, I decided that if I could earn about $80,000 in passive income, I would leave my job permanently and work on Financial Samurai while traveling instead. So I left work in 2012 at the age of 34. Once my wife decided to join me in 2015, also at the age of 34, I decided to shoot for $200,000 in passive income.
With $200,000 a year in passive income, I would have enough income to provide for a family of up to four in San Francisco, given we bought a modest home in 2014. Now that we have a son, I’m happy to say that $200,000 indeed does seem like enough, especially if we can win the public-school lottery to avoid paying $20,000 to $50,000 a year in private-school tuition.
The 5 keys to building enough passive income
1. Save until it hurts each month. Most people think they are saving enough through their 401(k) or IRA, but they are not. Developing passive income requires an aggressive after-tax, after-401(k)/IRA savings amount each month, because you can’t draw from your pre-tax retirement accounts before the age of 59 1/2 without a 10% penalty.
You must sacrifice the pleasures of today for the freedom you will earn tomorrow. In my 20s, I shared a studio with my best friend from high school and drove beater cars worth less than 10% of my annual gross income. I’d stay until after 7:30 p.m. at work in order to eat the free cafeteria food. International vacations were replaced with staycations since work already sent me overseas two to four times a year. Clothes were bought at thrift shops, of course.
2. Focus on income-producing assets. Internet growth stocks may be sexy, but they provide no income. To build a large enough passive-income stream to survive, you must invest in dividend-generating stocks, certificates of deposit, municipal bonds, government Treasury bonds, corporate bonds, and real estate. You’re free to invest in non-income-producing assets for capital appreciation too. You just want to earn reliable income when the day comes to leave your job.
My favorite type of semi-passive income was rental property because it was a tangible asset that provided reliable income. As I grew older, my interest in rental property waned because I no longer had the patience and time to deal with maintenance issues and tenants. Online real estate became more attractive, along with tax-free municipal-bond income once rates started to rise.
3. Start as soon as possible. Building a livable passive-income stream takes a tremendously long time, largely because of declining interest rates since the late 1980s. Gone are the days of making a 5%-plus return on a short-term CD or savings account. Today, the best 12-month CD is at 2.5%, and the best money-market rate is about 1.85%, which is not bad, considering such rates were below 0.5% just a couple of years ago. Know that every $100 you save can generate at least $2.5 in passive income.
I knew I didn’t want to work 70 hours a week in finance forever. My body was breaking down, and I was constantly stressed. As a result, I started saving every other paycheck and 100% of my bonus since my first year out of college in 1999. By the time 2012 rolled around, I was earning enough passive income (about $78,000) to negotiate a severance and be free.
4. Calculate how much passive income you need. It’s important to have a passive-income goal — otherwise, it’s very easy to lose motivation. A good goal is to try to generate enough passive income to cover basic living expenses such as food, shelter, transportation, and clothing. If your annual expense number is $30,000, divide that figure by your expected rate of return to see how much capital you need to save. Unfortunately, you’ve got to then multiply the capital amount by 1.25 to 1.5 to account for taxes.
For example, $30,000 / 3% = $1 million in capital needed to generate $30,000 gross. But since you must pay tax on the $30,000 income, you really need closer to $1.25 million to generate $30,000 in after-tax income at a 3% rate of return.
With the birth of our son in 2017, we were aiming to generate roughly $250,000 a year in passive income by the time he starts kindergarten in 2022.
5. Make sure you are properly diversified. Capital preservation is underrated. We saw a lost decade for tech stocks between 2000 and 2010 after the first dot-com bubble burst. It actually took 13 years for Nasdaq investors to get back to even. Investors in the Borsa Istanbul stock market index just gave up 10 years’ worth of gains after they saw a plunge in their currency, partially due to increased tariffs by the US and a lack of confidence in the government. Your passive income needs to be properly diversified in order to take the hits.
I currently have 10 main sources of passive income, as you’ll see in the chart below, with bonds as my largest source, at 30.4% of the total. I’ve worked through the 2000 and 2008 meltdowns and don’t plan to ever lose so much money again.
Most of my passive income comes from stocks and bonds, real estate, and e-book sales
In 2017, I sold my San Francisco rental home, which had been generating roughly $60,000 a year in cash flow after expenses but before taxes. Selling the house brought my passive income down to roughly $150,000 a year, which was a significant 28% step backward.
Within six months of selling, however, I had reinvested the proceeds from the home sale and brought total passive income for 2018 back up to an estimated $203,724. I’m not sure I would have sold the house without a clear plan for reinvesting the proceeds, since I’m bullish on the SF housing market long term. However, because I did have a plan, and the challenges of raising a newborn and dealing with rowdy tenants left me feeling a bit stretched, I decided to simplify and sell.
Interest income ($7,620 a year, 3.7% of total)
I’ve got a $185,000 CD generating 3% interest coming due. Although the return is low, it’s guaranteed. The CD gave me the confidence to invest more aggressively in risk over the years. My online interest income has come down since I aggressively deployed some capital at the beginning of the year and again during the February market correction. You’ll see these figures in my quarterly investment-income update.
Don’t underestimate the value of your cash and risk-free income, especially during times of uncertainty. The last thing you want to do is be a forced seller in a downturn because panic will be everywhere. Cash allows you to take advantage of corrections, pay for unexpected expenses, and worry less about your risk assets.
Stocks and bonds income ($103,344 a year, 50.7% of total)
In 2017, I ended up deploying roughly $611,000 into stocks and $604,327 into municipal bonds. The stock allocation should boost dividend income by about $12,500 a year, and the municipal-bond portion should boost income by about $18,000 a year after tax ($26,000 pre-tax). Therefore, total passive income gets an about $38,500 lift, which recovers over half of my $60,000 loss from selling the house.
A good portion of my stock allocation is in growth stocks and structured notes that pay no dividends. The dividend income that comes from stocks is primarily from S&P 500 index exchange-traded funds. Although this is a passive-income report, as I’m still relatively young I’m more interested in building a large financial nut through principal appreciation rather than through dividend investing. As an entrepreneur, I can’t help but have a growth mindset.
With interest rates reaching two-year highs, I will be allocating more cash flow to bonds and high-interest short-term CDs for the remainder of the year, thereby boosting passive income.
Real-estate income ($43,080 a year, 21.1% of total)
I’ve now only got an SF rental condo and a Lake Tahoe vacation rental in my real-estate-rental portfolio. Although I miss my old house, I certainly don’t miss paying $23,000 a year in property taxes and another mortgage, and dealing with leaks and managing terrible tenants. I drove by the other day and couldn’t believe how much noisier and busier the street was than where I currently live. I wouldn’t be comfortable raising my son there.
In January 2018, I missed my chance of raising the rent on my new incoming tenants because it didn’t come to mind until very late in the interview process. I didn’t write about my previous tenant’s sudden decision to move out in December 2017 after 1.5 years, because they provided a relatively seamless transition by introducing their longtime friends to replace them. I didn’t miss a month of rent and didn’t have to do any marketing, so I felt I’d just keep the rent the same.
After these tenants move out, I’m thinking of just keeping the rental empty with furniture. It sounds stupid to give up $4,200 a month, but I really hate dealing with the homeowner association, move-in/move-out rules, and maintenance issues. Given that the condo doesn’t have a mortgage and I have to pay taxes on some of the rental income, I’m not giving up that much. The condo can be a place for my sister, parents, or in-laws to crash when they want to stay in SF for longer than a week or two.
The Lake Tahoe property continues to be 100% managed by a property-management company. It feels amazing not to have to do anything. I can’t wait to bring up my boy this coming winter to play in the snow! I could go up this winter, but I want him to be able to walk and run comfortably before he goes. I’ve been dreaming of this moment for over 10 years now. The income from the property is highly dependent on how much it snows. Summer income is always very strong.
Alternative income ($49,680, 24.4% of total)
Book sales ($36,000 a year): Sales of How to Engineer Your Layoff” continue to be steady. I expect book sales to rise once the economy starts to soften and people get more nervous about their jobs. It’s always best to be ahead of the curve when it comes to a layoff by negotiating first. Further, if you are planning to quit your job, then there is no downside in trying to engineer your layoff so you can get WARN Act pay for several months, a severance check, deferred compensation, and healthcare.
What’s crazy is that my book income is more than my SF condo-rental income. Yet I didn’t have to come up with $1.2 million of capital (the minimum cost to buy my condo today) to create my book. All I needed to create my book was energy, effort, and creativity. I truly believe that developing your own online product is one of the best ways to make money.
Venture debt ($12,240 a year): The first venture-debt fund has returned almost all my initial capital, so I decided to invest $200,000 in the second fund. I took a risk investing $150,000 in my friend’s first fund, so I’m hoping there’s less risk in the second fund, given he has four more years of experience on top of his 12-plus years of experience running a venture-debt portfolio for another company.
The whole idea of investing in venture debt is trying to get a mid-to-high-teens annual return with less risk than private equity. Venture debt lends money to well-funded private companies with one-to-three-year terms. They go in and out, collect their interest, and sometimes get a warrant. They’re higher on the capital structure as well.
Peer-to-peer lending ($1,440 a year): I’ve lost interest in P2P lending since returns started coming down. You would think that returns would start going up with a rise in interest rates, but I’m not really seeing this yet. Prosper missed its window for an initial public offering in 2015-16, and LendingClub is just chugging along. I hate it when people default on their debt obligations, which is why I haven’t invested large sums of money in P2P. That said, I’m still earning a respectable 7% a year in P2P, which is much better than the stock market is doing so far in 2018!
Real-estate crowdfunding ($9,600 a year): Once I sold my SF rental, it was natural to reinvest some of the proceeds into real-estate crowdfunding to keep sector exposure. I didn’t invest a lot in some of my favorite real-estate investment trusts because I felt a rising interest-rate environment would be a stronger headwind for REITs. But if I could be more surgical with my real-estate investments by identifying specific investments in stronger employment-growth markets, I thought I could do better.
In the summer of 2017, I first reinvested $250,000 into a RealtyShares domestic-equity fund. I already had $250,000 invested with it, and I liked the projects it was choosing. After spending the rest of the year making lowball offers on SF real estate and losing, I invested another $300,000 in the fund in December 2017.
Given that 100% of my real-estate crowdfunding is equity investments, there is no set monthly dividend. Each of the 12 investments in the fund has a different timetable and objectives. I’m simply estimating that I’ll earn $9,600 for the year.
If my RealtyShares investments achieve their objective blended return of 15% a year, I could earn a compounded $70,000 to $120,000 a year, which would really boost my passive-income returns. However, I don’t expect them or any private investments to achieve their target. Instead, I’m hoping for a solid 8%-a-year return instead.
It feels good to simplify
It was easier recouping the lost $60,000 in rental-property income than I expected. For so long, my primary mindset for passive income was rental income. Having $815,000 less mortgage debt but still generating roughly the same amount of passive income with a much larger cash balance feels great. Further, my passive-income portfolio got even more passive, which is good as a stay-at-home dad to a newborn.
In order to live off $200,000 a year in gross passive income in expensive San Francisco, we own a humble 1,920-square-foot, three-bedroom, two-bathroom home and drive a car worth less than one-tenth of our gross income. We never buy any new clothes, and we take full advantage of all the free things the city has to offer during the weekdays.
That $200,000 a year might sound like a lot to you, but the median home price in San Francisco is roughly $1.6 million or almost eight times our annual passive income. For a family of three in 2018, the Department of Housing and Urban Development declared that income of $105,700 or below was “low income.” Therefore, I consider us firmly in the middle class.
We will continue to save and invest in more passive-income-generating investments just in case our son doesn’t win the San Francisco public-school lottery. That’s right — even if you pay $20,000 a year in property tax for a decade, your child has no guarantee of going to your neighborhood’s public school.
Although we forewent many luxuries since we graduated from college, there is not a day that goes by where we aren’t thankful for being able to leave our jobs for good at 34. We could have made more money working, but we decided that being free was more important. There’s always another dollar to make, but never another second to create.
Sam started Financial Samurai in 2009 as a way to cope with the financial depression. In 2012, he negotiated a severance after 13 years in finance in order to be free. He now spends his time taking care of his baby boy, writing about personal finance, and playing tennis.