Barry Ritholtz Interview - Tales from one of America's favourite investment commentators

Barry Ritholtz is one of the most widely followed investment commentators in America. He’s also the chairman and chief investment officer of one of the country’s fastest growing wealth advisers.

Over the past twenty years Ritholtz has forged a reputation for his no-nonsense views on finance and markets. He’s an advocate for low-cost, rules-based investing - and he’s also not afraid to call out bad industry behaviour when he sees it.

Apart from being a prolific writer on his Big Picture blog, Ritholtz hosts Bloomberg Radio’s Masters in Business show, where he’s interviewed some of the most influential names in finance and beyond.

Five years ago, he switched from a career in trading and research and set up Ritholtz Wealth Management. Together with Josh Brown (The Reformed Broker), Kris Venne (The Certifiable Planner) and Michael Batnick (The Irrelevant Investor), they tore up the existing rules for how wealth managers communicate with the outside world.

To build their business, they took their expertise in blogging and social media and used it to get their message out. This personable, authentic approach has worked well and their growing advisory team now doubles as some of the most widely followed investment writers around. Among them are people like Ben Carlson (A Wealth of Common Sense), Blair DuQuesnay (The Belle Curve), Tadas Viskanta (Abnormal Returns) and Nick Maggiulli (Of Dollars and Data).

I met Barry Ritholtz at his office on Bryant Park in Midtown New York City. I’d been in the city for less than 24 hours and just arrived from interviewing Joel Greenblatt (whom Ritholtz had interviewed for Masters in Business a couple of weeks earlier). This set the scene for a what turned into part-therapy, part-interview training, and part trying to get an interview out of Ritholtz, who, he said himself, is pretty hard to interview...

Barry, tell me about your writing. You were a commentator who rose up the ranks and now your interviews and opinions are some of the most followed in finance. How did that happen?

I started writing publicly back in the late 1990s. I should say I was writing quasi-publicly because nobody was actually paying attention. At the time, I was writing sporadically on Yahoo! GeoCities, which is hilarious to mention. I wanted to become a better writer, so I started reading other really good writers, and then writing for about an hour or so every day.

At the time of 9/11 my firm had an office in Two World Trade. That morning I called our head trader and took copious notes as he gave me a running narrative of everything that was going on. He was on Broadway heading up towards the Brooklyn Bridge when the towers collapsed. I posted it that night, and it was one of the first non-media descriptions of what happened to be posted online. It went insanely viral.

So what started out as a very sporadic set of posts became a daily thing, and then it became a twice-daily thing. Then it was my morning overview, followed later in the day by a chart you should see. Then I started doing the reads and the links of the 10 or 20 things you should see each day.

I’ve been doing that stuff for 15 years. There was a time when there weren’t a lot of people writing every day. I think I put out some interesting, quality things and that I was more right than wrong about a lot of big issues. Now it’s a firehose of everything from Twitter to LinkedIn to Facebook. I don’t think the same opportunity exists today that existed when I started, so I got a little lucky. I was interested in writing, and doing it in a very public venue. It was a wonderful opportunity that I don’t think anybody imagined would lead to all the places it’s lead to.

You often hear that there is way too much noise in investing. But do you think the kind of writing and content out there now is much improved on what used to be available?

When I started, most of the economics blogs were professors at colleges. There really were no financial blogs. The closest thing was when TheStreet.com first launched in the mid 1990s. Todd Harrison was Jim Cramer’s head trader at his hedge fund and he started writing about his trading posture. He used the metaphor of putting on his bull costume or his bear costume. “I have two legs in my bull costume”, he would say, or “I’m adding my third leg in my bear costume”, to reflect how bullish or bearish he was. That was the first time someone was talking about what they were trading in real time on the internet.

The professional financial media today has gotten much more sophisticated that it once was. But it was shockingly naïve and occasionally incompetent in the way it did its job.

Take the prediction business. There would be a parade of people coming on making predictions. Every article had a prediction - “so-and-so has said this is going to happen”. They never stopped to ask “how do you know know, what’s your track record of making these forecasts and why should anybody listen to you?” That’s how I eventually came to the conclusion that all forecasts are just marketing.

How did you get started in the investment industry?

I started as a trader, which I loved because it was so much fun. But you make a ton of money one month, you lose it, and you make it back the following month. At a certain point my wife said: “Listen, if you want to be a trader that’s fine, but it can’t be up 100, down 100 each month.”

So I made the decision to go into the research side. I joined a guy named Lawrence Hart at a firm called Prime Charter, which was later bought by Oppenheimer. He’d been a software analyst at Bell Labs in the ‘50s and he understood technology like nobody’s business - but nobody understood him.

I remember one of the retail brokers coming up to me and saying: “Look, we love Larry, but we don’t understanding a word the guy says.” He goes: “Do me a favour… I don’t need the technical explanation… just pick your two favourite recommendations from him and write it up in a way that we can understand.”

So I picked two: one was AOL, the gateway to the internet for middle America, which was five or six bucks at the time. It was a one-pager on why you could buy it from $6 to $15… and it could even go to $20! Obviously it ran up insanely. The other one was EMC, which I think was even cheaper.

The joke was my money was no good in any bar within a five-block radius. Every broker in that place that bought AOL and EMC at single digits, and they all went up 10x or 20x or 30x. You know, that’s a great start for a portfolio.

From there I become a strategist. I ended up at Maxim Group, which was a $7.5 billion firm formed by the merger of Herzog, Royce, Ernst & Co. and the big South African bank Investec. I was there for a couple of years. But at a certain point you come to the realisation that if you’re just in research, and not accumulating your own assets, you’re a cost not a revenue generator. I got tired of hearing that because it really doesn’t recognise what I think is the proper way to run a business.

You’ve been at the centre of the US investment industry for the last 20 years. When you look back now, what lessons do you think have been learned?

In the 1990s you had all these rumours, tips and mayhem, and then the dot-coms collapsed. Then you had the analyst scandals, the accounting scandals, the housing boom and bust and then you had the commodities run up, which collapses.

My pet theory is that about a decade ago, following the financial crisis, the individual investor in America turned around a said: “Hey listen, this whole market is rigged, so rather than play your game I’m just going to find a low-cost index fund. Here’s Vanguard. Here’s BlackRock. One is $6 trillion and the other is $5 trillion. I’m going to give them my money for the 30 or 40 years until I retire, and I’m not going to waste my time playing your game. Does it makes sense to spend a lot of time, energy and effort picking active fund managers or selecting individual stocks rather than owning a broad, globally diversified index of low-cost assets, rebalanced once a year for 30 or 40 years?”

Do you think that idea is now becoming more mainstream?

I’ve been saying the same thing for decades, it’s what we believe and it’s what we’ve always done. When I was at a firm that had a really interesting, quantitative methodology of stock selection, it was never a case of saying all your money should be in it. If you want to have a portion of your portfolio in something that is a little more juiced, and has the possibility of greater returns, first recognise that you’re taking on more risk and accepting the possibility of weaker returns. But if you want to have the slug of your portfolio in something active, I don’t have a problem with it. As long as the bulk of your money is in low-cost, global indices, no matter how much you mess around with everything else, you’re going to insure your retirement or your generational wealth transfer, or your foundation, is safe.

Tell me about Ritholtz Wealth Management. How did is get started?

It started with me, and then I brought in Josh, and then we brought in Kris and Mike: the four partners here. When we were first setting up, we took over a portfolio that was kind of abandoned. It had a dozen mutual funds in it and we slowly started replacing them with our own selections.

I met Josh about seven years ago at a conference, and I was impressed by him. He’s a funny guy, he’s very sharp and he writes really well. So I said to him: “Why are you on the sell side?” He said: “I hate the sell side; I’d love to be in the buy side. What do I need to do?” I said: “Cut that mullet, come over to us and it will be a great thing.”

He started working with me, and a couple of months later he’s says: “Dude, you have to stop turning down this money coming your way, it’s ridiculous.” I said: “The problem is the day-to-day operations are not my strength. That execution, dealing with the clients and the back office - I hate that stuff. Managing the money, the market analysis, the communication: I love that.” That’s basically how our partnership formed. We then went out and got Kris Venne, who runs the financial planning side, and Mike Batnick, who does the research, and that became the core nexus for us launching our own firm.

I knew we wanted to be fiduciaries from day one, but we spent a solid year thinking about what we wanted the firm to look like.We began to think about ways we wanted to run the firm, what we wanted to do, how we wanted to press it forward. I had a philosophy in terms of what our portfolio should look like. You’re constantly adjusting what you’re doing to conditions, but we launched and had a flow of new money come in.

Presumably you’ve been on a steep learning curve?

You very quickly learn that success and failure are surprisingly linked. When Visa became public and was trading at around $60 a share, we started buying it for our clients. As new clients came in we’d give them the standard portfolio. As the first year went by the stock went to $80, $100, $120 and eventually it went over $200. At this point you say: “Wait, this isn’t an asset class it’s a single, speculative stock.

If one client paid $100 and another client paid $200 and if I’m selling it at $180, it’s a big winner for one and it’s a big loser for the other. That doesn’t make any sense. But with an asset class you can count on some reasonable range of returns, no matter when a person is in it.

So going from being a specific trader and an abstract strategist to actually managing money was a pretty big change. When we launched the firm five years ago, setting up the portfolios in a way that was sustainable and not dependent on stock picking, was a priority.

You’ve taken a slightly different approach…?

I’d like to think that this is not your normal finance firm. Notice that there’s nobody on the phone. Our clients find us. We’ve built a model based on the idea of going out and saying to people: “Look, everything you think you know about investing is wrong - here’s what’s right. You can do it yourself but you have to be disciplined, you have to work hard, you have to understand what you’re doing and you have to keep your emotions in check. If you don’t want to, or don’t think you can do that, we can do it for you.” That’s really a very subtle message.

I guess it needs to be subtle because human behaviour and emotion can have a very negative bearing on investment returns, right?

I’ve spent 20-plus years reading and writing about it. It’s probably the single most important aspect of managing assets. You you can have the greatest stock selection in the world, but if your own behaviour has you panicking out in March ‘09 and not getting back in, your stock-picking prowess is irrelevant. I got emails from people in 2010, ‘11, ‘12, saying: “Look I followed you out of the market in ‘08, but when you said you were going back in in March ‘09 I thought you were crazy.” After you get enough of those emails, you think “maybe we should really get into this asset management thing?”

So we built this from scratch. There was no legacy business that we had to preserve. Most people in finance spend their days chasing clients. They need new assets, they need new accounts, they need new clients in order to generate revenues, commissions, transaction fees or whatever. So if you are spending your whole day chasing clients, then how much can you develop an expertise in anything to actually serve those clients? So I consider it our privilege. There’s no-one pumping the phones and it’s not the usual mania. You get to step back and think a little bit, and actually consider what is in the clients’ best interests, as opposed to being on that treadmill of more clients, more assets, more commissions.

But alongside the wealth management business your media work is obviously still really important, especially Masters in Business. You’ve built some strong relationships with some of those interviews, right?

It’s the most fun I have all week. When I sit down with Jack Bogle, Ray Dalio, Howard Marks, Cliff Asness...it’s insane.

With Cliff Asness I’m constantly charmed by him, even though he busts my chops. When I write anything, every now and then I’ll get: “That’s a pretty good column”. But usually it’s like: “Here’s why you’re an idiot who doesn’t know what you’re talking about.” Howard Marks of Oaktree Capital, who’s another superstar, tells a story of his chairman’s letters. He’s been writing these letters for more than 20 years. For the first 10 years, he’s writing these letters, and they’re literally folding them up, sticking them in envelopes and mailing them to all their clients. They would send out a few thousand, but there was not a peep from anybody. Imagine doing something with so much conviction but nobody responds, and you just keep doing it for a decade. So the fact that someone like Asness is busting my balls about something I wrote… I consider that a privilege!

Which has been your favourite interviews?

We’re over 200 interviews now. With most of them I normally get a sense that it was “good”, “very good” or “okay”. On rare occasions you’ll think, “oh my god this is a disaster” or “this interview is fantastic”. The Ray Dalio interview was really amazing. For a bunch of reasons I was in the right headspace, I knew his work and we chatted beforehand. So that was a fun interview that hit all my expectations. 

The same was true for Bill McNabb of Vanguard, Howard Marks and Cliff Asness. The one that was a giant surprise was Ken Feinberg, the Special Master of the September 11th Victim Compensation Fund. That was just horrifying and fascinating; he he told stories I’ve never heard before. So that was the one which was the most surprising because there are tons of stories that he just hasn’t shared publicly. It was an astonishing conversation.

Who would you most like to interview?

Charlie Munger would be fascinating... and I don't know what questions are left to ask Buffett that haven't been asked. Although you could have said the same thing about Jack Bogle, and that was a fascinating conversation. He comes into the room, a little hunched over. But he sits down, puts on the mic and... boom! He's just full voice, full strength for 90 minutes, it was astonishing.

The other white whale I have is Mike Bloomberg. I keep telling him that the series is incomplete without him. But he’s like: "No, no, I prefer to own the station."

Did you notice that I didn't look at my questions at all? I was taking a tip from you - I read that you just let your interviewee take the lead.

Those are the best interviews. My best interviews are when the ratio of asked questions to prepared questions is very low. But as for me, I'm an unruly interviewee, I'm not easy, and I'm over-caffeinated today.

Barry, thanks so much for your time.