What's your argument against doing grant lotteries? You mention Goodhart's law up front. But part of that idea is that if you replace suboptimal metrics with other metrics, people are just going to figure out other ways of optimizing for those new metrics. Even if they're fuzzier and less specific. So why not go all the way toward the unhackable? Maybe there's a comprise in there—with a threshold for who is a sufficiently trustworthy / competent applicant, or a weighted probability based on some subset of your proposed criteria.
The main problem that I have with your Trust Windfall argument is that it sounds, in essence, like the way being rich works. You get a little bit rich. This gets you closer to other rich people, who begin to trust you more. Then you get more rich, because they're in charge of the assets and these assets get allocated disproportionately to rich-adjacent you via trust. That totally seems like something you could hack! In your example, it's kind of like replacing one Lin-Manuel problem with another. The extent to which lots of people trust him is also going to go up after he won a Tony. So is the criteria to identify the trustworthy people who are most currently undervalued by the market, e.g., your gulag thespian friend?
To put my concern another way: you point out this is overtly based on choosing people who are your friends. I feel like people will either unconsciously choose in-group members / people they are interpersonally fond of; or they will try to address the unconscious bias and choose people who blatantly different from themselves. Both are hard to square with exactly how much they cloud of judgments of "merit". Curious to know what you think about that!
At any rate, I agree with the primary argument of your piece: Oxford degrees are useless.
Thanks for sharing this Adam--I really appreciate the problem statement. One concern with your proposed solutions is that networks tend to be very homophilic. I can easily imagine sincere, well-intended use of social networks to leverage trust working well, but also being very homogenous, and open to challenges that they constrain diversity.
Haha, I'm guessing you were applying to the FTX Future Fund? (We were also doing the same for Manifold Markets.) FTX FF is actually already trying something similar to Trust Windfalls with their regranting program: https://ftxfuturefund.org/announcing-our-regranting-program/
The entire space of new grant funding mechanisms seems to be very hot at the moment! I'm personally bullish on Impact Certificates/Retroactive Public Goods Funding, the "new type of philanthropic institution" that Scott alluded to.
A "Secret Shopper" method to award grant money? I like the tone of that. As you touched on, the greatest hurdle is for the Agent to maintain that secrecy-anonymity. One blabby remark and the Agent would be the proverbial lottery winner with "friends" and "long lost relatives" coming out of the weeds.
My own idea to supplement the usual alternatives of flat or random (lottery) funding is a bootstrap method: let anyone apply for a small amount, e.g. $20,000, where the bar for approval is basically being able to spell their own name. Then, if they can demonstrate success or progress (including null results), they unlock the chance to apply for double the money in a similarly streamlined process, and the process repeats.
Trust Windfalls are a really interesting idea. I wonder if they would end up captured, though? There's an adage that says "A's hire A's, but B's hire C's", meaning that people who are good-at-what-they-do in a general sense have the skills to recognize and admire other people who are good-at-what-they-do and want to work with them, whereas posers just want to work with people who will suck up to them.
So say you institute a Trust Windfall Foundation and identify your first round of 100 Agents. You do your best to identify people who are trustworthy, but because recognizing ability is hard, only 90 of them are really "A"s and the other 10 are "B" or "C"s. Now in the next round, a bunch of people get funded, and the baton gets passed. The people who get the batons also have the same level of noise as you did, so 81 of them are "A"s, and 19 are "B"s and "C"s. After a few iterations of this simple model, you end up in a situation where almost all of the Agents are not-As, and almost all of the money goes to cronies or people the agents want to suck up to them.
1. Is this model accurate? Probably not: generally people are pretty good at understanding other people, and I feel like people are generally pretty trustworthy as long as they're not incentivized not to be. But how do you incentivize the agents themselves to grant money to people who will actually do interesting things with it, rather than to people who are fun to hang out with or will use the money in ways that benefit the agent? How do you incentivize the agents to not reveal they are agents to their friends and use their granting power to encourage cronyism and suck-uping?
2. How to get around this? I suppose you, the Foundation, could keep records on the success of each grant, and terminate germlines that aren't fruitful after a few generations, while seeding new ones. I suppose you could select next-agents from a list of suggested ones, and inform your choice using information from broader sources.
This whole thing sounds a bit like a sampling problem to me. Given the social network, how do you get your money-sampling Monte Carlo chain to spend the most time around the nodes that maximize "Value"? A big part of that is defining "Value".
Could the idea of a basic income guarantee be viewed as the state trusting everyone? At least enough to ensure they have a walk away option from any unreasonable situation. Of course, it would not be substantial enough to let everyone do their best work e.g. a nuclear scientist with an intriguing (but unconventional) idea to improve fusion reactors which needs millions to test. But a basic income guarantee might be a good baseline on which to build a trust based grant system.
I like the idea. It reminds me of a system present in some artistic areas: when headhunters go to theaters and mark actors they like, so that when they need someone for a project, they can invite them.
Similarly, in scientific circles there are similar scheme when a project leader wants to hire a specific person they have encountered. And then the job announcement would be tailored as much as possible for this specific person. This hacks public selection procedures,but in reality implements exactly the proposed trust scheme. Someone (on, the grant agency through a grant application) gave the money to the project leader - and now they determine to whom the money should go.
This may sound a lot like corruption. But when I explicitly asked one person doing this, they responded it is not corruption. It would be had they been getting something inappropriate from that action for themselves - but they are not.
So, in a sense, the system does operate... in the shadows.
Apart from the trust issue, there is one more drawback of the proposed system. The grantee is passive in this approach, so it may stimulate "everyone is obliged to me" attitude.
But. Having funding schemes based on different principles mitigates that risk.
This is late and a bit superfluous, but, on the MacArthur grants specifically, Thomas Frank had a nice piece in The Baffler a few years back.
' What James English tells us about the countless foundations and academies that make these awards is that they are not simply neutral observers, impartially recognizing merit from some lofty height. They are always engaged in a cultural project of their own—usually to establish themselves as authorities and their own concerns as correct ones.
In pursuit of that project, all award programs face the same problems. Because the reputation of the prize must itself be established for the academy in question to set about judging the merits of others, all prize programs gravitate toward convention. They tend overwhelmingly to reward people whose reputations are already made. Indeed, as the competition between prizes grows more intense, English tells us, the pressure to associate a prize with safe and unquestionably prestigious figures only grows. This is why competing prizes within a field always tend to converge on the same individuals, virtual prize magnets who are fated to stagger through life under the weight of their accumulated laurels.
...
The real promise of the MacArthur Fellowship program is that it does not require grant writing, or applications, or even achievement of a conventional sort. It could theoretically be used to bypass the world of foundation favorites altogether. It could single out worthy individuals who have been unfairly overlooked, lift them up, launch their careers, and force the world to pay attention. That even seems to have been one of the ideas for the program in the beginning.
Well, today the program rarely does any of those things. Instead, and just like nearly every other prize program in the world, it chooses noncontroversial figures and rewards the much-rewarded, giving in to what James English calls “the desire to have already famous and massively consecrated individuals on their list of winners.”
Sort through that list of winners and you’ll find lots of the usual prize magnets, the foundation favorites, the celebrated New Yorker authors, the “20 Under 40”set, the people you heard profiled on NPR a short while ago, the person who just got the National Book Award, or the John Bates Clark medal, or a Ford Foundation Leadership Grant. The resumes of certain winners are thick with honors: Junot Diaz was a literary champion many times over by the time he won in 2012, while Robert Penn Warren, who received one of the very first MacArthur Fellowships, had won the Pulitzer three times by that point in his life and had received more than a dozen honorary degrees.'
What's your argument against doing grant lotteries? You mention Goodhart's law up front. But part of that idea is that if you replace suboptimal metrics with other metrics, people are just going to figure out other ways of optimizing for those new metrics. Even if they're fuzzier and less specific. So why not go all the way toward the unhackable? Maybe there's a comprise in there—with a threshold for who is a sufficiently trustworthy / competent applicant, or a weighted probability based on some subset of your proposed criteria.
The main problem that I have with your Trust Windfall argument is that it sounds, in essence, like the way being rich works. You get a little bit rich. This gets you closer to other rich people, who begin to trust you more. Then you get more rich, because they're in charge of the assets and these assets get allocated disproportionately to rich-adjacent you via trust. That totally seems like something you could hack! In your example, it's kind of like replacing one Lin-Manuel problem with another. The extent to which lots of people trust him is also going to go up after he won a Tony. So is the criteria to identify the trustworthy people who are most currently undervalued by the market, e.g., your gulag thespian friend?
To put my concern another way: you point out this is overtly based on choosing people who are your friends. I feel like people will either unconsciously choose in-group members / people they are interpersonally fond of; or they will try to address the unconscious bias and choose people who blatantly different from themselves. Both are hard to square with exactly how much they cloud of judgments of "merit". Curious to know what you think about that!
At any rate, I agree with the primary argument of your piece: Oxford degrees are useless.
This is excellent! It's a cool idea and definitely deserving of at least a few turns of the evidence dice.
Thanks for sharing this Adam--I really appreciate the problem statement. One concern with your proposed solutions is that networks tend to be very homophilic. I can easily imagine sincere, well-intended use of social networks to leverage trust working well, but also being very homogenous, and open to challenges that they constrain diversity.
Haha, I'm guessing you were applying to the FTX Future Fund? (We were also doing the same for Manifold Markets.) FTX FF is actually already trying something similar to Trust Windfalls with their regranting program: https://ftxfuturefund.org/announcing-our-regranting-program/
You might also appreciate the way Speculation Grants work, which also have this function of "moving trust into the hands of others" https://survivalandflourishing.fund/speculation-grants
The entire space of new grant funding mechanisms seems to be very hot at the moment! I'm personally bullish on Impact Certificates/Retroactive Public Goods Funding, the "new type of philanthropic institution" that Scott alluded to.
A "Secret Shopper" method to award grant money? I like the tone of that. As you touched on, the greatest hurdle is for the Agent to maintain that secrecy-anonymity. One blabby remark and the Agent would be the proverbial lottery winner with "friends" and "long lost relatives" coming out of the weeds.
Trust Windfalls remind me of self-organized funding allocation: https://www.science.org/content/article/new-system-scientists-never-have-write-grant-application-again
My own idea to supplement the usual alternatives of flat or random (lottery) funding is a bootstrap method: let anyone apply for a small amount, e.g. $20,000, where the bar for approval is basically being able to spell their own name. Then, if they can demonstrate success or progress (including null results), they unlock the chance to apply for double the money in a similarly streamlined process, and the process repeats.
Trust Windfalls are a really interesting idea. I wonder if they would end up captured, though? There's an adage that says "A's hire A's, but B's hire C's", meaning that people who are good-at-what-they-do in a general sense have the skills to recognize and admire other people who are good-at-what-they-do and want to work with them, whereas posers just want to work with people who will suck up to them.
So say you institute a Trust Windfall Foundation and identify your first round of 100 Agents. You do your best to identify people who are trustworthy, but because recognizing ability is hard, only 90 of them are really "A"s and the other 10 are "B" or "C"s. Now in the next round, a bunch of people get funded, and the baton gets passed. The people who get the batons also have the same level of noise as you did, so 81 of them are "A"s, and 19 are "B"s and "C"s. After a few iterations of this simple model, you end up in a situation where almost all of the Agents are not-As, and almost all of the money goes to cronies or people the agents want to suck up to them.
1. Is this model accurate? Probably not: generally people are pretty good at understanding other people, and I feel like people are generally pretty trustworthy as long as they're not incentivized not to be. But how do you incentivize the agents themselves to grant money to people who will actually do interesting things with it, rather than to people who are fun to hang out with or will use the money in ways that benefit the agent? How do you incentivize the agents to not reveal they are agents to their friends and use their granting power to encourage cronyism and suck-uping?
2. How to get around this? I suppose you, the Foundation, could keep records on the success of each grant, and terminate germlines that aren't fruitful after a few generations, while seeding new ones. I suppose you could select next-agents from a list of suggested ones, and inform your choice using information from broader sources.
This whole thing sounds a bit like a sampling problem to me. Given the social network, how do you get your money-sampling Monte Carlo chain to spend the most time around the nodes that maximize "Value"? A big part of that is defining "Value".
Could the idea of a basic income guarantee be viewed as the state trusting everyone? At least enough to ensure they have a walk away option from any unreasonable situation. Of course, it would not be substantial enough to let everyone do their best work e.g. a nuclear scientist with an intriguing (but unconventional) idea to improve fusion reactors which needs millions to test. But a basic income guarantee might be a good baseline on which to build a trust based grant system.
I like the idea. It reminds me of a system present in some artistic areas: when headhunters go to theaters and mark actors they like, so that when they need someone for a project, they can invite them.
Similarly, in scientific circles there are similar scheme when a project leader wants to hire a specific person they have encountered. And then the job announcement would be tailored as much as possible for this specific person. This hacks public selection procedures,but in reality implements exactly the proposed trust scheme. Someone (on, the grant agency through a grant application) gave the money to the project leader - and now they determine to whom the money should go.
This may sound a lot like corruption. But when I explicitly asked one person doing this, they responded it is not corruption. It would be had they been getting something inappropriate from that action for themselves - but they are not.
So, in a sense, the system does operate... in the shadows.
Apart from the trust issue, there is one more drawback of the proposed system. The grantee is passive in this approach, so it may stimulate "everyone is obliged to me" attitude.
But. Having funding schemes based on different principles mitigates that risk.
This is late and a bit superfluous, but, on the MacArthur grants specifically, Thomas Frank had a nice piece in The Baffler a few years back.
' What James English tells us about the countless foundations and academies that make these awards is that they are not simply neutral observers, impartially recognizing merit from some lofty height. They are always engaged in a cultural project of their own—usually to establish themselves as authorities and their own concerns as correct ones.
In pursuit of that project, all award programs face the same problems. Because the reputation of the prize must itself be established for the academy in question to set about judging the merits of others, all prize programs gravitate toward convention. They tend overwhelmingly to reward people whose reputations are already made. Indeed, as the competition between prizes grows more intense, English tells us, the pressure to associate a prize with safe and unquestionably prestigious figures only grows. This is why competing prizes within a field always tend to converge on the same individuals, virtual prize magnets who are fated to stagger through life under the weight of their accumulated laurels.
...
The real promise of the MacArthur Fellowship program is that it does not require grant writing, or applications, or even achievement of a conventional sort. It could theoretically be used to bypass the world of foundation favorites altogether. It could single out worthy individuals who have been unfairly overlooked, lift them up, launch their careers, and force the world to pay attention. That even seems to have been one of the ideas for the program in the beginning.
Well, today the program rarely does any of those things. Instead, and just like nearly every other prize program in the world, it chooses noncontroversial figures and rewards the much-rewarded, giving in to what James English calls “the desire to have already famous and massively consecrated individuals on their list of winners.”
Sort through that list of winners and you’ll find lots of the usual prize magnets, the foundation favorites, the celebrated New Yorker authors, the “20 Under 40”set, the people you heard profiled on NPR a short while ago, the person who just got the National Book Award, or the John Bates Clark medal, or a Ford Foundation Leadership Grant. The resumes of certain winners are thick with honors: Junot Diaz was a literary champion many times over by the time he won in 2012, while Robert Penn Warren, who received one of the very first MacArthur Fellowships, had won the Pulitzer three times by that point in his life and had received more than a dozen honorary degrees.'
https://thebaffler.com/latest/phone-rings-genius-cult