AI Safety Funders LOVE This One Secret Trick To Having An INSANE Impact
You won't BELIEVE reason number 3
If you’re interested in applying for funding to work on an AI safety talent development initiative, click the button below to be redirected to our Request for Proposals (RFP).
Just a couple of weeks ago, I hit my two-year mark at Open Philanthropy. I’ve learned a lot over this period, but I’ve been especially surprised by how great the ROI is on AI safety talent development programs after digging into the numbers.
Consider this graph:
I’ve heard at least four examples in the last ~year or so of this dynamic playing out at AI safety organizations that run fellowship programs.1 This amount of applications isn’t the norm per se, but it happens often enough that we literally have an emoji in our Slack called :fellowship-growth: that we use to celebrate these cases.
So yeah, now that I’ve seen how the sausage is made, I’m officially talent development pilled.
A brief case for talent development
Talent gaps are still very much a thing. Key institutions2 ruthlessly compete to hire from a small pool of qualified + high-context + productive people who can work on hard AI safety challenges. I’m regularly asked for leads on such folks, and I often share fewer names than would be ideal.
But at the same time, a large number of smart people are also trying to break into the world of AI safety, sometimes to little success (at least initially).
Here’s how to reconcile this discrepancy.
A lot of smart, talented, ambitious people are keen to get involved, but can’t immediately contribute because they lack training, domain-specific knowledge, connections, and context. Organizations, on the other hand, want people who can pretty much kick ass right away.
This is a tricky equilibrium, but talent development programs can help. Even a single 3-month fellowship can provide some people with surprisingly huge career boosts (I should know, I largely credit my start in this field to one such program).
We're still operating under massive uncertainty. Back in 2020 and 2021, Luke Muehlhauser, who leads our work on AI governance and policy, wrote a few blog posts about AI governance priorities and challenges. At that time, one of the key bottlenecks he identified was a lack of strategic clarity on which intermediate goals should be pursued.
We have relatively more strategic clarity than in 2021, but let's be honest: AI safety still involves navigating a wildly uncertain future. Given this, investing in talent remains one of the most robust bets we can make — we still don’t have enough strategic clarity for this not to be the case.
There's a multiplier effect. People who benefit from these programs often go on to train and mentor others. It's a positive feedback loop — today's fellowship participant becomes tomorrow's program director.
The cost-effectiveness is often excellent. A well-designed, well-excuted fellowship program can significantly alter someone's career trajectory for a relatively modest cost. It just isn’t that expensive to pay a fellow’s salary for three months. When you compare that to other interventions, the bang for your buck can be quite remarkable.
What to do about this
For donors: If you're looking to give to the AI safety space, talent development programs seem to deliver pretty great returns. If you're eager to fund programs like this (to the tune of $250k or more) and want to chat about specific opportunities, shoot me an email. If you’re looking to give less than that, I’d recommend donating to Horizon.
For entrepreneurs/people looking for their next thing: If you're between projects and have experience in the AI safety world, you should seriously consider starting up a new talent development organization (or starting a program at an existing organization). Some of the most successful programs started as scrappy experiments by one or two people who saw a gap and decided to fill it.
Here are a few examples of initiatives that are already happening in some form or another, but might serve as a good source of inspiration:
A research fellowship program for mid-career technologists (e.g., a PhD and/or 5+ years of experience) looking to transition into AI policy, featuring placements at think tanks and other research institutions.
A seminar program or series of workshops for economists, focused on explosive growth from AI R&D.
A program for experienced cybersecurity professionals interested in learning more about/transitioning into AI security.
If running a program like this appeals to you, apply to our RFP, or speak to people running existing organizations to see if any of them could use help starting something up.
For people trying to break into the world of AI safety: I remember exactly what it was like to be where you are. It was hard and emotionally draining — breaking into such a competitive field requires real grit and determination. But keep applying, and keep pushing forward!
Fellowships are just one example of a “talent development program”. Other models include: mentorship programs, residence programs, workshops, educational courses, and other programs that help individuals interested in working on AI safety develop career capital.
AI labs, the government, nonprofits, etc.
Came for the click bait title and stayed for the good advise.