How Will AI Reshape Work?

Deanna Lee

Hey, it's Deanna. A quick note before we start. We recorded today's episode on AI in the workforce in January. One of our guests, Rachel Slama, has since moved on from RAND, leaving in April. But the insights you'll hear her discuss from her RAND research are as timely and relevant as ever. Okay, here's the show.

You're listening to Policy Minded, a podcast from RAND. Today, our minds are on AI and the future of work. No matter how helpful it is, technology has always generated worker anxiety. For instance, tailors once rioted against quick-stitching sewing machines. And centuries later, the invention of mechanical switching likely bred a quieter desperation among telephone switchboard operators. That is, before their jobs ceased to exist. Today, artificial intelligence is also prompting worker apprehension. Those employed in a wide variety of fields, from computer coding and accounting to media and graphic design to law and medicine, are confronted daily with news that their jobs could be completely transformed by AI or even deleted altogether. But what do we really know about how AI might affect the American workforce? RAND researchers have been adding to the growing evidence base on this subject. In one recent study, they looked deeply into how technology affected occupations between 1976 and 2000, drawing lessons for how AI might reshape the future of work. Another publication harnessed the expertise of dozens of our researchers to provide important insights for policymakers as they consider how to best prepare the U.S. workforce, both civilian and military, for the rise of AI. I'm joined today by two experts who led these research efforts, Toby Sytsma and Rachel Slama. Toby is an economist here at RAND and Rachel is a senior policy researcher. Rachel also directs RAND's Labor and Workforce Development Program. Toby, Rachel, welcome.

Tobias Sytsma

Thanks for having us.

Rachel Slama

Yeah, it's great to be here.

Deanna Lee

Thank you. Okay, I want to start this conversation by giving our listeners some insights into maybe what we don't know yet about AI, a technology that is still in its relatively early stages. So what do we need to understand about AI before we understand how it could affect the workforce?

Tobias Sytsma

That's a good question to start with. I think you could probably fill a four-hour podcast with what we don't know about AI. Something we need to understand about the technology itself is that there is a lot of uncertainty and it's moving very quickly. That's kind of a twin problem. I mean, if it was moving very quickly and we knew exactly where it was going, that's one thing. And that has one implication for the workforce. If there's a lot uncertainty and it is moving really slowly, you know, that can also be managed. But the challenge with AI, I think, is that it's kind of both of these problems at the same time. It's moving fast, and there's a lot of uncertainty, which naturally creates a lot of uncertainty in labor markets as well.

Rachel Slama

I think I'll add to that, Toby, that I think we don't know exactly how it's been implemented and being taken up in workforce. And we do have some initial research from some of my colleagues at RAND that show that it's pretty specific to the occupation, how AI is being integrated, and even specific to the task. Like even, you know, take healthcare, for example, which we have a lot of colleagues to have studied this and we've talked to some providers on the ground who say that AI has gone from really revolutionizing some tasks that their providers do. So AI is really good, generative AI is good at processing text data. So for example, the notes that medical providers write that can be recorded from patient interactions and automatically produce a note and that has saved providers, we've heard, even like a whole day a week on the ground. So that's just a simple example of how that might radically change the task of writing a note. What that will mean for other occupations in an entire field I think is yet to be studied and a lot of work is needed on the ground to learn about how it's being implemented in different tasks and fields.

Deanna Lee

A lot to learn, to be sure. So now let's talk about some of the findings from your recent work. Toby, your study that you led recently looked at some past technological revolutions or technological shifts and how those affected different occupations. How is the rise of AI the same or different than those past revolutions, thinking of things like mass production and assembly lines or even the internet.

Tobias Sytsma

I think, in some ways, it's similar. But maybe in some critical ways, it's different. I think it's similar in the sense that, like most technologies, AI impacts the workforce at the task level. So when we think about the impact of technologies, it's easy to automatically go to, oh, this technology does my job, or this technology could do my job. Well, your job is made up of a bunch of different tasks that you do with different frequencies and require different levels of attention and different skills. And technologies kind of enter into that equation by pinging on these different tasks. And so with the internet, it disrupts certain tasks, but it makes other tasks more important for workers. Or the social media might automate some things or take away some things that people were doing before, but make other aspects of their job more important. And AI is kind of... potentially similar in that sense. So Rachel was talking about the use of generative AI in medical occupations. And there's some tasks that having a model that basically translates what this document says into billing codes, for example, that's really important. That could be really useful for a specific task. And it might free up the worker to do a bunch of other things that they weren't doing before. It's kind of the way that historically, technological revolutions have occurred and impacted the labor market is that they take over tasks, the workforce shifts, demands change, and workers pivot to new areas. One way that this is potentially different is that one of the end states of artificial intelligence is artificial general intelligence. And you hear companies like OpenAI and Anthropic and Google all talking about trying to develop this artificial general intelligence. We don't know exactly what that means. I don't think we have a clear definition of what artificial general intelligence really is, but the sort of working definition that people operate with is that it's something that can do every task as well as people can. So if you kind of take the previous way that technology has impacted the workforce, which is impacting specific tasks and allowing workers to pivot to new areas where they have some comparative advantage over technology, like they can do something that technology can't, for example. If you extend AI to where people who are developing the technology, at least think that it's going, you have this technology that can do a bunch of different tasks. So where do people pivot? Where is the human comparative advantage in a world where AI can do everything as well as people? I think that's kind of the one potential way that AI is different. And we haven't seen that happen and it's not clear whether that will happen, but that is a potential end state that wasn't really prevalent when you're thinking about the internet or even thinking about something like electricity, where that does a lot of things, but electricity can't do everything that people do, for example. So I think that there's that element that's a little bit different. On top of the fact that AI is intelligence. I mean, it's imitated intelligence, it's artificial intelligence, whatever you wanna think about it. Intelligence is an incredibly powerful economic force. It's hard to kind of overstate how important intelligence actually is for creating economic growth and economic opportunities for a worker and for a country in general.

Deanna Lee

That's a good segue to the next question. I mentioned this at the top of the show, but I think many of our listeners may be thinking about their own jobs, their own careers and how AI might affect them. So let's talk about what you found in your research, Toby, about which specific roles or occupations or even maybe it makes more sense to talk about it at the task level like you just mentioned. Which jobs or roles are most exposed to AI, and then on the other hand, which ones are more insulated or more protected from risk.

Tobias Sytsma

Yeah, so in some work that came out recently, we, a colleague, Éder Sousa and I, looked at the level of exposure to different AI technologies that different occupations have. So, key term here is exposure. It's a term that economists use a lot when talking about technology, but basically it just means is there some overlap between what the technology can do and what people can do. And the way that we approached this was we looked at patent data, going back from 1976 to 2020. And we pulled out different AI patents. And we looked at the description of what that technology does in the patent. And then we actually used some AI tools to map what it says that the technology can do to work tasks that are described in different data sets. That workers do. And so that level of similarity in those two descriptions of what the technology can do and what workers can do is our measure of exposure to the technology. And so we do this at a task level. So we look at, you know, for an economist, you know, my tasks are like, you write papers based on, or write reports based on economic data and economic theory. A lot of economists teach, so there's an element of teaching students about economic stuff. And so we look at those descriptions of those tasks, and we match them up with AI technologies that say, oh, this is an AI tool that can do economic research and read the latest working papers. And so, we say, oh, there's a lot of tech similarity here. This technology seems like it might be able to do this task. And from there, we come up with a waiting scheme that allows us to aggregate all of those tasks within an occupation, all that different occupation exposure to AI technologies into a single occupation score. And that sort of results in this overall, how exposed is this occupation to these different AI technologies based on how exposed each of their individual tasks are. And we found a bunch of stuff. So we have this relatively complicated way of figuring out how exposed the technology is. Or how exposed an occupation is to these different AI technologies. And at the end of the day, we found that occupations that require more education and pay higher wages are the most exposed to AI. That's different than we've found from other technologies. So if you look at all other patents, the pattern is a little bit flipped. It's occupations require less education and pay a little lower wages, the most exposed overall. But there's the shift that happened around 2000, where now it's kind of more... it's occupations that require more education and pay more than are most exposed. For the actual measure of what this means, it's hard to directly quantify and hard to say for sure that, oh, exposure to this technology resulted in this change in the labor market for this occupation. So instead, we look at associations and we say that this increase in exposure is associated with this change in employment. We can't say whether it's causal. But what we find is that it's actually interesting, and it's a pattern that we've seen historically as well, if you look back in the literature at impacts of technology on the labor market, which is that if you at the average occupation and you increase exposure to AI technologies by one unit, we measure this in sort of standard deviations, but you can think of that as like one unit increase in exposure, what does that mean for employment? We find basically no impact. But what we do find is that when you look at the differences across occupations, you find meaningful differences. So if you look at an increase in AI exposure for an occupation that just has a lot of really routine tasks as being core to that occupation, that increase in exposure actually is associated with a decrease in employment. If you look it occupations that have very few routine tasks, so a lot the stuff they do is very abstract. Cognitive work, an increase in exposure for those occupations actually was associated with employment growth. So, you know, overall you see this zero effect, but really that's because we see this divergence across these different types of occupations. So the impact on the labor market, at least up until 2020, which simultaneously feels like two decades ago and yesterday, for like a million reasons. And in terms of AI, 2020 might as well have been, you know, two decades ago, to be honest, just how quickly this technology is moving. At least up until that point, that's what we were seeing, is that more exposure to AI technologies is generally associated with either an increase in employment, if your job is very cognitive and abstract, or a decrease in employment if your, if your job is more routine. And those patterns have generally held in the literature as well. So we aren't the first to document this and we aren't t the only study to document this either.

Deanna Lee

And can you give some specific examples? And Rachel, maybe you can jump in too. I know we already talked about healthcare already, but can you get some specific examples about what these fields are or roles are and what the changes look like?

Tobias Sytsma

So we looked at this in a few ways. If you just look at overall occupation changes, like which are the occupations that are the most exposed to different AI technologies right now, you get things like captioners for natural language processing technologies, or audiologists for machine learning technologies, or search marketing strategists for knowledge processing type AI technologies. So we have a table where we kind of show, like, here's the technologies, and here are the top most exposed occupations as of 2020. It can be a little bit hard to pick out patterns when you're just looking at occupation titles. So we also looked at some of the skills that are required in the most exposed and least exposed occupations. And what we found was there's the most growth and exposure among occupations that have programming skills involved. So negotiation, persuasion, and writing skills. And there's the least exposure to AI patents for occupations that are primarily equipment maintenance, repairing, equipment selection, things like that. So there's kind of this, you can kind of see some of the patterns that emerge that I talked about earlier, where you have this diverging impact on the labor market. You can see it in the types of skills that are more exposed as well. Occupations that have sort of an emphasis on more cognitive skills, for lack of a better term, tend to be the most exposed.

Rachel Slama

Yeah, I think a great example of this is in the entertainment industry. And Toby and I had a chance to be part of a panel where we invited Meredith Steen, president of the Writers Guild of America West, and then her senior director, Laura Blum-Smith, to talk about how AI was impacting writers in the creative industry. Then that led to a really high profile strike. For I think almost 150 days back in 2023. And she talked about some of the issues at heart in that strike related to creative property and intellectual property, the use of, for example, scripts for training new AI models that would generate new content without crediting the writers. And so I think we're really gonna see AI as a disruptive force that we already have in many industries and including those that are creative that we associate usually with human qualities.

Deanna Lee

Let's move on to maybe the evidence, if there is any compelling evidence to suggest ways that AI is going to help workers in the workforce. Obviously, jobs will be created related to AI, but maybe we can get a little bit more into some of the nuance of how the pendulum could swing the other way for workers in terms of increasing opportunities or creating new roles or new skills things like that.

Rachel Slama

Yeah, I can talk a little bit about the sort of opportunities that I think AI presents. I do think for the foreseeable future, and others who maybe work more on the national security front may have a different perspective, but on the education and labor side, I do you think there'll be a role for what they call humans in the loop and humans kind of as a co-pilot to AI for the forseeable future. So that really begs the question of how can we upskill and train entire workforces and really think about some altering of existing education and training systems. And I think it's a much needed upgrade. So that might be that AI is the catalyst to give us a reason to make some important changes to our, for example, post-secondary education system that I think a lot in the policy sphere are starting to sort of pose the question of the value of a four-year degree. And so this is an opportunity for post-secondary systems to really think about are they providing the training opportunities that are going to prepare workers for this. We talk about workforce of the future, but I think the future is here. So I think it's a good catalyst for that at the K-12 level, you know, there have been a long history of standards-based reform and standardized tests, and there's a criticism that schools have, under pressure and being held accountable to these tests, kind of resorted to teaching to the tests. So I think it really puts that all in question about do we have the right standards that our students are being taught, or do we have the right curriculum in place? And then on the military side, which is relevant to a lot of our sponsors and partners in the field, there's a whole professional military education and training infrastructure that I think a lot of leaders will be starting to look at carefully and making sure that it's training the right skills to have a force that's ready for new AI, in-gen AI powered tools. Some of our colleagues at RAND have done, wrote very thoughtful commentaries on all of these areas, which I would encourage listeners to check out specifically about how K-12 education system should adapt and how post-secondary training systems must adapt and also military education. So I think this work is well overdue and I'm hopeful that perhaps AI can be a catalyst to put in place needed reforms.

Tobias Sytsma

Yeah, and I would just echo that. I mean, I'm hopeful that AI can be a catalyst as well. I think right now, I think that most occupations could benefit from AI as it is today. One of the things that I think gets missed sometimes, at least when economists talk about this stuff, and I'm guilty of this as well as an economist, is that we don't always do a great job of communicating that things like exposure to AI can be a good thing as well. Just because you're exposed to the types of things that a large language model is good at doesn't mean that it's gonna be bad for your labor market outcomes or for your employment opportunities. It could be really good for it because it can improve your productivity at these tasks and it could help you do more fulfilling work. And we've seen some research come out and I can provide links to these papers if that would be helpful. We've seen studies come out recently that suggest that there is a positive productivity impact of having access to generative AI tools. So whether it's software engineers or coders or even scientists doing research and development, the evidence so far seems to suggest that having access these tools does improve productivity. Whether or not the types of things that AI tools in general do sort of makes people feel less fulfilled in their careers, I think is an open question. There's some evidence that suggests that like, when you give people, particularly scientists that do research and development, access to AI tools, they are more productive, they file more patents, but they do report a little bit less satisfaction about their job. So there's something like that people value about sort of the creative process and ideating and things like that, the AI tool does it maybe, maybe it helps you do it better, but you do feel like maybe you lose something in that. And so we haven't seen a ton of research on this because it's obviously a very new field and a new technology, but I think more work is needed there. But in general, I'd say that most people could probably benefit from using sort of the chat bot style AI tools that exist right now just for their work in general.

Deanna Lee

Absolutely. And I'm sure a lot of our listeners probably already do make use of that for work or even in their personal lives. We've talked a little bit about education and how reforms to the education system will likely be coming with the rise of AI and how AI will facilitate those changes. What are some other keys to smoothing the transition for workers and employers alike? Are there things that individuals can do? Is this a purely a systems-based issue? What are some of the areas beyond education where shifts need to happen to create the best outcomes?

Tobias Sytsma

I'll jump in first, maybe. I think that the training reform, broadly defined, is probably useful, particularly if AI progress doesn't necessarily slow, but it maintains on its current path of this technology is things that is something that requires people to be in the loop, as Rachel mentioned. You have to monitor the output. You know, it can make mistakes, so you got to be careful. It's useful for doing certain tasks, but not other things. I think learning how to work with these tools is something that can be taught. It also requires some experimentation. So I think that encouraging, you know, whether it's employers encouraging, or just workers doing it on their own, encouraging experimentation with these tools is really useful. I think that things could change if we start to see AI agents that are really kind of autonomous AI workers, quote unquote, that can sort of start operating and doing things over a longer time period. So a chat with a chat bot is like you just type in the question, and it gives you a response. And like, OK, that's good. But it doesn't sit there and think about it for days at a time and then come up with some novel solution to a problem. It's sort of like this iterative process with the worker. But there's some evidence that suggests that these AI agents are likely to be deployed at some point in the near future. Whether that means that entire occupations go away is very unclear. Like I said, they could just make existing workers a lot more productive because of the things they do allow workers to transition to higher value activities. But I think that it has different implications for training. Different implications for workplace support in general. So, you know, and maybe a more extreme scenario that is at least worth entertaining as we're at the beginning of this, you know, rapid change is something along the lines of transition support. So you know this the thing I think about the most when it comes to AI displacement is actually international trade and what we saw in the early 2000s to mid-2000s where China entered the WTO, there was this large increase in imports of Chinese goods into the U.S., and it displaced a lot of manufacturing workers. This evidence came out 2016 onward, and we saw, oh, this actually had a pretty meaningful impact on manufacturing employment. One of the interesting things about the studies that have looked at this is that the workers that were displaced didn't use training programs to the extent that they were available. They went on disability and they went on unemployment insurance and things like that. But there were programs in place that were there to help workers with this transition. So the Trade Adjustment Assistance program is one of them. It has since, you know, it's been sort of discarded as a policy in general, it wasn't funded going forward in 2022 onwards, so it's kind of defunct. And there's, you now, I think there's sort of a, people aren't as super fond of that program because I think people feel like it failed a little bit in sort of promoting the type of transitions that were hoped for. But I think that the pillars of the program make a lot of sense for AI. So it's, you know, some income support if you lose your job. It's access to retraining programs so you can move into a new part of the economy that's growing. There was some relocation assistance in there, you know, access to healthcare. I think that these kind of like package policies make some sense when we're thinking about AI impacts on the workforce as well, as well as sort of just like the broader like retraining and ... and curriculum reform that Rachel was talking about earlier.

Rachel Slama

Yeah, a couple of notes. I think those are all really excellent points, Toby. I think in addition to upskilling the workforce and retraining, I think we also need to think about incentives, employer incentives, and retaining workers with AI, and particularly in the federal workforce, the focus of the series that we put out that we can share a link with our listeners. One of the aspects was thinking about the federal workforce and specifically how to retain AI talent within the federal work force. And I think that will emerge as a continuing problem because we know that there's a significant pay premium to AI work and that a lot of those workers will be drawn into industry where the premium might be a lot higher. And so I think the government jobs, which, you know, folks join for a range of reasons, I think we'll have a sort of dearth of AI talent unless there's some kind of intervention. So one of our, two of our colleagues in this series, Michael Mattock and Avery Calkins write a really thoughtful piece about how to retain workers and what are some different options and I think this might apply to other sectors as well. And so they talk about the trade-offs involved about either permanently raising compensation or thinking about bonuses or retention bonuses that are tied to service commitments, for example, and improving training opportunities in the government or really advertising the service, the reasons, other reasons besides pay that people want to serve. And so I think there should be a continued focus on making sure to retain AI talent in our federal workforce. So that's one thing that I think policymakers should be looking at. I think another fact in our AI workforce is just that most of that high-skilled labor is from a foreign-born employment, and so the largest share of graduates with AI-relevant degrees is foreign born. I think more than half. And our colleagues, Angela Clayton and Srikant Kumar Sahoo write a thoughtful piece that I encourage our readers to look at, thinking about our domestic AI talent pool and is it sufficient to meet the demand. So I think thinking about who comprises our AI talent, where are they likely to be drawn to? How can we provide opportunities for other segments of the population through some of the reforms that we talked about at the systems level? I think these are all really important considerations. And then just, the public sector can't compete with businesses in some areas, not only with AI talent, but just in terms of paying for the machinery that's needed to power AI and the energy requirements of AI, I think will have major implications as well. So I think policymakers need to think about, are there incentives to make sure AI talent is distributed in some of these other areas of the economy. So I encourage readers to follow up with those resources and do a deeper dive. I think thinking about how to engage existing workforces is another piece that employers should focus on. We know a lot about tech adoption in education and it's pretty low in education. There are tons of ed tech tools out there and adoption is much lower because I think in large part because teachers and those who would use the tools in the classrooms haven't been a part of the adoption process typically and haven't being properly trained on using the tools, haven't shown the value of using some of these tools in their classroom, haven't seen the value firsthand, haven't been provided guidance, et cetera. And so I think that lesson from low tech adoption in the K-12 system over many years can be a good lesson for other workforces, so how can employers engage their workers in learning and giving beta testing and giving input on the use of new tools, I think is a really good message for employers.

Deanna Lee

So we've talked a lot about the different layers of this issue, you know, AI is going to directly affect some workers and roles and positions. AI itself is going to transform education and education needs to transform to account for AI. There's a lot going on. It's worth noting we will not even, we're just scratching the surface today. I'm wondering, this may be a difficult question to answer, but how can we keep up? All these changes are happening so fast at these different levels. What needs to happen for us to keep up with this shift and even be able to make decisions at the rate at which the change is happening?

Tobias Sytsma

I wish I had an answer for that. That's a great question. And I mean, as someone who is probably too steeped in AI stuff, most of my work is very AI-focused right now, and it has been for the past, I don't know, four or five years. I don't know. I think that the pace of this technology, the pace that it seems to be moving is just orders of magnitude faster than workforces can adapt, and that policy can adapt to, as well. And that's a fundamental challenge, I think, is the pace at which things are moving. And for workers in particular, I think that that creates a lot of uncertainty. And that uncertainty creates, it can create inaction. And maybe there's this sense that we don't really know where this technology is going. And I mean, if you listen to the AI labs, this is going to take over every job. This is going do my job and everybody else's job. What am I going to do? That's totally understandable. And there are periods of time where I feel like that as well, to be totally honest. I feel, who knows? This could really go anywhere. But I think that there are a couple of things. So what can we do to keep up? Well, there's some option value in waiting, I think. In areas where you can make that decision, in areas where you're able to wait, maybe it makes sense to wait in some sense, just to see where does this end up going. If you're really concerned about making a big, big investment in your future work career trajectory right now, if you can, maybe it makes sense to wait a little bit. Obviously that's not feasible in every situation. So what do you do in the meantime? How do you keep up? I would say that some of the work that my colleagues and I have done suggests that the types of jobs and the types of activities that people do in work that seem to be the least exposed to AI and seem to pay relatively well are those that combine some technical expertise with what is commonly referred to as soft skills. I don't really love that term because it makes it seems like they're easier skills to learn or use than other skills, and they're not, right? Soft skills are like human skills, personal skills. These seem to be very valuable in the labor force. It's something that an AI in theory could do, but I think that there's something that gets lost when it's not a person doing it. You can think of an AI having all of this power and all of this ability and all this intelligence, but it's not a person. So there's something tangible about people skills that do seem to matter in the labor market. And so one way to keep up potentially is to think about investing in those types of skills because it seems like those are the types of jobs that are A, somewhat insulated right now. They pay pretty well if you have sort of technical expertise and the soft skills to go with it. And it seems an area where there's potentially going to remain some demand for human stuff. So that would be potentially one way of keeping up with these changes, apart from just using the tools and trying to understand how they fit into your workflow.

Rachel Slama

I think it's a great question. I think keeping up will probably mean different things depending on one's role in the labor market. And so, you know, nearest to RAND in the research community, I think we have a responsibility to the extent that we can to share and document kind of early adoption and learn from workers directly how it's impacting them in the field. We have colleagues that are running a survey through the American Life Panel, for example, that is a nationally representative sample, very large sample of adults, and we're asking them very specific questions about how their work is being changed by AI. We're in education labor doing some early explorations with school districts and ed tech providers to understand how AI tools are being used in the classroom. So the extent at which the research community can document early adoption and impact I think can be helpful. There's a whole research infrastructure in the funder community that I think also has a responsibility for sharing some of these stories and documenting and funding studies that are early exploratory studies. For example, the National Science Foundation has a whole set of AI institutes that have a convening function and are usually many partners involved. There's one community at the post-secondary level, there's several institutes at the K-12 level, also the Institute of Education Sciences has just funded five generative AI centers. So I think all of these centers have and do share what they're learning from the field. And I think it's always very important to engage not just the research community, but also practitioners and lay people. And so offering these learnings in various formats, including research briefs, but not limited to them. So podcasts for one, and infographics, and webinars, and other hands-on learning experiences. I think at the employer level, I think there's definitely a need to have some opportunities to share across roles. So for example, joining communities of practice that talk about what tools and infrastructure they are building at each employer level and learn from others. I know there was an effort to do this at the federal service level through some of our centers of excellence. There was a community of practice around AI tools in government and an effort document use of those tools. And then I think at the worker level, which Toby elaborated on in his response, I think we have to, you know, be wary of some of the educational opportunities, especially the for-profit opportunities that are being offered. Because I've seen an explosion of boot camps and certificates and credentials that promise to lead to the next big job and a pay raise of X number of dollars and so I think in my role, there are researchers at RAND that are working in our portfolio that think a lot about what's called credentials of value. That's really a movement across states to make sure that these not what's called non-degree credentials, such as certificates, industry certificates, boot camps, et cetera, these other types of learning experiences that fall outside of the four-year and two-year system. That they are going to in fact lead to some new skills and that those new skills will in fact lead to an employment opportunity. And so I think workers should be aware that not all learning experiences around AI are the same. And then I think that similarly policy makers have a responsibility to make sure that, for example, programs that are getting supports and subsidies from the government that they are offering this value, that they do lead to the jobs that they promise they'll lead to. So to your question, I think like keeping up means something different depending on where one sits in the economy, but I think there is a lot we can do.

Deanna Lee

One final question for both of you. We talked about what we can do to keep up, but now as two people who study this issue, what is it that keeps you up at night?

Rachel Slama

Besides my children? What keeps me up at night? So I mean, you know, where I sit is more in the education and training space. So the things that keep me up are, you know, related to the tool that's kind of not yet ready for prime time and will give us inaccurate information and the very sort of small example, like ChatGPT and Generative AI and all these tools are known to you know, generate responses that are just not factual. And it's called hallucination. It's a pretty widely documented problem. I think it's gotten a little bit better, but it does require some nuance in as the user of these tools to constrain the tools to make sure we're getting good information in return. And so I do worry that, you know that's just at the micro level. You know, I asked it to give me the other day, I asked to give some RAND reports that were related to a few different policy topics I was working on a policy memo for. And it gave me a list of reports and they all sounded really real and they were all completely made up. So, and even when I asked it to constrain itself to rand.org. And so I do worry that. You know, and I've been sort of trained to look for those types of things and give better prompts that are going to give me better in returns and better responses from the tools in return. But I do worry at scale when you put these tools in the hands of everybody without training that it's going to, you know, it has led to a lot of bad information out there as sort of dependence on these tools without critical thinking skills. And I do worry at like a generational level if we're not careful, we're really gonna suffer in terms of some of these softer skills that Toby mentioned like critical thinking, creativity, creative thinking, out of the box thinking, and that will just default some of these tools if we are not careful.

Tobias Sytsma

Yeah, I definitely have those generational worries as well. As someone with young children, I think about, I don't know, what kind of advice would I give them? He's only two-and-a-half, so other than play with this train or play the cement mixer makes this sound, not that advice. If I were to give them some advice or something, what would I tell them to focus on? I don't actually know for sure. I think Rachel's point's about like learning how to use these tools is exactly right. There's a lot of ways that the current versions of these chat bots can lead you down the wrong path if you're trying to use them for work. There's lots of ways you can use them to boost your productivity as well. So I think it's, I've mentioned this a few times now, but it's like the uncertainty and the speed that are, it just keeps me up thinking about like the we don't actually know what the end state of this technology is, and it seems to be moving very quickly, but maybe it's going to slow down, and every month or so there's like, oh, it's speeding up, oh it's slowing down, oh speeding up. And so there is a lot of back and forth, and I think the end points could be everything from like, this really helps everyone, to this really hurts a lot people, to, this doesn't really do anything, to there's a bunch of different end points where this could lead. There's some very, very deep uncertainty here. And that can make, I think, developing policy that's not just efficient, but also strategic and effective. I think that makes it challenging. And so that's the thing that keeps me up, is just that high degree of uncertainty.

Deanna Lee

Absolutely, I'm sure a lot of people share those feelings. I know I do, but I'm happy that we could end the episode with some practical advice for people. We've maybe raised more questions than we have answered, but I think we've also probably provided some assurances for people who may be worried about their jobs or their careers. Thank you both so much for being here. We will provide links to all of the research that we discussed today in the show notes at rand.org/podcast. And I'm sure this is a topic along with many other AI related topics that we'll be featuring on future episodes of the show. So Rachel and Toby, thank you so much for joining us today. It was great to talk to you.

Tobias Sytsma

Thank you.

Rachel Slama

Yeah, thanks for having me.

Deanna Lee

And of course, thank to our listeners. This episode was produced by me, Deanna Lee. I recorded it along with Toby Sytsma and Rachel Slama. The episode was edited by Evan Banks and Harper Rupert. And RAND's director of digital outreach is Pete Wilmoth. RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis.