'We Chose the Right Problem'

In the fall of 2019 — before a pandemic emptied campus, before AI became the buzz-acronym of the century — Ken Fleischmann sat down to explain the purpose of a new grand challenge called Good Systems. Its mission, he wrote, was to "design AI technologies that are driven by human values and that benefit society." The hard part, as he saw it, wasn't working out what artificial intelligence can do but deciding what it should — and should not — do. He borrowed the warning from a certain fictional UT mathematician who soon found himself being chased by dinosaurs in a so-called amusement park: the scientists were so busy asking whether they could that they forgot to ask whether they should. 

Fleischmann could not have known how fast the ground would move. Within months, COVID-19 shuttered the campus he was writing from and pushed daily life onto the very technologies Good Systems had set out to scrutinize. Then, in late 2022, ChatGPT arrived, and the uncanny fluency of large language models made a specialist's concern feel like everyone's, seemingly overnight. Nearly everyone, it turned out, had a stake in what AI should and shouldn't do. 

Now, seven years on, as the eight-year grand challenge nears its close, the founding and current chair of Good Systems — who is also a professor and department chair in the iSchool, a founding department of UT's new School of Computing — sat down again, this time to answer some questions. In the conversation that follows, Fleischmann delineated Good Systems' journey, from its deliberately local roots to a "Good Systems 2.0" the team is currently envisioning. 

The interview has been edited for length and clarity. 

When you look back at the original vision for Good Systems, what feels most different now? 

We started the planning about 10 years ago, and certainly there have been amazing advances and innovations in the AI space, all of which have reinforced my belief that we were right to focus on the societal impacts of AI when we did so. 

I'm not sure anyone exactly predicted the precise way that AI would unfold, but I think we already were far enough into seeing the impacts of social media in society to anticipate that if AI was going to be anywhere near the magnitude of social media — and I think the argument could be made that it's already having even more impacts than social media, and the trajectory is only going up — it would seem that it's important to be proactive and thinking about what those implications might be and how to steer AI innovation in responsible ways that will benefit society. 

I do think that many of us were already aware of some of the downsides of social media at that time. Things like cyberbullying were being brought to the fore. We were also starting to have a greater understanding of the role of social media in elections worldwide and in our political discourse worldwide. 

And I shouldn't be talking about social media and AI as if they're two different things, because they're incredibly intertwined. Of course, many of the accounts that we're interacting with in social media today are bots run by AI. And social media also provides material for training generative AI’s large language models, just as the rest of the web does. 

You wrote a lot in 2019 about human-centered design. What does that mean to you now? 

I've always believed that if you're going to design technology for people, first you go talk to the people; understand what they need, but also understand what they value. If we overlook people's values — what they consider important in life — we're missing a big part of the puzzle. If something accomplishes someone's needs but does it in a way that's opposed to their values, they likely won't adopt the technology. 

We also need to think about what technologies are going to enhance the reputation of an organization, because trust in a product — in a brand, in a corporation, in a federal agency, in a nonprofit — is built up slowly over time, with great expense and time and effort, and it can be lost in an instant. 

So it's really important that we're considering what could go wrong, what the challenges and implications might be, and addressing that in advance instead of finding out the hard way. 

"I've always believed that if you're going to design technology for people, first you go talk to the people; understand what they need, but also understand what they value."

Have there been major shifts in how you've thought about what could go wrong? 

I think we've always anticipated that technological advances, including but not limited to AI, have implications for the workforce. 

What I didn't anticipate at the time was the degree to which knowledge work would be challenged by advances in generative AI. The advent of large language models and the rapid innovation in generative AI has now put knowledge work much more in the crosshairs of augmentation or replacement. 

We were thinking about blue-collar workers and the importance of preserving blue-collar jobs, which I still believe is very important. But at the time, I thought that was probably the most at-risk type of job, and right now, I feel pretty good for them, that they've got pretty good job security relative to us knowledge workers. 

Can you give an example of a Good Systems project that evolved in response to those concerns? 

I'll give one example from the smart hand-tools project. Our original idea was building the AI into the tool itself. Part of the idea is putting the power of AI into the hands of skilled trade workers. But in doing our user research, we quickly realized the importance of things like data privacy, data agency and data ownership — making sure that workers actually own and control their own data. 

In logistics, for example, a lot of workers are given smartphones and told to use them all the time, and it tracks how long their bathroom breaks are and things like that. So it can be a very intrusive kind of monitoring that happens in the workplace. 

We actually wanted to design a technology that's owned and controlled by workers. Because tools are typically provided by the employer and not by the worker, it actually makes more sense for the final product to be an attachment to a tool rather than part of the tool itself. 

Another thing we didn't necessarily anticipate when we started the project is, of course, that a technical background is not even necessary for coding anymore. Any of us can use AI coding tools to build technologies. So now we envision ways that we can build interfaces where workers could design new features of the software without needing to be able to code them themselves, and of course many skilled trade workers are also coders.

"While so many issues have become incredibly polarized in our society, I think for the most part, AI has remained relatively bipartisan, and there's been a pretty good-faith effort by a wide range of stakeholders to think about AI in thoughtful, responsible ways."

What accomplishments of Good Systems are you most proud of? 

I'd say I'm really proud of the network that we built and our collaborations across sectors. I'm still proud of our partnership with the City of Austin, and the majority of our core research projects are still collaborating with the City of Austin. We intentionally chose to start locally. 

Now we've been involved in AI discussions at the local, state, federal and even global levels. Several Good Systems researchers were part of the City's AI Advisory Council. Several members of Good Systems testified at hearings for the Texas Responsible Artificial Intelligence Governance Act, which was signed into law by the governor in 2025. 

We've also had a delegation of representatives from the European Union Parliament who came to learn about our approach to ethical and responsible AI research. At the end, they described the goals and the work of Good Systems as “very European,” which we thought was the highest compliment they could give us.

What have you made of the policy response to AI over the past decade? 

One thing I am proud of, and I think is really critical, is I do believe that we've been able to build a lot of bipartisan consensus around AI innovations and the need to regulate AI in a responsible manner. I've been part of panels where we've had elected officials from multiple parties — Republicans and Democrats — and we've been able to reach a lot of common ground in terms of how we view these technologies and the impact they have on our society. 

So while so many issues have become incredibly polarized in our society, I think for the most part, AI has remained relatively bipartisan, and there's been a pretty good-faith effort by a wide range of stakeholders to think about AI in thoughtful, responsible ways.

"Even though the stakes are far greater, I do remain cautiously optimistic about AI generally being a force for good in society."

Do you feel more optimistic or less optimistic now about the future of AI ethics than when Good Systems began? 

I'm more optimistic about how important it is. I mean, I believed it at the time, but now I don't even have to believe. I can just look around and see it. 

When we began planning Good Systems in 2016 and launched in 2019, we were all interacting with AI, but we didn't think about it. We were getting recommendations of what our next song or our next movie was going to be. We were getting navigation guidance. Just about everything that we've done with our smartphones for the last 10-plus years has been leveraging AI in some way. 

But I think that certainly advances in generative AI — the public, splashy launch of ChatGPT in particular — have greatly enhanced public awareness of AI. 

I think people's idea of AI is often limited to generative AI. So I think it's important that we're still thinking about these more insidious ways that AI has insinuated itself into our lives. 

When gig workers are completing delivery routes, as my colleague Min Kyung Lee in the iSchool says, they're being “algorithmically managed” to get from point A to point B. In past decades, we would have had a human manager who was giving instructions about how to actually navigate multiple deliveries in a circuit or something like that. But now it's all happening in real time, with the algorithm managing employees, often in ways that are entirely opaque to the workers themselves. 

Certainly I've seen a lot of companies create AI ethics boards. I've seen a lot of new job descriptions. Job titles like “AI ethics specialist” and “AI governance analyst” didn’t exist in 2016. At the same time, I've seen a lot of those same companies disband ethics boards. So what can be built can also be torn apart. 

Again, I'm most optimistic about — and most confident about — the degree to which we chose the right problem, and that indeed, this is an important societal challenge. But I would say, even though the stakes are far greater, I do remain cautiously optimistic about AI generally being a force for good in society. 

What comes next for Good Systems? 

I think the key thing is the work will continue. We're really proud of what we've accomplished through what we're now calling Good Systems 1.0. 

Whatever form Good Systems 2.0 takes, I think there are a lot of exciting advances ahead of us, so I'm excited for our continuing mission.