Could AI write our laws next?

The U.S. Congress is not usually what we think of as the leading edge of innovation. In fact, the legislative body’s collective lack of technological proficiency is often pointed to as a liability when it comes to regulating new tech.

But lawmakers in Washington and around the country have increasingly been turning to software to help them write and analyze legislation.

Mohar Chatterjee is a computational journalist at Politico who recently covered the issue, and she says there’s pretty much just one program lawmakers all use: LegisPro.

Marketplace’s Meghan McCarty Carino spoke to Chatterjee about what this process looks like in practice.

The following is an edited transcript of their conversation:

Mohar Chatterjee: LegisPro is one kind of software that helps lawmakers amend draft track bills as they make their way through legislatures. Now, the way you’d kind of think about it is it kind of looks like a word processor. A lot of different legislatures use it. So California, the U.S. Congress, soon Oregon, there’s a ton of people on that list. But all of these legislatures work a little bit differently, their needs are a little different. So the software is tailored a little bit differently for each of these legislatures.

Meghan McCarty Carino: What are some of the ways that different legislative bodies are using this? Can you give me some specific examples?

Chatterjee: Yeah, what happens is you have amendments to bills, or you will have new versions of bills that are introduced. And it’s difficult to tell, you know, what exactly is the difference between a new version of a bill and an old version of a bill. And generally what happens is like, you’ll have staffers like, actually print those things out and do a diff check, essentially, what’s the difference between the two, and they’ll do it by hand. And it takes a lot of time, it takes a lot of understanding. And this is essentially a way to automate that process. It’s like legal diff check, but it has a lot of sort of wide reaching consequences.

Like one, it helps you catch those sort of unintentional errors. They’re called Scrivener’s errors, which, you know, is Congress meant to say something, but the actual language says something that’s maybe a little interestingly interpreted like, for example, the Affordable Care Act at one point had Scrivener’s error of sorts that became a legal case. And it all kind of came down to oh, you know, are tax subsidies like applicable to just state-run bodies or state- and federally-run bodies? So there’s a lot of minutiae there. But there’s also this other part where you have a lot of lobby groups who come to Congress people and be like, hey, you should totally have this particular line in this bill you’re introducing. And then you need to figure out what does that line actually do. And so this is where trying to understand what this language does, in context of the entire bill of the existing legal code, is really important. And that’s what the software sort of helps you do. I mean, it’s kind of a really smart word processor. It’s like looking at Google Docs. It’s version control.

McCarty Carino: What about legislation and the legislative process specifically makes this such a useful tool to have?

Chatterjee: Well, Congress deals with a lot of churn. In as far as like legal language goes. Basically, by the time a bill makes it to law, there are so many hands on it, so many different wording changes that it gets really difficult to keep track of all of them. So this is kind of just helping to do that.

McCarty Carino: So when we hear about, you know, lawmakers, having these kind of marathon all night sessions, and then voting at four o’clock in the morning on a 500 page piece of legislation where it seems like there’s no way they could have read it or understood it, maybe, this kind of thing is helping them out.

Chatterjee: Yeah, no, absolutely.

McCarty Carino: How is this tool different or similar to the kinds of generative AI we’ve been hearing about, like, ChatGPT?

Chatterjee: Yeah, I’m glad you asked. Because they’re fundamentally different. And there’s a reason for that. Lawmakers, lawyers, they get very skittish when you try and introduce AI into the business of legal interpretation. So specifically, like AI making recommendations, suggestions, that sort of stuff. Which is not to say there isn’t machine learning happening here. It happens, but in a very limited context. It happens to try and understand legal language. There’s ways to sort of use language understanding to automate an otherwise kind of boring process. But any of the like spicy stuff when it comes to generative AI, this thing doesn’t go anywhere near and it’s likely that it never will.

McCarty Carino: What are some of the limitations of this type of software?

Chatterjee: Well, one, it can’t legally interpret for you. The software is pretty limited in what it is able to tell you about an amendment. It can tell you where the amendment goes, you know, what the change to the existing code will be, but it won’t be able to tell you what that practically does, like what the spin out effects of that, it can’t predict the future. So it can’t really model out the impact of a change to a law. So that’s a limitation. One that maybe we want to have because suppose you are able to like model out future impact. Then you have questions about, OK, how far can we trust the model? What’s the training data for it? So that’s sort of where you get into all of the snarls that machine learning AI runs into generally.

Related Links: More insight from Meghan McCarty Carino

Mohar Chatterjee reported this for Politico along with Alec Snyder. The details include LegisPro converting the entire US legal code into XML, a machine-readable format of text.

Xcential, the maker of LegisPro, then developed a “natural language processing tool” which is the model that underlies AI chatbots like ChatGPT and allows humans to interact with machines in a language that we better understand.

Now, Chatterjee reports, Xcential is tied up in a legal battle with a lobbying firm that claims one of its lawyers came up with the idea.

Chatterjee mentioned that LegisPro is fundamentally different than generative AI like ChatGPT. But Congressman Ted Lieu from Los Angeles used ChatGPT to draft a resolution calling on Congress to study how to best regulate AI like ChatGPT.

He used the prompt: “You are Congressman Ted Lieu. Write a comprehensive congressional resolution generally expressing support for Congress to focus on AI.”

He said he was surprised by how quickly and accurately the tech was able to do just that, and wrote about it in an op-ed for the New York Times.