AI for Local Government: Our Task Force Experience, Policy Guidance, and What’s Next

by: Kate Garman Burns

On June 23, 2023, MetroLab announced its second “In the Lab” initiative: GenAI for Local Governments. The timing was ripe; we just announced the publication of data governance policy guide, and ChatGPT launched just seven months prior. The task force was designed as a “by practitioners, for practitioners” initiative to infuse local government leaders and researchers with the goal of producing best practices and policy guidance on AI.

Artificial intelligence and Generative AI have massive implications. It’s the sort of disruption / innovation that is akin to the internet: a sweeping evolution across multiple sectors with potential for good and for harm. We wanted to hone in on the service delivery and policy areas that were specific to local government. The task force voted for the following focus areas: community engagement, Open Data + 311, cybersecurity, optimizing permits, procurement, and transportation.

Right away, we welcomed our academic colleagues to help teach the task force the fundamentals of the technology. We also asked our private sector task force colleagues to present the world of the possible; how can their products utilize AI or help local governments better use it. Then, we asked our local government members “what is preventing you from using AI today?” You can find our complete write up on the results here, but in short, five issues surfaced:

1. Local governments are hesitant to use GenAI without proper policies in place first — including fundamental data governance processes.
2. Privacy concerns.
3. Lack of diverse vendors (including small businesses and local organizations).
4. Technology capabilities and the risk of inaccuracies. Hallucinations are a serious concern. Context is, too. For example, what does “safety” mean to me, versus what it may mean for you?
5. Staff capacity and expertise.

As the task force continued, we met in subcommittee meetings over the course of several months. We also saw other organizations put together initiatives to support local governments considering use cases and policies. This brought a shift to our approach, as we recognize helpful resources for local governments, including the GovAI Coalition’s guidance on vendor agreements and the National Association of Counties AI County Compass: A Comprehensive Toolkit for Local Governance and Implementation of Artificial Intelligence.

MetroLab tracked developments at both the state and federal levels. This included helpful documents like OMB’s guidance. The guidance focuses on improving the capacity of federal agencies to acquire an AI system or service. It lays down the considerations that need to be assessed to ensure that the components of an AI system or service, such as data, model, are not operating in a manner that negatively impacts rights and safety. And state legislation is moving forward at a rapid pace. You can find a state AI legislation tracker here.

With multiple levels of government working on AI policy, we wanted to bring together a group of stakeholders to discuss how we could work together. On June 27, 2024, MetroLab Network hosted an in-person roundtable to bring together multiple stakeholders with the specific intention of understanding how multiple levels of government can come together to develop guidance and regulatory schemas on artificial intelligence. This roundtable included representatives from six federal agencies, three local governments, two state legislators, and several local government associations.

Ultimately, the roundtable was an opportunity to identify the questions we need to answer to move forward in an effective and coordinated way. These questions include:

From State and Local Governments:

Multiple Levels of Government Creating Regulations

1. How can we define the level of responsibility of different governments (federal, state, and local)?
2. As states are considering AI legislation, three questions come to mind: What is the breadth that states should govern, generally? What do state legislators need to regulate now as federal and local governments are deliberating next steps? What should cities be focusing on, what is their appropriate purview?
3. How do state and local governments advocate for more resources to understand and use AI?
4. Can federal agencies “translate” existing or upcoming guidance for local governments, highlighting specific areas that may impact other levels of government?

Evaluation + Bias

1. Since local governments (mostly) are not the entity building the AI tools, but use them to gain insights, how can it be ensured that bias existing in historical plans is not perpetuated?
2. How can local governments conduct rigorous evaluation?
3. What does it mean to be ethical in AI in practice? How can we operationalize AI ethics? For example, how can a city address bias in criminal justice and public safety?
4. How can we, as a government, create tools for processes such as impact assessments?
5. Local governments need clarity on the required capabilities and resource requirements to adopt and use AI. There is a need for resources and assistance in developing rigorous testing and quality review methodology.
6. How can cities share their lessons learned with others, for example, on vetting different vendors? How can we share documentation to help develop a “rights and safety” checklist, or impact checklist?

Workforce + Training

1. How can we provide training opportunities to state and local employees to upskill?
2. How can we empower people to think about completing their foundational work to enable easy AI adoption?
3. What is the most efficient use case for small cities and counties?

Questions from Federal Agencies to State and Local Partners:

1. What is the best way to deliver the data to you? Is data at the local level AI-ready?
2. What are lessons learned from state and local governments that federal agencies can learn from?
3. How can AI bring efficiencies to the federal grant life cycle?

The June 27th roundtable also generated a list of research questions that could help all levels of government move forward with AI policy. Those questions are included in our publication “AI Research Needs in the Service of Local Government Frameworks and Use Cases.”

This leads us to our final outcomes from the GenAI for Local Government Task Force. In sum, the task force gathered:

• 45 local governments
• 22 universities
• 5 co-chairs
• 7 Subcommittees (with 11 individuals leading those subcommittees)
• 50 resources cited at our Resource Library — a one stop shop of publications and policy frameworks from and for local governments

As a result of this work, MetroLab is publishing three deliverables. First, we are publishing a policy guide specific to community engagement. This guide includes ways to proactively engage communities to proactively shape GenAI policies such as community education, ways in which local governments can increase transparency, and more. Second, a call to the research community. At every turn, we asked what research is needed to further efforts on understanding the technology and its impact. We are providing an updated list of research questions as identified by the dozens of stakeholders involved with the task force. And finally, a white paper on cybersecurity and privacy. This whitepaper identifies the unique ways AI is impacting cybersecurity and privacy protections, for example, citing the increasing difficulty in enforcing consent and the right to be forgotten mechanisms.

Here are some other things we learned:

1. The difference between “AI” and “GenAI” — these mean very different things technologically, but often conversations would casually swap the two.
2. Policy development for this particularly innovation is extremely challenging as it requires a minimum understanding of how it works. The degree of this requirement has not been seen before (for example, I don’t need to know how the engine of a car works to understand parking policies).
3. The number one way members of our task force are getting news on AI developments is at conferences and in-person/virtual meetings.
4. We later surveyed our task force and while there is concern and worry about bad actors, survey respondents felt optimistic about the promise of AI.
5. Impact assessments are seen as a critical element to support ethical AI, but state and local governments need help in: Standardizing impact assessments, Resources to have staff + expertise to perform them, and Guidance on auditing / third party vendors who can vet or audit.
6. Moving forward, thinking about specific use cases is a helpful way to organize policy discussions. MetroLab and its ecosystem should consider some specific use cases we can all support and use as an entry point to determine what resources and regulatory frameworks are required.

As is the case with our Data Governance for Local Governments Policy Guide, we intend to make these living documents, evolving along with best practices and newly discovered research needs. We will also update our resource library regularly on our website. We encourage any and all feedback, please email us at info@metrolabnetwork.org.

We are extremely grateful to every individual who participated in our task force. While our meetings have concluded, our work continues to support local governments who seek to develop policy guidance and use cases for AI.. We look forward to seeing the development of policy guidance and use cases, and look to research and science from our university partners to bolster these successes.

About MetroLab: MetroLab Network is a nonprofit in Washington DC that aims to equip local governments with science and research. It serves as a convener of an emerging academic practice focused on integrative, use-inspired, community-focused research, done in partnership with local government and communities. We cultivate partnerships between universities and local governments to drive research-informed, evidence-based policy and enable data and technology transformation; we foster a peer network of stakeholders from academia and local government that constitute an applied, interdisciplinary field of research and practice; and we connect to an ecosystem of federal, philanthropic, and civic partners with a shared interest in the promise of civic research and innovation.