Council signs off on controlled use of generative AI in reports and emails

Council chiefs have approved a generative artificial intelligence (AI) policy and have said great care will be used to avoid potential problems.

Carmarthenshire cabinet members have approved a policy which says staff can use the technology for things like drafting reports, emails, and presentations but must review them for accuracy and appropriateness.

Generative AI can create new content such as text, images and audio based on patterns it has learned from large datasets, a report before cabinet said.

The policy said generative AI did not “know” facts in the way humans did. “They generate content based on probabilities and patterns, which means they can produce inaccurate, misleading, or biased outputs,” said the report.

Content that appeared plausible but was factually incorrect, misleading or entirely fabricated were known, it said, as hallucinations.

All information generated by generative AI must be reviewed and edited for accuracy prior to use, according to the policy, and staff must not enter sensitive or personal data into public tools such as ChatGPT.

Cllr Philip Hughes, cabinet member for organisation and workforce, said the technology offered exciting possibilities and could help the council operate more efficiently while maximising value for money. “As an authority we will embrace this agenda but with great care,” he said.

The policy added that all generative AI use must be explainable, reviewed by humans, and align with public service values. Compliance is mandatory and breaches may lead to disciplinary action.

Council leader Linda Evans said generative AI “can do amazing things” but had to be used in a way that protected the authority.

The report cited examples of things going wrong – a council which faced legal challenges after relying on generated data that incorrectly identified properties for council tax reassessment, and a screening of job applicants by a public sector organisation which inadvertently introduced bias and led to discriminatory hiring practices.

Carmarthenshire Council’s new policy has been shaped by the UK’s National Cyber Security Centre and Welsh Government.

Cllr Aled Vaughan Owen, cabinet member for climate change, decarbonisation and sustainability, said generative AI tools had a significant energy and carbon footprint and that their use had to be “in line with our decarbonisation aims.”