New York City’s public hospital system announced that it would not be renewing its contract with Palantir as controversy mounts in the UK over the data analytics and AI firm’s government contract.

The president of the US’ largest municipal public healthcare system, Dr Mitchell Katz, testified last week before the New York city council that the agreement with Palantir would expire in October.

He said at the hearing that the contract, which focused on recovering money for insurance claims, was always meant to be short-term, and that there was an “absolute firewall” preventing Palantir from sharing information with US Immigration and Customs Enforcement. He said that the agency has “not had any incidents”.

The contract and related payment documents shared with the Guardian by the American Friends Service Committee and first reported by the Intercept, show that NYC Health + Hospitals has paid Palantir nearly $4m since November 2023. The contract noted that Palantir would be able to review notes about patient’s health and help the hospital claim more money in public benefits through programs such as Medicaid. It also includes a line stating that with permission from the city agency, Palantir can “de-identify” patients’ protected health information and use it for “purposes other than research”.

NYC Health + Hospitals said in an email to the Guardian that it will be transitioning to systems that were made entirely in-house, and there will be no data shared with Palantir or use of the company’s applications after the contract expires. “NYC Health + Hospitals’ use of Palantir technology is strictly limited to revenue cycle optimization, helping the public healthcare system close gaps between services delivered and charges captured, protect critical revenue, and reduce avoidable denials,” the agency said in an emailed statement.

Palantir said it would correct “inaccuracies” in the public record but did not elaborate by press time.

Palantir presence grows in the UK

As New York City’s hospital system prepares to part ways with Palantir, the company is facing similar scrutiny over privacy issues in its £330m agreement with the UK’s National Health Service (NHS). Health officials in the UK are concerned that the controversy surrounding Palantir may stop the nationwide rollout of the company’s data system, even though Keir Starmer is trying to speed up deployment. As of last summer, not even half of the country’s health authorities had started using Palantir’s technology amid concerns from the community and doctors. A 12 March briefing by Medact, a health justice charity, said Palantir’s software could enable “data-driven state abuses of power”, including US-style ICE raids. Palantir has denied that the data could be used in this way, noting that it would be illegal and a breach of contract.

Palantir, which also contracts with the British government’s Ministry of Defence, is expanding its influence in the country – despite backlash from activists and some lawmakers. The Guardian revealed last week that Palantir is trying to gain access to sensitive national financial regulation data.

The Financial Conduct Authority, a watchdog for thousands of financial bodies from banks to hedge funds, awarded Palantir a contract to investigate internal intelligence data to help root out financial crime. That has sparked outcry from some MPs, who have urged the government to halt this agreement. Liberal Democrats called on Monday for a government investigation into the contract. Starmer has dismissed suggestions that the UK has become “dangerously over-reliant” on American tech companies, including Palantir, but noted he preferred to have more domestic capability.

Medact has raised privacy concerns in the UK about Palantir’s ability to access de-identified patient data. (De-identified data refers to data that has been stripped of characteristics that could indicate who an individual is, such as names and social security numbers). In a 12 March briefing for health officials, Medact argued that the NHS’s data privacy protections are insufficient; NHS England has said that data is de-identified as it moves through its national software system, the NHS federated data platform (FDP). But Medact cited concerns that this data can be easily re-identified.

An NHS spokesperson said in an emailed statement to the Guardian that the supplier of the FDP “was appointed in line with public contract regulations and must only operate under the instruction of the NHS, with all access to data remaining under NHS control and strict contractual obligations protecting confidentiality”.

Data privacy concerns

Data privacy experts interviewed by the Guardian said that there are risks in Palantir accessing New Yorkers’ de-identified data for purposes other than research, especially given the company’s vast access to government records, willingness to cooperate with the federal government and ability to connect and analyze large datasets.

“De-identification is not the guarantee it used to be, and it’s getting easier with AI capabilities to re-identify information,” said Sharona Hoffman, a law professor at Case Western Reserve University.

Ari Ezra Waldman, a law professor at UC Irvine who has researched how governments and tech companies use data about individuals, says that we should be concerned “whenever a company like Palantir or a hostile government collects information on vulnerable populations”. He’s particularly concerned about the contract’s provision to use the information for “purposes other than research”. That tells him the government didn’t have enough power to push back on Palantir when negotiating the contract, or didn’t care or know the risk, he says.

Activists claim a victory

Despite the hospital system’s claims that the partnership had no real risks for patients, activists living in New York City, and beyond, are counting this as a win.

Nurses, pro-Palestinian activists and social and climate justice groups applied pressure on the city government as part of a nationwide campaign known as Purge Palantir to stop the company from contracting with government agencies, universities and corporations.

“We don’t think that the same AI systems that are targeting immigrants here in the United States for ICE, as well as choosing places to bomb in Iran, should be the same AI systems used in hospitals,” said Kenny Morris, an organizer with the American Friends Service Committee. The group obtained the NYC Health + Hospitals contract with Palantir through a public records request, and shared the document with the Intercept and the Guardian. The national nurses union and the Boycott, Divestment, and Sanctions (BDS) movement were also involved in the campaign.

Groups with the “No Palantir in our NHS” campaign in the UK are hoping New York City’s public hospital system’s decision to let the Palantir contract expire fuels their own fight, too. Medact and Amnesty International UK told the Guardian in emailed statements that they are calling on the NHS to follow New York City’s example and terminate its £330m contract with Palantir.

“As campaigners in New York have shown, workers and communities can hold our health institutions accountable and push them to make the right choice. We will do the same here, and force NHS England to cancel this contract,” Dr Rhiannon Mihranian Osborne, corporate campaigns lead at Medact, which is in touch with Purge Palantir.