Thread by Dan Hendrycks
- Tweet
- Apr 14, 2023
- #ArtificialIntelligence
Thread
Since Senator Schumer is pushing for Congress to regulate AI, here are five promising AI policy ideas:
* external red teaming
* interagency oversight commission
* internal audit committees
* external incident investigation team
* safety research funding
(🧵below)
* external red teaming
* interagency oversight commission
* internal audit committees
* external incident investigation team
* safety research funding
(🧵below)
1. External red teaming
Require mandatory red teaming by third parties; they would need x level of access for y months before release to check for dangerous failure modes.
Red teamers are external auditors who discover vulnerabilities or flaws in the safety of AI systems.
Require mandatory red teaming by third parties; they would need x level of access for y months before release to check for dangerous failure modes.
Red teamers are external auditors who discover vulnerabilities or flaws in the safety of AI systems.
2. Interagency oversight commission
Require reviews of major AI models before releasing them into the wild.
When Apple wants to release a new iPhone to the public, it first has to submit a prototype & documentation to the FCC.
This makes companies pay more attention to safety.
Require reviews of major AI models before releasing them into the wild.
When Apple wants to release a new iPhone to the public, it first has to submit a prototype & documentation to the FCC.
This makes companies pay more attention to safety.
3. Internal audit committees
Require AI companies to have an internal auditing team that provides an independent risk assessment to the board members.
The team should be independent from senior management. This is common in the financial sector to reduce risk.
Require AI companies to have an internal auditing team that provides an independent risk assessment to the board members.
The team should be independent from senior management. This is common in the financial sector to reduce risk.
4. Incident investigation team
When an airplane crashes, an independent team investigates to find out what happened and how to prevent it from happening again.
We still don’t know why Bing chat repeatedly threatened users. We want to prevent future AIs from threatening people.
When an airplane crashes, an independent team investigates to find out what happened and how to prevent it from happening again.
We still don’t know why Bing chat repeatedly threatened users. We want to prevent future AIs from threatening people.
5. Safety research funding
Nearly all research is directed at making models more powerful and capable. My current estimate is that 1-2% of NeurIPS papers are safety-relevant.
Personally there should be more funding so that at least 30% of AI research is safety research.
Nearly all research is directed at making models more powerful and capable. My current estimate is that 1-2% of NeurIPS papers are safety-relevant.
Personally there should be more funding so that at least 30% of AI research is safety research.
Many of these risk management ideas are found in consumer protections, information security, and finance.
None of these ideas are silver bullets, and there are many other ideas (e.g., required screening for trojans of models trained on potentially poisoned web-scale data).
None of these ideas are silver bullets, and there are many other ideas (e.g., required screening for trojans of models trained on potentially poisoned web-scale data).
Mentions
See All
Yo Shavit @yonashav
·
Apr 14, 2023
This is an excellent thread of common-sense steps for reducing harms from AI. It’d maintain the US’s AI lead (limiting regulatory burden) while mitigating many of the near-term and long-term safety risks. Esp. if it targets frontier systems & exempts the rest Let’s just do it.