Critera for solutions in theory

1. Could do superhuman long-term planning

2. Ongoing receptiveness to feedback about its objectives

3. No reason to escape human control to accomplish its objectives

4. No impossible demands on human designers/operators

5. No TODOs when defining how we set up the AI’s setting

6. No TODOs when defining any programs that are involved, except how to modify them to be tractable

​

philosophical problem → computer science problem