AI Ethicists and Alignment Researchers train AI to be moral by establishing the mathematical frameworks, constitutions, and training techniques (like Reinforcement Learning from Human Feedback) that guide AI to differentiate right from wrong.
Organizations that use AI ethically follow five key principles: fairness, transparency, accountability, privacy, and security.
Dr. Dario Amodei: Co-founder and CEO of Anthropic. He and his team developed "Constitutional AI," a system that trains models to abide by a set of foundational moral principles (a constitution) regarding helpfulness and harmlessness.
Qualifications: Holds a Ph.D. in Biophysics from Princeton and spent years as a senior AI researcher at Google Brain and OpenAI.
Dr. Eliezer Yudkowsky: Co-founder and senior researcher at the Machine Intelligence Research Institute (MIRI). He is one of the foundational voices who popularized the field of AI alignment.
Qualifications: Self-taught researcher in decision theory and computer science with decades of specialized, peer-recognized expertise in AI existential safety and risk mitigation.
Merve Hickok: Founder of AIethicist.org. She works primarily at the intersection of AI governance, algorithmic bias, and societal impact, advising companies on integrating ethics into engineering design. She is focused on AI bias, social justice, DE&I, public benefit and participatory development and governance – as they translate into policies and practices.
Qualifications: Long-term background in corporate governance and human resources, combined with specialized advisory roles at institutions like the Center for AI and Digital Policy.
Dr. Paul Christiano: Former Research Scientist at OpenAI who pioneered much of the reinforcement learning from human preferences (RLHF) used to align Large Language Models (LLMs). He now leads the Alignment Research Center (ARC).
Qualifications: Holds a Ph.D. in Computer Science from UC Berkeley and specializes in designing reward functions and training frameworks for AI agents.
Qualifications to Perform the JobSuccess in this field typically requires a multidisciplinary blend of hard sciences and humanities.
Common qualifications include:
Technical Background: Advanced degrees (often a Master's or Ph.D.) in Computer Science, Mathematics, Machine Learning, or Statistics.
Research Experience: Experience training and fine-tuning neural networks, handling synthetic datasets, and applying probability/game theory to algorithms.
Moral Philosophy: Foundational knowledge of ethics, social sciences, political science, and human cognition to establish what principles the AI should actually prioritize.
Personally, I find this somewhat troubling. It seems like a collection of intellectuals all from the same school of thought.
Most of the people doing the actual training are young tech geeks who appear to not have any real world experience.
Wisdom comes from age and experience. I'd feel more comfortable if they included people outside the tech and university culture including different schools of philosophy, thought and maybe religion too.
Wolfhag
