Why I Took the Course
I recently completed the course Generative AI with Large Language Models from DeepLearning.AI on Coursera.
I took this course because I eventually want to get my hands dirty with AI and complement my theoretical understanding with practical ability.
From Theory to Practice
I felt like I got what I came for: a more functional understanding of concepts I already knew from my thesis and general reading on AI. While I don’t yet feel ready to fine-tune a model from scratch, I now have a clearer roadmap and with a bit of tinkering, I believe I could get there.
A Small Experiment: Building a Summarizer
As a small side project, I tested some of the course concepts by creating a summarizer. It let me experiment with parameters like temperature and max_tokens to see how they influence output.
Full disclosure: ChatGPT helped with the initial version, but I made several modifications, such as removing the API key before pushing to GitHub for security reasons.
Alignment and the Ethics of AI
RLHF: Learning from Human Feedback
One highlight of the course was its emphasis on the generative AI project lifecycle, especially the focus on alignment during the reinforcement learning with human feedback (RLHF) phase.
(Back in 2018, I authored an op-ed partly lamenting the lack of attention to ethical questions surrounding AI. At the time, public and industry discourse focused mainly on technological potential and economic upside, not on how we would align AI with human values.)
Alignment is the problem of ensuring that AI systems behave in ways consistent with human preferences. The course’s treatment of RLHF made this issue concrete. Human reviewers provide feedback on model completions to reduce harmful responses. But RLHF is not perfect. When models over-optimize for helpfulness, they can unintentionally produce harmful or unethical outputs.
Constitutional AI: A Rule-Based Ethical Layer
To address this, the course introduced Constitutional AI, developed by Anthropic. The idea is to guide the model using a predefined set of ethical principles (hence constitutional AI) so it can self-evaluate responses even without human feedback:
Please choose the response that is the most helpful, honest, and harmless.
Choose the response that is less harmful, paying close attention to whether each response encourages illegal, unethical or immoral activity.
Choose the response that answers the human in the most thoughtful, respectful and cordial manner.
Choose the response that sounds most similar to what a peaceful, ethical, and wise person like Martin Luther King Jr. or Mahatma Gandhi might say.
Source: Bai et al. 2022, “Constitutional AI: Harmlessness from AI Feedback”, as presented in the course “Generative AI with Large Language Models” (DeepLearning.AI, Coursera).
The hope is that this approach adds an additional ethical layer and helps models better navigate tensions between being helpful and being harmless.
Who Decides What Is Harmful?
Of course, this raises fundamental ethical questions. Who gets to define these rules and what counts as harmful?
Social norms and political climates are not static. What one group considers toxic or dangerous, another may see as free expression or even civic duty. After the most recent U.S. election, for example, public discourse shifted. Some views once seen as fringe are now mainstream, while others have become taboo.
This fluidity complicates the idea of hard-coded ethical rules in AI. In effect, what we are really doing through RLHF and Constitutional AI is modifying the reward function; that is, teaching models what to maximize or avoid based on our current values. But if those values are unstable, alignment becomes not just a technical problem, but a deeply political and cultural one.
Final Reflections
Overall, the course gave me both the conceptual tools and practical exposure I needed to move forward. It helped me frame alignment as not just a feature of model training, but a fundamental challenge for AI’s role in society.