Learning AI Engineering as a Software Developer
My journey from building Java backends to exploring LLMs, RAG pipelines, and prompt engineering. What worked, what didn't, and what I'm still figuring out.
1. Where I Started
Six months ago, I was comfortable. Building REST APIs in Spring Boot, frontend in React, database work in PostgreSQL. Traditional full-stack development.
AI was something I used—ChatGPT for quick answers, Copilot for autocomplete—but not something I understood. The difference between a model and an API was vague. I couldn't explain how GPT actually worked beyond "it predicts the next token."
I wasn't trying to become an ML researcher. I wanted to build applications that use AI—which is what most companies actually need.
2. Why AI Engineering
A few things pushed me:
- Curiosity: Everyone's talking about AI. I wanted to understand it beyond the headlines.
- Career: AI integration is becoming expected in products. Pure backend work is changing.
- Capability: I saw what others were building with AI and wanted those tools in my arsenal.
- Fear of missing out: Honestly. The space was moving so fast I felt left behind.
The Distinction
Early on, I realized there's a difference between:
- ML Engineering: Training models, data pipelines, MLOps
- AI Engineering: Using existing models, building applications, prompt engineering, RAG
I went for the second path. It's more accessible from a SWE background and more aligned with what I actually want to build.
3. First Steps (and Missteps)
Here's what I did wrong:
Mistake 1: Starting with Theory
I spent weeks reading about transformers, attention mechanisms, backpropagation. Interesting? Yes. Practical for building apps? No. I learned a lot but couldn't build anything.
Mistake 2: Tutorial Hell
Followed LangChain tutorials, built what they built, understood nothing. When something broke, I was stuck. Copy-pasting isn't learning.
Mistake 3: Chasing Every New Thing
New model drops → new framework → new technique. I was overwhelmed trying to keep up instead of deeply learning anything.
// what finally worked
- One project at a time
- Learn just enough to build the next feature
- Go deeper when something doesn't work
- Accept that I can't learn everything
4. Resources That Actually Helped
// hands-on learning
Build things. Start with a simple chatbot. Then add RAG. Then add tools. Iterate. Break things. Fix them.
most important// documentation over tutorials
Official docs for LangChain, OpenAI API, Anthropic API. They're drier than tutorials but more accurate and comprehensive.
ground truth// andrej karpathy's neural networks series
Finally understood what's happening under the hood. Not necessary for building apps, but gives intuition that helps debug.
theory foundation// 3b1b neural network videos
Visual explanations of how networks learn. Made the math intuitive.
visual learner friendly// jerry liu's llm apps talks
Practical perspectives on building with LLMs. Less hype, more reality.
practical insightWhat Didn't Help
- Coursera-style long courses (outdated by the time you finish)
- Twitter/X threads (too superficial)
- Hacker News debates (mostly academic)
- Papers (unless you're implementing something specific)
5. Learning by Building
The best way I found to learn: have a project, hit problems, solve them.
Project 1: Simple Chatbot
Started with OpenAI API, basic conversation handling. Learned about tokens, context windows, temperature.
Project 2: RAG Pipeline
Built a document Q&A system. Learned about embeddings, vector databases, chunking strategies, retrieval metrics. This is where things got interesting.
Key lessons:
- Chunking strategy matters more than the model
- Retrieval quality is the bottleneck
- Simple often beats complex
Project 3: Agent with Tools
Built something that could use tools—search web, run code, read files. Learned about function calling, tool schemas, agent loops.
This is where I understood the difference between "chat with AI" and "AI that can do things."
Every project taught me one thing well. That's better than learning ten things superficially.
6. Mindset Shifts
Coming from traditional software, I had to unlearn some things:
Deterministic → Probabilistic
In traditional code, same input = same output. With AI, same prompt can give different results. Testing becomes harder. Debugging is different.
Exact Specification → Iteration
You can't spec an AI feature like you spec an API. You iterate on prompts, test outputs, adjust. It's more like design work than coding.
Code Quality → Prompt Quality
Bad code can be refactored. Bad prompts are harder to untangle. Writing clear instructions is a skill I'm still developing.
Few-Shot Learning
Showing the model examples of what you want is often better than describing it. I learned to think in examples, not just instructions.
7. What I'm Still Learning
I'm not an expert. Here's what's still on my list:
- Fine-tuning: When to do it vs when to use RAG
- Evaluation: How to measure if my AI features are actually good
- Cost optimization: Balancing quality with API costs
- Multimodal: Working with images, audio beyond just text
- Agents: Building more sophisticated autonomous systems
The field moves fast. What I learned six months ago is already dated. The skill isn't knowing everything—it's being able to learn quickly when you need to.
8. Advice for Fellow SWE's
If you're a software engineer looking to get into AI engineering:
- Start with APIs: OpenAI, Anthropic, Gemini. Build something small this week.
- One project at a time: Don't try to learn everything. Pick a project and learn what you need.
- Your SWE skills transfer: System design, error handling, testing, API design—all still relevant.
- Don't over-theory: You don't need a PhD to build AI apps. Understand enough to debug, then move on.
- Accept imperfection: AI outputs are never 100% reliable. Build systems that handle that.
- Stay practical: The hype cycle is intense. Focus on what helps you build better software.
The goal isn't to become an AI researcher. It's to become a better engineer who can work with AI.
I'm still on this journey. Learning constantly, building when I can, accepting that I'll never know everything. That's the nature of a field this fast-moving.
If you're starting out, feel free to reach out. Always happy to chat with fellow engineers figuring this out.