AI for Engineering Leaders: A Practical Playbook From Leading 20+ Engineers
I lead 20+ engineers and AI runs half my playbook. No theory - practical tools, prompts, and workflows you can steal today.
I’m not going to sell you on “AI will change everything.” You’ve heard that pitch a hundred times.
Instead, I’ll show you what I actually do. Every week. With real tools, real prompts, and real results. Some of this will sound obvious. Some of it will sound borderline crazy. All of it works.
I lead 20+ engineers across multiple teams. I don’t have time to manually connect dots between quarterly reviews, Sentry logs, post-mortems from 8 years ago, and GitLab merge request data. Nobody does. So I stopped trying to do it by hand.
Here’s everything I use AI for as an engineering leader - and how you can steal it.
📋 What’s inside
🧠 My AI advisor that knows me better than my therapist
📓 Daily journaling with AI as a thinking partner
👥 The “Advisory Board” I built for my developers
🔍 Cross-referencing Jira, Sentry, and GitLab with Claude Code
🛠️ Writing custom tools in an afternoon (that used to require budget approval)
🎙️ Analyzing meeting transcripts across 6 teams, I can’t attend
🧠 My AI Advisor Knows My Blind Spots Better Than I Do
This is the single highest-impact thing I’ve done with AI.
I built a custom Gem in Gemini that acts as my personal CTO advisor. Not a generic chatbot - a brutally honest sparring partner that knows my strengths, my weaknesses, and the specific dynamics of my teams.
Here’s what makes it different from just “asking ChatGPT a question”:
The prompt itself isn’t magic. The context is. What makes this Gem genuinely useful is how much I’ve fed it about my world. Over time, I loaded it with:
My personality tests (Gallup StrengthsFinder, DISC, others) - so it calibrates advice to how I actually think and communicate, not generic leadership tips
Quarterly performance reviews for every team member - after several quarters, it can spot behavioral patterns, cultural drift, and warning signs I’m too close to see
My stakeholders - who they are, what they care about, how they operate, what their priorities look like - so when I’m preparing for a difficult conversation or a strategic pitch, the Gem factors in the actual people I’m dealing with
My quarterly goals and division strategy - this turns the Gem into something like a strategic accountability partner. It checks whether my daily decisions align with where I said I wanted to go. When I’m about to commit time to something, it can push back: “How does this connect to your Q2 priorities?”
Numerical data from Jira - cycle time, throughput, lead time - which I cross-reference against my qualitative reviews
That last point is where it gets powerful. I ask the Gem: are the people I flagged as “culturally off” also underperforming on metrics? Or is it the opposite - high performers who are quietly toxic? Connecting these dots manually across 20+ people would take me days. The Gem does it in minutes.
And because it knows my personality profile and my strategy, the advice isn’t generic. It’s calibrated to my decision-making style, my communication patterns, and the specific goals I’m working toward.
I use this advisor for everything: strategic planning for my engineering division, preparing for difficult conversations, sanity-checking my read on team dynamics, and validating whether I’m drifting from my own strategy. It’s become the most critical tool in my stack.
📓 Daily Journaling - AI as a Thinking Partner
I wrote a full article about this approach, and it turned out to be my most popular piece on Substack. So I’ll keep this brief and recommend you read that one for details.
The short version: I do daily reflections with AI. Either in a dedicated Gem or in Claude (Claude works better for this, actually). I write about how my day went - what decisions I made, what felt off, what I’m unsure about.
But here’s the key part. Each reflection runs through a prompt designed to challenge my assumptions. It pushes back. It asks follow-up questions. It looks for cognitive biases I might be falling into.
I’ve split my reflections into several sections (you’ll find the exact breakdown in that article), but the pattern is simple: write honestly, then let AI poke holes in your thinking. Over time, this practice changed how I make decisions entirely. Not because AI is smarter than me, but because it forces me to slow down and articulate what I actually think - and then defend it.
BTW - this is my most popular article on Substack, so chances are you’ll find value in the detailed version.
UPDATE: A while back I discovered an app called Aion Mind that does almost exactly what I described above - structured journaling with AI as a thinking partner. I've been using it for some time now and it genuinely simplifies the process. If you want to check it out, here's my referral link. Full disclosure: it's an affiliate link, but I'd recommend it either way.
👥 The “Advisory Board” I Built for My Developers
I also wrote about this one as a guest post on Alex’s newsletter (hi Alex 👋), so I’ll cover the key ideas here and link to the full piece.
I wanted my developers to use AI for their own growth - not just for coding, but for navigating their careers, making better decisions, and developing leadership skills. So I built a mini-program around it.
The idea is simple: each developer builds their own “Advisory Board” - a custom Gem or Claude project with multiple expert personas. A senior architect. A product manager. A career coach. Whatever mix fits their needs.
It works similarly to my own CTO advisor, but with a twist.
I ask them to include me as one of the board members. Not the real me - their description of me. How they understand my expectations, my communication style, what I value.
The results were fascinating. Developers started telling me: “I asked the AI what Karol would think about this approach, and it gave me a perspective I hadn’t considered.” It’s not about creating a yes-man version of their manager. It’s about helping them internalize the feedback loop that happens naturally in 1:1s - but making it available 24/7.
Beyond the Advisory Board, I also prepared a set of prompts for refinements that teams can use during their planning sessions. And I recommend journaling to them too - so it’s a package: Advisory Board + prompts + journaling.
The last piece I recommend is building an AI assistant that helps them navigate their quarterly goals. I’m a big fan of working with goals across teams and individual contributors. But what I noticed over the years is that people forget about their goals mid-quarter. They set them, feel motivated for two weeks, and then real work takes over. The AI assistant I recommend simply keeps goals visible and checks in regularly - like a lightweight accountability partner.
I wrote about the full quarterly goals framework [link to quarterly goals article] if you’re interested.
🔍 Cross-Referencing Jira, Sentry, and GitLab with Claude Code
This is where things get operational.
I use Claude Code connected to Sentry and Jira to do analysis that would be impractical by hand. Here’s a real example.
We were running a massive migration - dozens of services needed to be both modernized and migrated. Two separate epics in Jira. The obvious question: are we covering everything? Are there services we’re migrating but not modernizing? Or modernizing but not migrating?
I exported our service list from GitLab, pointed Claude Code at both Jira epics, and asked it to find discrepancies. It flagged gaps we’d missed - services that existed in one epic but not the other. The team fixed it by simply adding the missing tickets. Took 20 minutes total.
Without AI, someone would have manually cross-checked two massive epic boards against a service registry. That’s a full afternoon of tedious work - and human eyes miss things in large datasets. Claude Code didn’t.
Sentry log analysis works the same way. I ask Claude Code to analyze recurring error patterns, group them by root cause, and flag what needs attention. Instead of scrolling through dashboards hoping to spot trends, I get a structured summary.
But the best one was this: I asked Claude Code to analyze several hundred post-mortem tickets from the last 10 years. Every production incident we’d ever documented - the problem, the root cause, the resolution, the timeline. All of it is sitting in a Jira project, mostly forgotten.
The insights were phenomenal. Claude Code grouped recurring root causes, identified patterns that spanned years, and proposed an action plan. Things that happened 7-8 years ago were connected to problems we were still seeing today. No human was going to connect those dots across a decade of incident history. But AI did, in one session.
🛠️ I Wrote Custom Software in an Afternoon (That Used to Need Budget Approval)
Our GitLab instance sits behind a VPN. I wanted cycle time metrics for my teams, but every tool on the market required external access to our repos. I searched for months. My team searched too. Nothing fit.
Then I had a thought. What if I just... build it?
I wrote a prompt, started the generation with Gemini (to save tokens on the longer context), and went to a 1:1 meeting with one of my developers. When I came back - I had a working application. A console tool that connects to GitLab through VPN and calculates cycle time metrics.
Did it need some tweaks and bug fixes? Sure. But in roughly 3-4 hours of total effort, I had something that would’ve previously required vendor evaluation, budget approval, procurement, security review, and cross-department installation.
That realization changed my thinking entirely. I now use AI to build tools I never had before - not because they didn’t exist, but because getting them was too expensive or too slow.
Here’s another example. After analyzing our lead time data, we identified that review time was the biggest bottleneck. So I wrote another tool: when someone opens a merge request, it automatically creates a temporary Slack channel, invites the relevant reviewers, pings them about comments, and archives the channel when the MR is merged.
Review time dropped by dozens of percent. The tool is custom-built for our workflow, costs nothing, and took one Friday afternoon to build.
Commercial alternatives exist, but they’re more expensive, less customizable, and require contracts and procurement. I built ours between lunch and the end of the workday.
🎙️ Analyzing Meeting Transcripts Across 6 Teams
With 6 teams, I can’t sit in every daily standup. I’m only present for the key moments. But I still need to know what’s happening.
We use Gemini’s meeting transcription feature. Every refinement, every daily, every planning session gets recorded and transcribed by default.
Then I feed those transcripts into AI and look for things I’d never catch otherwise: Who’s consistently quiet in meetings? Who’s coming up with the strongest ideas? What are the recurring reasons for project delays? What’s the general vibe in each team right now?
I’ll be honest - when we first introduced default recording, some people found it strange. But these are team meetings, not private conversations. Everyone knows they’re being recorded. And over time, nobody pays attention to it anymore. The upside is significant: I get visibility into team dynamics across all 6 teams without being omnipresent.
Company-wide meetings are even more revealing. Monthly all-hands meetings where different departments present their updates? Feed a few months of those transcripts into AI and ask for patterns.
The results can be brutally honest. Why did the finance lead avoid addressing a specific topic? Why did the quality manager stay silent when quality was explicitly discussed? Why did no one ask the obvious follow-up question?
These are the things you’d miss sitting in the meeting, because you’re processing information in real time. AI reviews the transcript without the cognitive load, without social dynamics, without being distracted by the presenter’s charisma.
One caveat: always double-check. Sometimes a person is “quiet” because they were on vacation that week, not because they’re disengaged. AI doesn’t know that. But as a first pass for spotting patterns and formulating questions? It’s incredibly effective.
The Pattern Behind All of This
If you look at everything above, there’s one common thread: AI is most powerful when you give it context that would take a human too long to process.
Performance reviews across multiple teams. A decade of post-mortems. Six teams’ worth of meeting transcripts. Hundreds of Jira tickets across multiple epics. GitLab metrics behind a VPN.
None of these are tasks where AI “replaces” me. I still make every decision. I still run every difficult conversation. I still set the strategy.
But AI processes the raw material faster than I ever could. And that changes the game - because now I spend my time on judgment calls, not data gathering. On connecting with people, not connecting spreadsheets.
If you’re an engineering leader managing multiple teams, start with one thing from this list. Build the advisor. Try the journaling. Analyze your post-mortems. Write a small tool.
You’ll be surprised how quickly it compounds.
A quick ask: If this article helped you think differently, I’d appreciate you shared it with another engineering leaders who might benefit. Or leave a comment with your own experiences navigating these situations - I read every one, and they help me understand what’s actually working (or not working) in the field.
Substack’s algorithm rewards engagement, which means more people discover articles when readers interact with them. So if you found value here, a share or comment genuinely helps other engineering leaders find this.
Thanks for reading.
The post-mortem analysis is the sleeper hit here. Most orgs have a decade of incident history sitting in Jira, and nobody ever goes back to it. Feeding that into the AI tool you have access to and asking for root cause patterns across years is one of the highest signal/effort ratios I've seen.