The AI Frontier: Get Paid To Build Smarter LLMs

The future of AI is being built by a handful of companies: OpenAI, Anthropic, and DeepMind, among others. One of the core ingredients for improving the performance of LLMs is the use of reinforcement learning from human feedback (RLHF).

Join us for a live conversation with Mercor's cofounder and CTO, [**Adarsh Hiremath**](https://www.linkedin.com/in/adarsh-h-623941204) hosted by Taro's cofounder and CEO, [**Rahul Pandey**](https://www.linkedin.com/in/rpandey1234/). Mercor is a key partner to enable RLHF for many frontier model companies. As part of this, Mercor offers 1000s of part-time, remote-friendly roles with competitive compensation.

This is a new type of white-collar work that involves debugging, validating, and creating scenarios for LLMs. As the capabilities of models improve, the expertise required to train models keeps increasing. One of the highest-priority areas is around coding, which means Software Engineers have a special role to play in this type of work.

This event explains the pros and cons of AI lab gig work, what exactly the work entails, and how much you can earn.

Some topics we’ll cover:

- Typical hourly rates and the compensation model
- Project-to-project variability
- Unique challenges of remote collaboration in this type of work
- What does the day-to-day look like with this type of work?
- How does this work impact the development of LLMs and AI agents?
- How will this work evolve over the next 5 years?

There will also be time for open Q&A from the audience.

Mercor is an AI-powered recruiting platform that has become one of the largest sources for human labor for foundation model companies. Mercor has raised more than $100 million dollars and is located in San Francisco, CA. Explore [**relevant jobs on Mercor here**](https://work.mercor.com/?referralCode=91aab11f-ce55-40b3-9c2d-f74576cac28f) (100+ Software Engineering jobs available).

Decoding the One Big Beautiful Bill Act for Tech Workers

New tax law, same tax code for tech workers… mostly.\
\
President Trump signed the One Big Beautiful Bill Act into law on July 4, 2025.\
\
In terms of the impact to your taxes, this bill largely maintains the status quo by making permanent most of the tax changes introduced on a temporary basis through the Tax Cuts and Jobs Act in 2017.\
\
It does, however, add a few new wrinkles to the tax code.\
\
I’m hosting a live webinar next week to break it all down for tech professionals like you.\
\
🎙️ Webinar: Decoding the One Big Beautiful Bill Act for Tech Workers\
🗓️ Date & Time: July 24, 2025 from 2-2:45 PM Pacific Time\
📍 Where: Live on Zoom\
💡 Especially relevant for:\
👉 Anyone in a high-income bracket that pays a lot in state & local tax\
👉 Folk planning to make Roth conversions in the next few years\
👉Those considering retiring at 65+ in the next few years\
👉 Anyone considering taking a career break or on an ACA plan\
\
We’ll cover:\
\
✅ Changes to the deductibility of state and local taxes\
✅ How the new bill affects ISO exercises and your likelihood of paying AMT\
✅ What's up with Section 174?\
✅ Changes to QSBS\
✅ "No tax on tips/overtime"\
✅ Deductibility of new U.S. car loan interest\
✅ The new "Trump account"\
✅ Impacts to Medicaid\
✅ The new bonus deduction for seniors\
✅ Updates to clean energy and EV credits\
✅ How the new rules impact ACA health insurance subsidies

Machine Learning System Design Ask & Learn

**Event description**

Join us remotely for an interactive Q&A with Ilya Reznik—Founder of MLEPath, veteran ML engineer with nearly 15 years at Meta, Twitter, and Adobe, and interviewer of hundreds of real-world ML system design (MLSD) interviews (plus nearly 1,000 mock sessions in the last two years). Whether you’re tackling your first MLSD interview or leveling up for senior roles.

- **Core frameworks & approaches**: how to structure your end-to-end design, balance scalability, latency, cost, and reliability
- **Key components & trade-offs**: data pipelines, model serving, feature stores, monitoring, and A/B testing
- **Real-world insights**: pitfalls Ilya’s seen candidates fall into—and what top performers do differently
- **Live feedback**: get tailored advice on designing everything from recommendation systems to real-time anomaly detection

Perfect for ML engineers aiming to land roles at FAANG, startups, or anywhere that demands rock-solid system design chops. Come ready with your toughest questions—and leave with a clear, actionable roadmap for your next MLSD interview.

How To Properly Measure Your Data

In this lesson, we explore the essential discipline of **measuring data effectively**, a foundational step in the DMAIC process. Drawing from Amazon’s own journey, we explain why rigorous data management, real-time reporting, and a culture of confronting the brutal facts are key to achieving operational excellence.

### Key Takeaways:

- **We built a disciplined system for data measurement**: Measurement goes far beyond selecting metrics—it requires clean data, reliable pipelines, centralized data warehouses, governance, security, and standardized reporting tools like Tableau or Looker.
- **We invested in an unbiased analytics culture**: Jeff Bezos asked our CFO to form an independent analytics team whose sole job was to uncover the truth, not protect orgs or executives. These analysts had both financial and operational alignment but prioritized integrity in reporting.
- **We created mechanisms to confront reality**: Inspired by Churchill and Jim Collins, we institutionalized a culture of *brutal facts*—establishing metrics and systems that allowed leadership to face problems head-on, rather than hide or distort them.
- **We operationalized this with the Weekly Business Review (WBR)**: The WBR report was auto-generated by 7 a.m. every Monday, standardized, and packed with hundreds of metrics—enabling every department to identify trends, anomalies, and action items before the company-wide WBR each Wednesday.
- **We learned that slow is fast**: Building accurate, automated, real-time measurement systems took time and leadership buy-in. But in the long run, these systems helped us move faster and make better decisions at scale.

Ultimately, this lesson shows that **data measurement is not a tech problem—it’s a leadership imperative**. By treating it as a strategic function, we created a system that helped Amazon scale with confidence and clarity.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Amazon Case Study: Detail Pages

In this lesson, we apply everything we've learned about input metrics by walking through a real-world case from Amazon—how we evolved our **selection metric** to better align with customer needs. This story illustrates the complexity, rigor, and iteration involved in making metrics truly actionable and impactful.

### Key Takeaways:

- **We started by identifying strategic input metrics**: Inspired by Amazon’s **growth flywheel**, we identified controllable levers—like low prices, broad selection, and fast delivery—that directly improved the customer experience and fueled long-term growth.
- **We didn’t wait for perfect metrics to begin**: We initially used simple metrics like the number of product detail pages to represent selection, even if they were imperfect. Early measurement got teams moving in the right direction.
- **We learned through missteps**: Our first metric for selection encouraged teams to add lots of SKUs—regardless of demand. Sales didn’t grow, and excess inventory piled up. This forced us to iterate.
- **We refined our metric to reflect actual customer experience**: Over time, we evolved to more nuanced metrics like **detail page views for in-stock items**, and ultimately to **Demand-Weighted Coverage FastTrack In-Stock Percentage (DWC FT %)**—a metric that captured whether customers could *find*, *want*, and *get* an item quickly.
- **We aligned goals with customer value**: Category managers were now measured by how well their products met real customer demand and delivery expectations—not just raw catalog size.
- **We replaced estimates with precise real-time measurements**: When anecdotes didn’t match our metric outcomes, we realized the problem wasn’t in customer feedback—it was in how we were measuring. We moved from snapshot-based approximations to real-time software tracking, which required serious engineering investment.
- **We treated metric-building as an ongoing process**: Measurement accuracy improved over years of trial and error, and nearly every customer-facing metric eventually transitioned from rough proxies to live calculations.

### To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>

- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>

- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>

- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Weekly Business Review

###### In this lesson, we introduce the **Weekly Business Review (WBR)** as Amazon’s core mechanism for consistently analyzing, improving, and controlling business performance. This structured, weekly ritual helped us align leadership around metrics, dive deep into customer experience, and create a culture of truth-seeking and operational rigor.

### Key Takeaways:

- **We used the WBR to bridge strategy and detail**: Contrary to conventional advice, we didn’t choose between high-level strategy and operational detail—we embraced both. The WBR let senior leaders dive deep while still steering big-picture strategy.
- **We built the WBR around the customer journey**: Our report flowed from financials to marketing, browsing, shopping, fulfillment, and customer support. Each department leader owned and explained their metrics—not analysts—ensuring accountability and depth.
- **We started before everything was perfect**: We launched the WBR even before our metrics were fully refined. By including many metrics upfront, we learned over time which ones provided signal and which were noise.
- **We continuously refined our metrics**: In each WBR meeting, we didn’t just review results—we asked whether we were measuring the right things and how to improve clarity and relevance of our metrics.
- **We made it a disciplined habit**: Reports were automated and delivered every Monday by 7 a.m., enabling prep ahead of the all-company WBR. This rhythm created organizational focus, enabled early issue detection, and aligned everyone around the brutal facts of performance.

The WBR was not just a meeting—it was a **weekly dose of operational truth** that made Amazon more customer-obsessed, data-informed, and execution-focused.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

The Do's and Don'ts For Good Metrics

Here's the exercise worksheet: [Metrics Template and Worksheet Combined (B2C + B2B)](https://firebasestorage.googleapis.com/v0/b/tech-career-growth.appspot.com/o/courses%2Fslides%2FMetrics%20Template%20and%20Worksheet%20Combined%20$B2C%20%2B%20B2B$.xlsx?alt=media&token=3951bfd7-ef9e-4c81-9245-80043a4973d0)

In this wrap-up, we assign a powerful, hands-on homework exercise: to generate a **comprehensive set of input and output metrics** that can answer three foundational business questions. This process, built on the tools and frameworks we've covered, is the first real step in building a metrics-driven, customer-obsessed organization.

### Key Takeaways:

- **We focus on three key questions**:

  1. How did our business do last week?

  2. What was the customer experience like last week?

  3. Are we on track to hit our goals?

- **We use four core methods to identify input metrics**:

  1. **Metrics Mapping** – Drill down from business goals to uncover metric drivers.

  2. **Customer Experience Mapping** – Map and measure every customer interaction.

  3. **Customers** – Mine support data and engage with end users directly.

  4. **Defects** – Use root cause analysis (like CoEs) to find and track preventable errors.

- **We cast a wide net—hundreds of metrics is a good start**: This is not about picking the “best” metrics yet. We brainstorm 200–300 across all functions without concern for whether they’re inputs or outputs, or even if they can be measured immediately.

- **We avoid vague or compound metrics**: Good metrics are discrete, observable, and repeatable by different people or systems. Avoid averages where possible—percentiles and RMS error are often better indicators of variation and performance.

- **We assign priorities and assess data readiness**: After brainstorming, we tag each metric as Priority 1, 2, or 3 and indicate whether the data is currently available. This helps build a phased implementation plan for metrics collection and review.

This assignment launches the foundational work needed to operationalize a **weekly business review (WBR)**. It sets the stage for measurement, analysis, improvement, and control—ensuring our teams are focused on the right inputs to drive the right outcomes.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Learning From Mistakes

In this section, we reflect on how **mistakes and operational defects** became valuable sources of new input metrics at Amazon. Through a disciplined **Correction of Errors (CoE)** process, we turned setbacks into opportunities for learning, system improvement, and metric discovery.

### Key Takeaways:

- **We embraced mistakes as learning opportunities**: We openly discussed errors and failures—not to assign blame, but to understand what went wrong and how to prevent it next time.
- **We asked predictive questions**: For every defect, we asked: *What metric could have predicted or prevented this?* Often, the answer revealed a missing input metric that we needed to start tracking.
- **We built systems to measure what we were missing**: Once we identified a critical input metric, we prioritized building the tools and infrastructure to capture it—and ensured it was reviewed in our Weekly Business Review (WBR).
- **We used a structured CoE process**: The CoE included five steps—identifying the error, documenting and responding, root cause analysis (often using the “Five Whys”), developing corrective actions, and then implementing and monitoring those changes.
- **We integrated input metrics into operational reviews**: Many of the metrics that emerged from the CoE process became part of our regular metric reviews, helping us catch future problems early and improve reliability across systems.

This practice helped us create a culture where **defects became a trigger for innovation and deeper operational control**, not just patchwork fixes.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Get Hands-On Experience With Frontline And Customers

In addition to data and structured methods, **immersive, first-hand experiences and direct customer research** gave us some of the most impactful input metrics and ideas at Amazon. These approaches helped ground our innovation efforts in real customer needs and operational realities.

### Key Takeaways:

- **We gained critical insight from frontline experience**: During Amazon’s early years, leadership—including Jeff Bezos—regularly worked in fulfillment centers and answered customer service calls during peak seasons. These visceral experiences helped us better understand operational pain points and identify new input metrics.
- **We institutionalized hands-on learning**: The value was so great that we made it a requirement—every Amazon employee spends time in frontline roles each year. This creates empathy, sharpens customer focus, and surfaces measurable opportunities for improvement.
- **We used customer research to drive innovation**: Surveys, focus groups, and third-party data sources gave us clear, actionable feedback. A simple survey on why customers didn’t shop more often revealed four key barriers—shipping cost, delivery speed, selection, and price—which directly shaped the creation of Amazon Prime and our long-term priorities.
- **We traced key customer complaints back to inputs**: From that early research, we began measuring and improving core inputs like in-stock rate, selection breadth, delivery speed, and pricing—cementing them in our flywheel as long-term business drivers.
- **We leveraged sales and customer interactions in B2B contexts**: Especially in business-to-business scenarios, we worked closely with sales teams and met customers in person to gather input-rich feedback, helping us refine offerings and discover metrics specific to client pain points.

Together, these methods reinforce the importance of **direct, empathetic engagement with customers and operations**—not just to build understanding, but to generate input metrics that we can track, improve, and build entire services around.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Mine Customer Support Data

We expand beyond metrics mapping and customer experience mapping by introducing **additional, nuanced techniques** to help us continuously refine our input metrics. These customer- and defect-driven approaches give us real-world signals that are often overlooked—but are rich with opportunities for improvement.

### Key Takeaways:

- **We believe customer and shareholder interests are aligned**: At Amazon, we were driven by the conviction that obsessing over customers naturally serves the long-term interests of shareholders. This philosophy shaped our approach to metrics and improvement.
- **We treat every customer interaction as a source of insight**: Whether the "customer" is a buyer on Amazon or an internal team we support, we use their feedback to uncover meaningful input metrics that help us improve service and outcomes.
- **We mine customer support data as a diagnostic tool**: We tracked the top reasons customers contacted support—questions like “Where’s my stuff?” or complaints like “You sent the wrong item”—and used these to create new metrics such as “the switcheroo,” which measured item mix-ups.
- **We made problem frequency visible and actionable**: These contact reasons and metrics were reviewed weekly in our WBR, ensuring we remained focused on resolving the most impactful customer issues.
- **We talked directly to customer-facing teams**: We routinely engaged with customer support representatives to gather firsthand insights about recurring pain points by category—adding human intelligence to our data-driven approach.

These additional techniques may not generate as many metrics as mapping exercises, but they provide powerful context and depth to our understanding of what truly matters to customers.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Amazon Metrics Map Example

In this lesson, we walk through **two core techniques—Metrics Mapping and Customer Experience Mapping**—that helped us systematically discover and define the input metrics that truly matter. These methods gave us a more granular, customer-focused way to understand and improve our business performance at Amazon.

### Key Takeaways:

- **We used Metrics Mapping to break down strategic goals**: Starting with broad objectives like lowering cost structure, we recursively drilled down to discover the specific drivers (e.g., shipping cost, pick cost, forecast accuracy) and converted them into measurable input metrics.
- **We cast a wide net during brainstorming**: In the early stages of metrics mapping, we didn’t worry about whether we could measure something yet or whether it was immediately useful. Our priority was to be comprehensive.
- **We built Customer Experience Maps to capture every user touchpoint**: From site entry to search, checkout, and delivery, we identified every customer interaction and defined metrics for speed, quality, and cost—measured both from the customer's and Amazon’s point of view.
- **We stayed grounded in the customer’s perspective**: Instead of focusing on internal outcomes like revenue per page view, we measured how intuitive, fast, and helpful each experience was for the customer, ensuring our metrics aligned with long-term value creation.
- **We operationalized customer obsession**: These techniques gave us a structured, measurable way to uphold our customer-obsessed culture. Other companies have found these frameworks to be transformative in how they assess and improve their own customer experience and operations.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Control

In this final phase, we ensure that the gains achieved through improvements don’t erode over time. Control is about sustaining operational excellence, detecting deviations early, and building mechanisms—both human and automated—that keep the system performing at its best.

### Key Takeaways:

- **We implement control mechanisms to preserve gains**: Once we achieve desired service levels, we don’t assume they’ll stay that way. We actively maintain them through tools like regular audits, alarms, and ongoing data validation.
- **The Weekly Business Review (WBR) reinforces control**: Our WBR report includes 200–300 metrics. We review only the outliers, but by tracking all key indicators weekly, we maintain vigilance and are able to quickly detect when something goes off-course.
- **We rely on automation for precision and scalability**: From real-time alarms to machine-driven decision-making, we automate wherever possible. These systems are monitored to ensure they perform within tolerance thresholds.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Building A Metrics Map

In this lesson, we explore **practical techniques** for identifying the input metrics that matter most. Through methods like **metrics mapping** and **customer experience mapping**, we gain a clearer, more quantitative view of what drives our business forward—especially from the customer’s perspective.

### Key Takeaways:

- **We prioritize input metrics and customer interactions**: Too often, businesses rely heavily on output metrics. We recommend looking deeper into input metrics—particularly those tied to customer behavior and experience.
- **Metrics Mapping helps us discover drivers of key metrics**: We start with a business goal or known metric, then recursively ask what drives it and how we can measure those drivers. This technique reveals a broad and useful set of input metrics.
- **The process is recursive and comprehensive**: We drill down from each metric or goal until we can’t go any further, building an extensive network of quantifiable drivers. Repeating metrics across branches is fine—what matters is completeness.
- **The end goal is a flat, exhaustive list of metrics**: While the mapping exercise may begin as a hierarchy, we ultimately compile all the discovered metrics into a single, non-hierarchical list for practical use and monitoring.
- **Finding input metrics is an ongoing process**: This isn’t a one-time task. We continuously revisit and expand our metric set as our business, goals, and customer needs evolve.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Improve

In the Improve phase, we focus on **deploying resources and initiatives** that shift our input metrics in the right direction, knowing that this in turn drives our desired output results. Improvement at Amazon has always been a cycle of innovation, experimentation, and rigorous measurement.

### Key Takeaways:

- **We prioritize inputs to drive results**: We align our teams, projects, and resources around improving leading indicators—our input metrics—which ultimately influence lagging outputs like sales and customer retention.
- **Click to Deliver Time became a key input metric**: We tracked every step of the fulfillment process and launched targeted initiatives to speed up delivery, knowing it directly impacted customer satisfaction and engagement.
- **We scaled operations through iterative improvements**: Over two decades, we grew our global delivery infrastructure—fulfillment centers, delivery stations, air cargo, and driver networks—through thousands of iterative changes, many of which were experiments that included both successes and failures.
- **We formalized improvement with the Working Backwards PR/FAQ process**: This mechanism ensures we innovate from customer needs, forcing teams to start with a press release and FAQ to define how a proposed improvement solves real problems.
- **Improvement and analysis form a feedback loop**: Every improvement is followed by measurement and analysis. This iterative cycle lets us reinforce what works and eliminate what doesn’t, ensuring continuous refinement of our systems.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Analyze

In this phase, we dive into understanding how our **inputs drive outputs**, using structured analysis to detect patterns, pinpoint issues, and reduce variability. Building a culture of disciplined, data-driven evaluation was essential to improving business performance at Amazon.

### Key Takeaways:

- **We focus on understanding input-output relationships**: The Analyze phase helps us distinguish true patterns (signal) from random fluctuations (noise) and enables us to control variability in performance.
- **We use structured mechanisms for analysis**: DMAIC demands that we move beyond ad hoc methods by creating formal processes and tools for analyzing metrics across the company.
- **We built a culture of rigorous data review**: Regular, structured reviews became part of our operating rhythm, reinforcing analytical thinking throughout our teams.
- **We institutionalized the Weekly Business Review (WBR)**: One of our most powerful tools, the WBR was developed to enable comprehensive and recurring analysis, and it remains a central part of how we stay aligned and data-informed across all business units.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Measure

In the **Measure** phase of the DMAIC process, we focused on building a consistent, reliable foundation for capturing and interpreting data. This phase was critical in enabling accurate decision-making and accountability across our rapidly growing organization.

### Key Takeaways:

- **We standardized measurements across the company**: To avoid confusion and misalignment, we worked for years to unify definitions, formulas, and data structures across multiple data warehouses and teams.

- **We eliminated self-serving metrics**: Jeff Bezos directed our CFO, Warren Jensen, to form an independent analytics team that would report the unbiased truth, rather than letting individual teams present metrics that suited their own narratives.

- **We invested in strong data infrastructure**: Reliable measurement required robust systems for data capture, storage, access, and reporting—along with clear accountability for maintaining these systems.

- **We built a culture of impartial data analysis**: By establishing dedicated analysts and leaders to oversee data integrity, we ensured that our business decisions were driven by objective, consistent information—not wishful thinking.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>

- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>

- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>

- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Define

In this section, we explain how we at Amazon adopted the **DMAIC process**—a structured Six Sigma methodology—to bring clarity and control to our increasingly complex operations. This approach helped us stay focused on what drives business performance, even as we scaled globally and diversified our offerings.

### Key Takeaways:

- **We turned to Six Sigma to manage complexity**: As Amazon grew from a book retailer into a multi-product, international company, we needed a structured way to measure, monitor, and improve our processes. Six Sigma’s DMAIC framework offered exactly that.
- **DMAIC became our operational foundation**: The five steps—**Define, Measure, Analyze, Improve, Control**—allowed us to break down complex systems into manageable parts, identify input metrics, and drive improvements with precision.
- **We embraced Donald Wheeler’s philosophy**: Following his principle that understanding variation is key to improvement, we focused on how changing inputs affects outputs, aligning our culture around continuous improvement.
- **We built clarity into invisible systems**: Since much of Amazon’s operations (software, fulfillment, logistics) were either virtual or distant, we created standard definitions and measurements to make the system’s performance tangible and actionable.
- **We aligned every part of our system to inputs**: From product selection and pricing to software reliability and delivery speed, we learned that managing the inputs across all departments ultimately shaped our outputs like revenue and customer satisfaction.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Intro To Input Metrics

In this lesson, we explore what input metrics are, why they matter, and how shifting our focus from outputs to controllable inputs transformed the way we operated at Amazon. We learned—sometimes the hard way—that sustainable success comes not from chasing end results, but from rigorously managing the elements we directly control.

### Key Takeaways:

- **We learned to distinguish between input and output metrics**: Outputs like revenue or share price are lagging indicators and largely outside our direct control. In contrast, input metrics—customer-facing and operational—are controllable levers that drive those outputs.
- **We saw the importance of focusing on controllable inputs**: By investing resources in areas like software, marketing, operations, or process improvements, we influenced outcomes in a predictable and sustainable way. Examples include inventory record accuracy and marketing spend.
- **We realized the limitations of reactive output chasing**: Trying to salvage quarterly financials with last-minute promotions often distracted us from what truly mattered—improving the customer experience.
- **We began managing our business as a system of processes**: We stopped reacting to fire drills and started proactively measuring and optimizing the activities (inputs) that lead to better long-term results.
- **We gained a competitive edge through disciplined input tracking**: While most teams naturally measure outputs, few rigorously track inputs. By doing so, we developed a significant operational advantage that helped Amazon scale sustainably.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Introduction To The Course

Welcome to our course on Input Metrics, grounded in our experience at Amazon. Our goal is to help you understand and apply the principles that drive consistent business success. This course is designed to transform how we manage performance—shifting from relying solely on output goals to mastering the inputs that lead to great outcomes.

### Key Takeaways:

- **Course Goals**: We’ll learn how to deeply understand input metrics, identify the right ones for our business, measure and analyze them, conduct effective Weekly Business Reviews (WBRs), and implement the WBR reporting tool to visualize data clearly.
- **Real-World Application**: We share Amazon-proven strategies and offer downloadable templates, documents, and tools to support real-world execution of these practices.
- **Amazon’s Shift in Strategy**: We recount Amazon’s turning point in 2000—moving from output-focused goals to a system of input-driven process control, which delivered lasting results.
- **Input vs. Output Mindset**: Instead of telling teams to just hit output targets, we equip them to understand and improve the specific inputs that lead to success—driving better outcomes like increased sales and customer satisfaction.
- **Why It Matters**: Studying input metrics gives us greater control over operations and helps us empathize with customer pain points—fueling continuous improvement and innovation.

To dive deeper into Input Metrics Mastery and learn more from Bill and Colin, follow these links:

- Take the full course at their website: <https://workingbackwards.com/>
- Buy the book: <https://www.amazon.com/Working-Backwards-Insights-Stories-Secrets/dp/1250267595>
- Bill's LinkedIn: <https://www.linkedin.com/in/bill-carr/>
- Colin's LinkedIn: <https://www.linkedin.com/in/colinbryar/>

Master the Art of Receiving Quality Feedback (from an ex-Meta Staff Researcher)

In this time of uncertainty in the workplace and job market, what’s one skill you can develop that would **yield disproportionate rewards**?

It’s hard to pinpoint *the one* skill, but I believe the **ability to ask for, receive, and iterate on feedback** is incredibly important. Adaptability is what helps us not just survive, but thrive—and adaptability begins with how well we take in and work with feedback.

When you learn to process feedback effectively, **you develop a kind of likability and professional superpower** that organizations and leaders find hard to ignore.

We hear feedback all the time, so what exactly are we trying to get better at? Most of the feedback we receive is vague and hard to act on. Think about examples like “improve your communication” or “increase your coding speed”—where do you even start? The **key is in asking for feedback** proactively, clarifying and chunking it down into something actionable, working on it, and then circling back to check if you’ve understood and implemented it well.

In this talk, we’ll learn how to do exactly that—a process I call “**exhausting the feedback process**,” a phrase I often use in my coaching practice. We’ll also explore how to handle feedback that’s just plain off the mark (spoiler: it’s rarely useful to say that outright to the person giving it).

Yogi on LinkedIn: <https://www.linkedin.com/in/yogi1/>

Tech Career Growth Videos

Our instructional videos provide software engineers with insights needed for career growth. Masterclasses go incredibly in-depth on a specific tech topic while case studies are real examples from established engineering leaders.