Paper Reading: Llama 2: Open Foundation and Fine-Tuned Chat Models

Session led by [Mike A](https://www.linkedin.com/in/malik-mike-a-32b63818b/)

Join us for the second installment of our three-part paper reading series on the LLaMA series of Large Language Models (LLMs).

In this session, we will dive into **Llama 2**, the second iteration in the LLaMA family of models, introduced by Meta AI in mid-2023. This follow-up to the original LLaMA models brings significant advancements, offering foundational and fine-tuned versions with parameter counts ranging from 7B to 70B. Llama 2 models set new benchmarks in helpfulness and safety, showcasing impressive capabilities in dialogue-based tasks and outperforming many open-source alternatives.

We will explore the innovations behind the **Llama 2** models, focusing on their **pretraining methodology**, the **fine-tuning process** that incorporates Reinforcement Learning with Human Feedback (RLHF), and the novel techniques Meta AI used to enhance model safety and alignment. Particular emphasis will be placed on the **Llama 2-Chat models**, optimized for multi-turn conversations, along with references to their performance on real-world safety benchmarks.

This session will be an insightful discussion of how the Llama 2 models have helped to close the gap between open-source and closed-source systems like ChatGPT, examining their architecture, training data, and the steps taken to ensure responsible release and deployment. We'll highlight key takeaways from Meta AI's comprehensive evaluation of these models and discuss how they are still reshaping the landscape of open-source AI.

Whether you're a researcher, developer, or AI enthusiast, this paper reading will offer you an in-depth understanding of the **Llama 2** models and their contributions to the ongoing development of LLMs. This is a unique opportunity to review cutting-edge advancements in **LLM safety**, **dialogue optimization**, and their impact on the future of open-source AI systems.

Mark your calendars for an informative hour of "Llama 2: Open Foundation and Fine-Tuned Chat Models," as we continue to chart the progress of AI language models and explore the innovations that continue to shape the future of the field!

[https://lmarena.ai/\
https://github.com/meta-llama/llama](https://lmarena.ai/￼https://github.com/meta-llama/llama)

STAR

If you want direct interview support from Alex and Uzair, take their live Maven course with discount code **TARO10**: <https://maven.com/alex-kliotzkin/acing-em-interviews>

Engineering manager interviews focus heavily on **behavioral assessments**, and using a structured response framework like **STAR** can help clearly demonstrate leadership competencies and impact. STAR stands for **Situation, Task, Action, and Result** and is widely recommended for leadership roles due to its clarity and depth.

- **Situation**: Set the context of your story—when, where, and why it happened—so the interviewer understands the environment and background.
- **Task**: Define your responsibility in that situation and any challenges you faced while addressing it.
- **Action**: Detail the specific steps *you personally* took, what you owned, and how you worked with others.
- **Result**: End with measurable outcomes like cost savings, quality improvements, time to market, or customer/team impact. Show the success and its scale.

Why Don't More Big Tech Engineers Contribute To Open Source?

The modern software industry runs on open source software. Should more software engineers (especially those in Big Tech) contribute back to open source?

The Mistake Engineers Make During The Reverse Interview

Too many engineers don't think critcally about the impact of the last 5-10 minutes of an interview.

Business Dynamics

If you want direct interview support from Alex and Uzair, take their live Maven course with discount code **TARO10**: <https://maven.com/alex-kliotzkin/acing-em-interviews>

Many engineers struggle with engineering manager (EM) interviews because they underestimate how different they are from individual contributor (IC) roles. This course aims to prepare candidates for the unique expectations, mindset, and evaluation criteria involved in EM interviews.

- **Shift in Role and Expectations**: Transitioning from IC to EM is not just a promotion but a full career change, requiring a shift from deep technical implementation to broad people and strategy responsibilities.
- **Interview Criteria Differences**: For EM roles, **80% of the evaluation is behavioral** (trust, leadership, communication) and only 20% is technical (system design, operational excellence), the reverse of IC interviews.
- **Understanding the Hiring Manager's World**: EM interviewers, typically directors or above, prioritize reliability, trust, emotional maturity, and ownership. Candidates must show they can simplify complexity and manage uncertainty.
- **Research and Business Alignment**: Successful candidates understand the business context of the role (e.g., quality for physical products, speed for startups, security for regulated industries) and tailor their answers accordingly.
- **Empowerment Over Expertise**: EMs don’t need to be the most technical on the team. Their primary responsibility is to ensure the team delivers by empowering others and focusing on strategic growth and impact.

Tailoring Your Answers

If you want direct interview support from Alex and Uzair, take their live Maven course with discount code **TARO10**: <https://maven.com/alex-kliotzkin/acing-em-interviews>

To deliver unforgettable interview answers, we need to learn how to tailor our responses to match the company, the role, and the specific audience we’re speaking to. This session explains how to adapt our stories based on context, organization type, and our own leadership philosophy—even when we haven’t faced a specific scenario before.

- **Tailor Stories to the Team and Business**
  - We must deeply understand the hiring team and the company’s business context to highlight the most relevant parts of our story.
  - Even small adjustments—altering 10% of a story—can have a big impact in aligning with the company’s values and needs.
- **Adapt Answers Based on Company Type**
  - For example, when answering a question about influencing stakeholders, we need to highlight different dynamics depending on whether it’s a startup (external stakeholders like customers or investors) or a big org (internal cross-functional teams).
  - Similarly, questions about balancing delivery vs. quality should be approached differently for consulting roles (focus on fast delivery and milestones) vs. product roles (focus on long-term impact, maintainability, and operational excellence).
- **Responding to Questions You Haven’t Faced Before**
  - If we haven’t directly experienced a situation (e.g., managing underperformers or doing capacity planning), we should draw on adjacent stories and clearly articulate our leadership philosophy and approach.
  - It’s helpful to describe what we’d do in the first 90 days on the job—such as building a plan or upskilling—and how that reflects our values and learning mindset.
- **Use Leadership Philosophy as a Guide**
  - Our answers should reflect consistent values, such as building high-performing teams, holding people accountable, and prioritizing quality.
  - Even without a direct experience, framing our response through this philosophy helps convey credibility, ownership, and potential.

How Do I Know If I'm Making Progress With Leetcode?

It's demoralizing to get stuck on similar problems repeatedly. How do you know if you're making progress?

Course: <https://www.jointaro.com/course/crash-course-data-structures-and-algorithms-concepts/>

Verify Your Severance Package

We explain what to do immediately after a layoff to protect yourself financially and legally, and to reduce unnecessary stress. Here are the core points from the lesson:

- Most large tech companies provide severance due to legal requirements and long-term reputational incentives, but layoffs are messy, so you must carefully review all paperwork and verify dates, vesting, and severance amounts.
- Never sign termination documents on the spot; take time to review them, ask questions, and involve a lawyer if possible, especially for severance disputes or visa-related considerations.
- After a layoff, it’s critical to understand your personal burn rate and runway by reviewing your finances, since clarity around how long you can sustain yourself reduces anxiety and helps you plan your next steps.

The Plan To Make Taro Q&A Suck Less - 9/13/2022

- **Update:** Literally a few hours after recording this video, the ability to like answers on Taro web was shipped.
- The #1 priority with the Q&A platform is adding notifications. Here are some important ones:
  - Being notified when someone answers a question that you have asked
  - Being notified when someone likes an answer that you have shared, as you should be able to feel good if you have shared your wisdom in a high-quality answer
- Overall strategy: Make the process of asking a question and getting answers delightful and silky smooth, as you can't have answers without questions. From there, make it a great experience and rewarding to answer questions.
- Ideas:
  - The ability to write a private "Thank You" note to someone who left a really great answer on your question.
  - Question askers can choose and highlight an answer on each of their questions, which helped them the most. This is another way to let question answerers know the impact they're having.

Taro Book Club: The Pragmatic Programmer - Session 6

Session Led by [Bikram Kashyap](https://www.linkedin.com/in/kashyapbikram/)

In this #taro-bookclub discussion, we'll go over [**"The Pragmatic Programmer"**](https://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/dp/020161622X) by Andrew Hund & David Thomas

Here's the agenda:

- Design by Contract
- Dead Programs Tell No Lies
- Assertive Programming
- How to Balance Resources
- Don't Outrun Your Headlights

Please send your questions ahead of time to [**Bikram Kashyap**](https://www.linkedin.com/in/kashyapbikram/) in the [**Taro Premium**](https://www.jointaro.com/membership/) Slack, so that they can be included in the discussion. *If you don't have the book or want more details on how this all works, check out the #taro-bookclub channel in the Taro Premium Slack as well!*

Get ready for another engaging and informative session!

Why There Will Be Fewer Product Managers In Tech

The trend is clear: engineers will have increasingly more power in the workplace, and the number of PMs needed for 'alignment' will diminish.

Build Support Groups

We explain why building community after a layoff can dramatically improve both emotional resilience and job-search outcomes. Here are the core points from the lesson:

- Layoffs create a natural sense of solidarity, and bringing laid-off peers together helps transform an isolating experience into a shared support system.
- Being proactive about forming or joining support groups—through spreadsheets, LinkedIn, or communities like Taro—creates powerful networks of talented people who can help one another.
- Regular check-ins, shared encouragement, and activities like mock interviews provide structure, accountability, and momentum during the job search.

Machine Learning System Design Ask & Learn

**Event description**

Join us remotely for an interactive Q&A with Ilya Reznik—Founder of MLEPath, veteran ML engineer with nearly 15 years at Meta, Twitter, and Adobe, and interviewer of hundreds of real-world ML system design (MLSD) interviews (plus nearly 1,000 mock sessions in the last two years). Whether you’re tackling your first MLSD interview or leveling up for senior roles.

- **Core frameworks & approaches**: how to structure your end-to-end design, balance scalability, latency, cost, and reliability
- **Key components & trade-offs**: data pipelines, model serving, feature stores, monitoring, and A/B testing
- **Real-world insights**: pitfalls Ilya’s seen candidates fall into—and what top performers do differently
- **Live feedback**: get tailored advice on designing everything from recommendation systems to real-time anomaly detection

Perfect for ML engineers aiming to land roles at FAANG, startups, or anywhere that demands rock-solid system design chops. Come ready with your toughest questions—and leave with a clear, actionable roadmap for your next MLSD interview.

Homework: Define Your Engineering Management System

If you want direct interview support from Alex and Uzair, take their live Maven course with discount code **TARO10**: <https://maven.com/alex-kliotzkin/acing-em-interviews>

To answer engineering manager (EM) behavioral interview questions effectively, it’s essential to clearly articulate your **leadership style and decision-making frameworks**. Interviewers expect thoughtful, structured responses that show how you operate as a leader across the core responsibilities of the role.

- **Three Core EM Responsibilities**: Every EM must deliver results, lead the team, and manage stakeholders. You must show systems and habits that support all three.
- **Delivery Systems**: Explain how you ensure *sustainable delivery*—using tools like KPIs, OKRs, sprint metrics, dashboards, and demolocation strategies.
- **Team Leadership**: Be ready to share your approach to building *high-performance cultures*, setting team structures, conducting 1:1s, growing individuals, and setting vision and expectations.
- **Stakeholder Management**: Detail how you *influence and align* stakeholders—through communication systems, relationship building, regular reporting, and rituals like syncs and buy-ins.

Writing down these frameworks helps reinforce your leadership identity and provides a solid foundation for compelling interview storytelling. Let me know if you’d like help turning this into a worksheet or practice guide.

Why Big Tech Companies Do Open Source (From A Director at Meta)

Companies like Meta, Google, Microsoft, and Hugging Face have made enormous contributions to open source. These companies often allocate tens or hundreds of millions of dollars to creating or maintaining open source software. Why? 

- These companies often rely on the software for their production code
- Companies can use popular open-source projects as a tool to recruit top engineers
- Engineers familiar with a company's software will onboard faster

Paper Reading: The Future of Large Language Model Pre-training is Federated

Session led by [Mike A](https://www.linkedin.com/in/malik-mike-a-32b63818b/)

Is the future of model pre-training decentralized?

Join us for a captivating session as we explore the cutting-edge breakthrough of federated learning in language model pre-training, featuring an in-depth discussion of the "The Future of Large Language Model Pre-training is Federated" paper!

This paper reading event will shed light on how federated learning (FL) could be set to revolutionize the training of large language models (LLMs). Discover how FL could mobilize vast, underutilized data and computational resources from across the globe, enabling large-scale collaboration and democratizing the pre-training of LLMs. Learn about the innovative approach that allows organizations, and hopefully, soon individuals, to collectively train LLMs using limited resources while matching or even surpassing the performance of traditional centralized models!

This event is a must-attend for AI researchers, software developers, data scientists, and tech enthusiasts eager to grasp the future of LLM pre-training and its implications for AI advancements. Whether you're interested in scaling AI models, leveraging federated learning for collaborative training, or simply curious about the next big thing in AI technology, this paper reading offers invaluable insights and discussions on some of the latest AI research.

Mark your calendars and join us for an exciting hour of "Federated Learning: The Future of Large Language Model Pre-training" exploration!\
\
**Reference Links:**\
[https://flower.ai/\
https://github.com/adap/flower](https://flower.ai/￼https://github.com/adap/flower)

Team Dynamics

If you want direct interview support from Alex and Uzair, take their live Maven course with discount code **TARO10**: <https://maven.com/alex-kliotzkin/acing-em-interviews>

To stand out in EM interviews, it’s not enough to understand the company—you also need to empathize with the **team’s specific challenges**. By researching and anticipating what the team needs, you can tailor your answers to resonate with the hiring manager and demonstrate high potential fit.

- **Put Yourself in the Hiring Manager’s Shoes**: Use empathy to analyze what the team is responsible for and what challenges might be preventing sustainable delivery.
- **Anticipate Common Team Challenges**: These may include stakeholder alignment in large orgs, speed in startups, product robustness, scaling issues, or even interpersonal conflicts like team infighting.
- **Showcase Relevant Experience**: Prepare stories from your own background that reflect how you’ve solved similar problems or would approach the challenges facing this specific team.
- **Turn the Interview into a Conversation**: Ask insightful questions to learn more about the team’s current pain points. This shows curiosity, initiative, and models what working with you would be like.

Why You Should Be Racist Toward European VCs (An Argument For YC)

James lists his location as San Francisco Bay Area simply to avoid the horrible European VCs (Venture Capitalists).

Competitive Pay

Big Tech companies are widely known for offering extremely competitive compensation packages, which is often one of the primary reasons engineers aim to work at these firms. This segment outlines both personal anecdotes and structural strategies that explain why compensation in Big Tech is so high.

- **Top-Tier Compensation Strategy**: Meta’s compensation goal, for example, is to pay in the 90th percentile—more than 90% of other tech companies—making total compensation significantly higher for engineers who make the transition into Big Tech.

- **Drastic Pay Increases**: It's common for engineers switching from smaller or non-FANG companies to experience dramatic increases in pay—often as much as 50–100%. Even the speaker was surprised by the magnitude of the raise upon joining Meta.

- **High Cost of Living Justification**: These lucrative salaries help offset the cost of living in expensive tech hubs like Silicon Valley, New York City, and Seattle, where basic housing or property ownership can be out of reach without this level of income.

- **Historical Offers Data**: In 2014, new grad offers ranged from $164K to $270K annually from Big Tech firms—Google, Facebook, and others. Salaries have only increased since then, due to ongoing talent wars and inflation.

- **Prestige + Pay Combo**: Many engineers underestimate the financial upside of Big Tech and initially assume a 20–30% increase is generous. In reality, the combination of prestige and substantial pay is a powerful draw.

Lessons From Crossing $10M ARR (Entirely Inbound)

How did James achieve viral growth as a product analytics company? How does the open-core model of the company help them grow? How deliberate were they with growth?

Tech Career Growth Videos

Our instructional videos provide software engineers with insights needed for career growth. Masterclasses go incredibly in-depth on a specific tech topic while case studies are real examples from established engineering leaders.