Paper Reading - Gemini: A Family of Highly Capable Multimodal Models

Paper Discussion led by: [Akshika Wijesundara PhD](https://www.linkedin.com/in/akshika/)

Abstract: Google researchers recently unveiled Gemini, a ground breaking family of multimodal AI models capable of understanding and reasoning across images, audio, video, and text. The largest Gemini model, Ultra, achieved human-expert performance on several benchmarks and advanced the state of the art across a range of multimodal tasks.

In this paper reading, we will discuss key details from Google's paper introducing Gemini models. Topics will include:

- The Gemini model architectures and how they achieve cross-modal understanding
- Performance benchmarks and state-of-the-art results across text, image, audio, and video tasks
- Applications and use cases enabled by Gemini's multimodal reasoning
- Approaches to responsible deployment of large language models like Gemini

This paper provides a comprehensive overview of the exciting new capabilities of Google's Gemini project. Whether you're an AI researcher, engineer, or simply curious about the state of the art in multimodal AI, join us for an in-depth look at this new model family.\
\
Paper link: [**https://arxiv.org/abs/2312.11805**](https://arxiv.org/abs/2312.11805)\
Event link:  **[https://www.jointaro.com/event/paper-reading-gemini-a-family-of-highly-capable-multimodal-models/](https://www.jointaro.com/event/paper-reading-gemini-a-family-of-highly-capable-multimodal-models/)**

Behind the scenes of a $320K → $1.05M negotiation in Silicon Valley

[Yogi Sharma](https://www.linkedin.com/in/yogi1/), an early employee at Facebook, shares an epic story about negotiating a massive compensation increase from his offer at Uber while discovering his true worth in the process.

---

I am a mathematician and engineer at heart, and I hate negotiations. I want people to evaluate the value of my work and pay me accordingly.

That is the mindset I had before this negotiation with a big ride-sharing company that opened my mind to the possibilities of a different way of operating.

In this candid behind-the-scenes conversation, I will discuss the mindset, preparation, and tactics that went into a negotiation (for the role of a SWE) that started at $320k and ended at $1.05M.

Even harder than doing such negotiation was the decision to decline such a lucrative job offer. (Thanks to that decision, I ended up finding my current profession.) Towards the end, I will also share my rationale behind such a seemingly crazy decision.

If you feel you are doing the work needed for a promotion or raise, but don’t seem to get it, this talk is perfect for you. I used to be that person and struggled to get the rewards I thought I deserved. This negotiation changed me deeply, and I hope to share a few insights that would help you too.

Connect with Yogi Sharma: 

- LinkedIn: <https://www.linkedin.com/in/yogi1/> 
- Taro: <https://www.jointaro.com/p/yogi-sharma/>

Handling a PIP: The Ultimate Guide To Navigating A PIP

There are two options after a PIP: either the employee will remove all the doubts about the performance concern, or the employee will be terminated.

Your job as an employee is to judge your manager’s intentions. Your manager is the most important person in the PIP, and more generally, in your career. Many managers view the PIP as a formality before termination, and they’ve already given up on you. In that case, **you need to leave.** You don’t win any bonus points by "proving them wrong."

If the manager is responsive to your questions, provides constant feedback, and has a reasonable PIP plan, you may have a chance to succeed.

However, in the vast majority of cases, our recommendation is that you **must start looking for a** **new job.**

The biggest reason a PIP is not worth fighting is because **you want to work for people who believe in you**. This is critical for career success: you need someone who is unfailingly supportive of you, whether that's a friend, mentor, or manager. A PIP does not feel like support. In fact, it feels like the opposite. Your manager felt your performance was weak enough that they literally spent hours documenting how you fell behind, and then informed you in a legal manner.

The joke is that PIP should stand for “Paid Interview Period”, and that is honestly our recommendation for many tech workers who receive a PIP.

Here is the **PIP Evaluation Framework** in order to make the best decision once you’ve received the PIP. If 1+ of the following are true, you should **not** plan to have a long-term future at your current company.

- Your manager is uncommunicative or closed off
- The time horizon for the assigned work on the PIP is unrealistic (e.g. 1 week for a launch)
- The goalposts for the projects keep shifting
- The PIP work and your “normal duties” result in an unmanageable workload
- Your team is unsupportive

Conclusion: The Ultimate Guide To Navigating A PIP

Thanks for being part of this course! Here are the main takeaways:

- A PIP **does not** mean your career is over
- Make it easy to receive feedback
- You should (probably) start looking for a new job
- Don’t over-share information
- Discussions:
  - [I have been put in PIP. What should I do?](https://www.jointaro.com/question/i2RqeCiAQJ04eU1rp0eQ/i-have-been-put-in-pip-what-should-i-do/)

  - [Bouncing back after termination](https://www.jointaro.com/question/iDARmEmqYKlCIhzKIMNh/bouncing-back-after-termination-what-can-i-do-to-move-forward/)

  - [Advice on PIP with unsupportive team?](https://www.jointaro.com/question/93GBTJ73ayNLPY0sLuHz/advice-on-pip-with-unsupportive-team/)

PIPs and Interviews: The Ultimate Guide To Navigating A PIP

The PIP is as big a deal as you make it out to be. This is the key insight as you interview at other companies: **Don’t share information** **that hurts you.** 

In the same way that you wouldn’t share your previous salary or your compensation expeditions in the interview, you also shouldn’t share past performance struggles. You don’t get points for transparency. The thinking should be that you only share information if it helps you with the job interview. In particular, you should *never mention that you got PIP-ed*.

Instead, use neutral phrasing like “*I’m super excited about this company. I deeply resonate with the mission and people.*” Or *“The previous role wasn’t a good fit”*

Life After PIP: The Ultimate Guide To Navigating A PIP

One of the biggest concerns for anyone who has received a PIP is about how it may effect their future job prospects. The answer is, no, a **PIP does not need to follow you around**. You can (and will) find a fresh start and a thriving career if you want it. 

Remember, a PIP is about **both** employee and employer. When you change jobs, your employer is changing, so the situation leading to the PIP is very different.

As you reflect on your next role, it’s worthwhile to allocate some time to reflect: 

- Collect feedback from others: at the minimum, save a copy of your PIP and figure out what feedback there is transferable to another job.
- Collect feedback from yourself: Write down your reflections: what did you enjoy (or not) about your past work experience. What types of projects and people were fun for you?

Meta Manager Case Study: The Ultimate Guide To Navigating A PIP

This was an Engineering Manager at Meta who has put 20+ employees on PIPs throughout their career.

- PIPs come from a mismatch of a “square peg, round hole”

- PIPs should never be a surprise. PIPs get discussed with the skip manager and HR

- Designed to be achievable with effort

- Across 20+ PIPs, has never had anyone continue employment

- **Advice**: “Clarity is more important than agreement”

MongoDB Manager Case Study: The Ultimate Guide To Navigating A PIP

This was a manager at MongoDB who shared a specific story of putting one of their employees on a PIP.

- PIP delivered to people who needed a lot of handholding

  - “Problem presenters” instead of “problem solvers”

- PIPs are hard but not impossible. Goal is to help the employee overcome the situation and contribute to the health of the team.

- **Advice**: “Understand every single word in the PIP”

Meta Case Study: The Ultimate Guide To Navigating A PIP

This was a veteran engineer with more than 15 years of experience at other large tech companies before joining Meta.

- **Before**: Hired in as Staff Engineer \[**E6**\], brought in as a domain expert. Worked for almost 2 years before the PIP

- **During**: Not surprised based on rating trends. Main issue was code productivity, inability to use tooling such as version control

- **After**: Struggled with the PIP for a few months, then negotiated a severance package

Joining a company at too high a level is not good

If an employee's performance is subpar, it's better to act on that early, both for the employee and the team.

At a high level, your job is to lead the team and teach others. It’s ok if you don’t know the tooling, but you should not let that get in the way of landing code. Code output was poor.

Pinterest Case Study: The Ultimate Guide To Navigating A PIP

This was a non-engineer who had several years of experience before joining Pinterest.

- **Before**: 90 days after joining, was told they needed to improve

- **During**: “Completely blindsided” when placed on PIP at the 4 month mark. Goalposts kept shifting, didn’t receive feedback until 4 weeks later

- **After**: Took the termination after taking leave for interview prep

MassMutual Case Study: The Ultimate Guide To Navigating A PIP

This was a new grad engineer who joined MassMutual, a 100+ year insurance company in Massachusetts.

- **Before**: First job out of college, shared a project with another new grad engineer which lacked support

- **During**: Shocked to receive it (did not know what a PIP was). Told not to ask for help to test his velocity

- **After**: Chose to leave the company, since the manager lost faith

Amazon Case Study: The Ultimate Guide To Navigating A PIP

Case studies are valuable to (1) realize that you are not alone and (2) learn from the experiences of others and see what you can apply.

We’ll start with a new grad Amazon engineer who joined during the pandemic.

- **Before**: First job out of college, was on Focus after 6 months
- **During**: Shocked when they found out (URA quota suspected). Manager became more “HR-like”, less candid
- **After**: Went through the PIP and failed, even though they showed improvement

Quitting a PIP: The Ultimate Guide To Navigating A PIP

You got the PIP, and you made the difficult (or maybe not-so-difficult) decision to quit. You’ll soon be off to greener pastures, but there is a strategy around leaving the company that is critical to get right.

Once you've decided to leave your company, your priority should shift to optimizing your professional and financial prospects. Most companies will provide a severance package to employees who are laid off or terminated. A severance package is the *compensation that an employee receives when they leave a company*.

While companies may care (at least a tiny bit) about the well-being of their employees, generosity is **not** the primary reason that businesses provide severance. In exchange for severance, companies will require you to sign certain documents to protect the business:

- **Non-disparagement** so you agree not to speak poorly about the company in public. The company protects their business interests by maintaining relationships and reputation.
- **Invention assignment** so there is no theft of intellectual property (IP) or source code.
- **Other agreements** such as not suing the company or not poaching employees for a set amount of time.

The Human Resources (HR) team will want your agreement on these terms, which means **you have leverage**. The asks from these companies are not too onerous, which means it doesn't cost much to agree -- very few employees end up suing their former employer anyway.

### Severance checklist

Here is the "severance checklist", the most common dimensions on which you can negotiate:

- Salary

- Health insurance (COBRA payments)

- Last day of employment

- Equity vesting acceleration

- Outplacement services

Remember, **you only get what you ask for**. Since you're on your way out anyway, it's fine to get rejected when asking for more. If you express your concern and headache from the termination ("*I just moved my family here from out of state*"), companies are often willing to meet you halfway for things like:

- Your last day of employment
- Extending and paying for health insurance for several additional months
- Accelerating the next vesting date by a few weeks

Finally, we present a "goodbye checklist":

- Contact info for relevant colleagues

- How will this be communicated to the team?

- What will your employer disclose to a future employer?

- What’s the “cooldown period”?

**Relevant links**:

- [The Key Insight Behind Negotiation](https://www.jointaro.com/lesson/YliP9kVbd1zym9kmv9Gg/the-key-insight-behind-negotiation/)
- [\[Masterclass\] Understanding And Optimizing Your Pay In Tech](https://www.jointaro.com/lesson/gk408yBIV2fa5RjVR213/masterclass-understanding-and-optimizing-your-pay-in-tech/)
- [Amazon VP talks about negotiating a severance package](https://www.linkedin.com/feed/update/urn:li:activity:7161390728533741568/)

**Discussions**:

- [Does severance package get reduced in subsequent rounds of layoffs?](https://www.jointaro.com/question/FIFJWOVz2ILICZ0dOLqy/does-severance-package-get-reduced-in-subsequent-rounds-of-layoffs/)
- [Severance in companies tied to tenure?](https://www.jointaro.com/question/9nzrgmga50Dn0UEIH6Ad/severance-in-companies-tied-to-tenure/?revalidate=true)

Reactions After A PIP: The Ultimate Guide To Navigating A PIP

If you received a PIP, be ok with a flood of emotion. Anger, sadness, and humiliation are normal, and you should give yourself at **least half a day** simply to process. Don’t take any action erratically. 

Once you’ve had time to process, confide in a mentor or trusted friend. Online communities like Taro can help you feel less lonely. You should **not** over-share this information with your colleagues. There is a negative bias associated with a PIP, so sharing the news with people who don’t truly care about you may end up backfiring. 

Do ✅:

- Confide in a mentor or trusted friend
- Find a community (like Taro!)

Don't ❌:

- Act from a place of emotional instability
- Over-share with colleagues

A PIP is not entirely your fault, but it's also not entirely your company's fault. Do some honest reflection about what has been working and what has not.

Some recipients of a PIP are (understandably) outraged and want to take take legal action. In theory, you could sue the company, but in practice is almost never recommended. Given that employment is “at-will”, you’ll need ot clear a very high bar to show that you were unlawfully targeted to be let go or put on a PIP. In addition, to collect evidence of unfair treatment which could take dozens of hours, there’s also a high cost associated with finding legal assistance. Ou recommendation is to focus your energy on the next career move, where you can work with people that value you, rather than focusing on an unfortunate job that will soon be in your past.

How To Avoid PIPs: The Ultimate Guide To Navigating A PIP

**Prevention is the best cure**, so let’s talk about what steps you should take to avoid a PIP. The #1 thing you can do is get high-quality feedback from the people you work with. The issue is that most tech workers are horrible at asking for feedback in a way that actually gets them something substantive.

Your guiding principle should be to **make it easy to receive feedback, especially constructive feedback.**

Here’s a system you should employ to get high-quality feedback: 

- Schedule dedicated “feedback 1:1s” with key people: manager, XFN partners
- Write down what they say
- Follow up: short-term and long-term

A PIP should **never** be a surprise, and yet it often Is. This system will prevent you from being negatively surprised. With a manager, you ask how you’re trending and discuss a hypothetical rating, broken down by the axes of evaluation at your company.

We also cover two examples of proactively addressing concerns with your manager.

Are you a PIP risk? The Ultimate Guide To Navigating A PIP

The most obvious way to identify if you’re at risk for a PIP is if you’ve received poor ratings in prior performance review cycles. But this is a **lagging indicator**, so it may be too late to do anything at this point. Ideally, we’d like a **leading indicator** so you can take steps to avoid a PIP by discovering the warning signs early enough. 

The best leading indicators are: 

- Check-in with your peers and managers and get feedback
- Compare markers of progress: code output, design docs, and updates on tickets of projects. 

On the topics of ratings, you should understand the breakdown of what percent of employees get each rating. Companies may not reveal this info directly, so you can ask your manager or look up internal documentation.

If your manager and teammates start ignoring you or removing your scope, that’s an unhealthy sign.

PIP In Big Tech -- Amazon, Meta, Google

In this section, we cover the path for employees to enter a PIP for three Big Tech companies — Amazon, Meta, and Google.

At **Amazon**, the poor performance rating is called **Needs Improvement (NI)**. Poor performers are put on a Focus, which is a development plan to get back on track. Think of it like a warning. If the focus does not do well, the employee is put on a Pivot, at which point the options are to leave the company or accept a PIP. Learn more here: <https://www.jointaro.com/topic/amazon/performance-review/>

At **Meta**, the poor performance rating is called **Meets Some (MS)** or **Meets All (MA)** Expectations. If the HRBP (Human Resources Business Partner) is part of your review meeting, a PIP is likely. One unique policy of Meta is something called "up or out." This requires engineers to get promoted every few years until hitting Senior Engineer \[E5\]. Learn more at <https://www.jointaro.com/topic/meta/performance-review/>

At **Google**, the poor performance rating is called **Not Enough Impact (NI)** or **Moderate Impact (MI).** A "Support check-in" indicates poor performance and candidate for a PIP. Google is generally more laid-back when it comes to performance compared to other companies. Learn more here: <https://www.jointaro.com/topic/google/performance-review/>

Why Companies Do PIPs: The Ultimate Guide To Navigating A PIP

A PIP can seem cruel, but there’s a clear reason that companies use them year after year. Let’s start with the top priority for a manager: ensure the optimal performance of the team. For a manager, getting their reports promoted or singing their praises is important, but only insofar as it’s in service of the optimal performance of the team.

Another detail to realize is that the high performers on any team do an outsized amount of work. This is especially true in jobs that have tremendous leverage for the most skilled workers, such as software engineering (thus the famed “10x engineer”). When a company is large enough, a “bell curve” of performance will emerge. A PIP is focused on the employees who are deemed as low performers, typically the bottom 10% of the bell curve.

So, returning back to the original question: the reason why companies do PIPs is to **ensure optimal productivity and impact** for the overall employee base. If high performers sense that poor performance is not being addressed, they may leave the team. So a PIP is a way to push poor performers off the team.

The company also wants to push people out in a way that **minimizes the chance of a lawsuit**. Even though employment in the US is at-will, there’s still a chance of getting sued if the employee argues that they were fired for an illegal reason (e.g. due to a protected characteristic).

The company or manager may claim that the PIP is designed to help you, but that’s almost never the case. They’re more concerned about minimizing the negative impact on the broader team.

Finally, PIPs are often “weaponized” to fire people under the guise of performance when the company doesn’t want to call it a layoff. This is especially relevant now in the current economy as companies may enter an age of austerity.

PIP Stereotypes - The Ultimate Guide To Navigating A PIP

Given the secrecy and consequences of PIPs, you've probably a bunch of stereotypes about PIPs. This is a non-exhaustive list of beliefs that are simply not true:

- *"You must be incompetent or lazy"*
- *"Your manager or company must be horrible"*
- *"It’s impossible to keep your job on a PIP"*
- *"You'll no longer be employable"*

This thinking is reductive and will lead to more harm than good. A better way to think about a PIP is not to cast blame on one party or another. Instead, a PIP means **there was a mismatch in expectation between employee and employer**. 

The reality is that there are many potential reasons why you may receive a PIP, ranging from lack of support, poor management, lack of motivation, shifting company needs, etc.

Throughout the PIP process, one of the most difficult parts is that you’ll feel zero (or negative) support from key people around you. In order to find success at your job, and more broadly in your career, **you need to be an advocate for yourself**. Have the confidence that you are valuable. Too many engineers are their own worst critic, which exacerbates the negative spiral of performance.

What is a PIP? The Ultimate Guide To Navigating A PIP

A PIP is a **Performance Improvement Plan**. A PIP covers specific areas of performance deficiencies and sets clear expectations for what needs to happen in the future to remain employed. Think about a PIP as an ultimatum, essentially telling the employee that "Something needs to change." The employee has been deemed (whether fairly or unfairly, as we’ll see) to be missing their targets.

The PIP will outline a plan to improve the employee’s performance, along with milestones to hit over 1-3 months. The manager will document instances where your performance was lacking, pulling in quotes from colleagues, individual code reviews, or emails you may have sent (or have not) sent. A PIP can feel like a personal attack: the manager will spend hours compiling an argument "against you", explaining why you're not meeting expectations.

We’ll talk about specific strategies to use if you’re on a PIP, but for now, just remember this: **You can still have an amazing career after a PIP.**

Being on a PIP is like being on an extended interview. You have all the anxiety and judgment from a high-pressure environment, but it's actually worse: instead of starting from neutral, you're starting with a negative perception. And a multi-month interview sounds terrible.

Employees on a PIP are not allowed to switch teams. The company has concerns about whether the employee can productively contribute to the company, so they want to avoid the chaos of allowing team switches. This is unfortunate because you may just have a really poor fit with your manager.

Because of the overhead involved with a PIP, generally only medium and large companies will issue them.

The Human Resources (HR) team is involved in every PIP, which is a bad sign. The main function of HR, if not the sole function, is to protect the company’s interests, not the employees’. Once HR gets involved, and the manager has started documenting gaps in the employee's performance, clearly that indicates that there’s been a loss of trust.

This course will cover 4 phases of the PIP lifecycle:

- Before the PIP (but perhaps in the danger zone)
- Receiving the PIP within the company and deciding what to do
- Case studies of Individual Contributors (ICs) who received PIPs, along with managers who have delivered PIPs
- After the PIP -- how to ensure you have a great career even if you received a PIP

Tech Career Growth Videos

Our instructional videos provide software engineers with insights needed for career growth. Masterclasses go incredibly in-depth on a specific tech topic while case studies are real examples from established engineering leaders.