Paper Reading: Enhancing Reasoning Abilities in LLMs through Geometrical Understanding

Session Led by [Lucia Mocz](https://www.linkedin.com/in/lucia-mocz-ph-d/)

The advancement of large language models (LLMs) for real-world applications hinges critically on enhancing their reasoning capabilities. In this paper read, we explore the reasoning abilities of LLMs through a geometrical framework. We establish a connection between the expressive power of LLMs and the density of their self-attention graphs.

The analysis performed in this work demonstrates that the density of these self-attention graphs defines the intrinsic dimension of the inputs to the MLP blocks. In particular, it shows, through theoretical analysis and toy examples, that a higher intrinsic dimension implies a greater expressive capacity of the LLM. Additionally, it provides empirical evidence linking this geometric framework to recent advancements in methods aimed at enhancing the reasoning capabilities of LLMs.

Don't miss this opportunity to deepen your understanding of how geometrical insights can drive improvements in AI reasoning. Whether you're an AI researcher, practitioner, or enthusiast, this talk promises to offer valuable perspectives and groundbreaking ideas.\
\
Paper: [**https://arxiv.org/abs/2407.02678**\
(Accompanying](https://arxiv.org/abs/2407.02678￼\(Accompanying) paper: [**https://arxiv.org/abs/2312.01648**](https://arxiv.org/abs/2312.01648))

What Engineers Get Wrong About Open Source (From the OSS Lead at Meta)

Open source software is not just about the code! It's about understanding and working with project maintainers to solve a problem.

Break Into Tech And Become A Software Engineer [Part 17] - The Chicken vs. The Egg

In this lesson, Brian addresses the challenge of gaining experience as a software developer when transitioning careers, offering practical solutions beyond traditional routes.

This lesson covers:

- The limitations of open source projects, highlighting their high barriers to entry and explaining why they may not be the best route for everyone
- Freelancing as a more straightforward option for gaining experience, which can lead to valuable work and opportunities
- Turning a side project into a legitimate business, providing either career validation or even a pathway to financial independence without needing a traditional job

If you need extra help breaking into tech, join Brian's exclusive, cohort-based bootcamp (just 10 students per batch). Use this special link for $800 off: <https://www.parsity.io/taro>

Connect with Brian on LinkedIn: <https://www.linkedin.com/in/brianjenney/>

Break Into Tech And Become A Software Engineer [Part 12] - Pick A Platform

In this lesson, Brian emphasizes the value of choosing the right platform for writing online, with a strong recommendation for LinkedIn.

This lesson covers:

- Why LinkedIn is a great platform for building a professional presence, with less anonymity and trolling compared to platforms like X or TikTok
- The advantage of standing out on LinkedIn, where recruiters and employers are more active, making it easier to connect with professionals
- How while platforms like X and TikTok offer cutting-edge tech content, LinkedIn provides a safer environment to network with verified professionals and receive genuine feedback

If you need extra help breaking into tech, join Brian's exclusive, cohort-based bootcamp (just 10 students per batch). Use this special link for $800 off: <https://www.parsity.io/taro>

Connect with Brian on LinkedIn: <https://www.linkedin.com/in/brianjenney/>

Become An Open Source Master [Part 12] - Identify A First Task

We can start contributing to open source projects by looking for existing issues that we can solve. We can use the "help wanted" or "good first issue" tags to find suitable tasks. Contributing to open source projects is a great way to get involved in the community and make the project better. Check out TinyBase as an example: <https://github.com/tinyplex/tinybase/issues>

Connect with James:

- GitHub: <https://github.com/jamesgpearce>
- Twitter (X): <https://twitter.com/jamespearce>

Become An Open Source Master [Part 9] - Solve A Need

When choosing an open-source project to contribute to, we should:

- **Focus on solving a need:** Choose a project that you'll benefit from and have a passion for working on.
- **Assess the project's health:** Look for signs of active maintenance, such as merged pull requests and recent activity.
- **Seek insights from others:** Talk to people who have worked on the project to get their perspectives on the community and development process.

Connect with James: 

- GitHub: <https://github.com/jamesgpearce>
- Twitter (X): <https://twitter.com/jamespearce>

How Managers Hire Great Engineers [Part 1] - How This Course Works

With this lesson, you’ll begin your learning journey of how to effectively manage the recruiting process to build a strong team. [John](https://www.linkedin.com/in/jwegan/) shares personal experiences and actionable insights for hiring success. We go through the course objectives, and after this course, you will:

- Understand the importance of prioritizing recruitment and building a robust candidate pipeline
- Learn how to evaluate and troubleshoot the recruiting funnel to ensure a steady flow of qualified candidates
- Discover strategies for sourcing top talent, especially when resources are limited
- Gain techniques for closing candidates who have multiple job offers, including specific tactics and phrases

Connect with John Egan on LinkedIn: <https://www.linkedin.com/in/jwegan/>

Clear Communication For SWEs [Part 13] - Effective Communication In Meetings

In this video, I discuss strategies for clear communication in team meetings, focusing on different types of meetings like stand-ups, sprint planning, technical design, and client meetings. I emphasize the importance of preparation, concise communication, and active listening during Q&A sessions.

**Action requested:** Implement these strategies in your next team meeting for better communication and collaboration.

Connect with me on LinkedIn: <https://www.linkedin.com/in/ryan-wiens/>

Clear Communication For SWEs [Part 9] - Problem-Solution-Benefit

In this video, I explain the Problem-Solution-Benefit framework, focusing on identifying a problem, proposing a solution, and highlighting the benefits. This framework is crucial for gaining stakeholder buy-in. I provide examples and steps on how to apply it effectively.

Connect with me on LinkedIn: <https://www.linkedin.com/in/ryan-wiens/>

Clear Communication For SWEs [Part 3] - Core Elements Of Communication

In this video, I delve into the core elements of effective communication - empathy, purpose, and responsibility. Understanding and mastering these elements are crucial for successful team interactions and project outcomes. I emphasize the importance of tailoring messages to the audience, defining clear purposes, and ensuring mutual understanding. Viewers are encouraged to adopt a mindset shift towards intentional and thoughtful communication.

Connect with me on LinkedIn: <https://www.linkedin.com/in/ryan-wiens/>

Grow From Senior To Staff Engineer [Part 20] - Generalist

An easy way to understand what a great staff engineer looks like is that they're a "Jack of all trades, master of all". This lesson teaches you:

- What the generalist staff "archetype" is
- The different types of skills staff generalists need to be good at
- When it makes sense to become a generalist
- How generalist staff engineers grow long-term

Grow From Senior To Staff Engineer [Part 16] - Specialist

When you grow your junior to senior, it's best to [pick something, focus on it, and go deep](https://www.jointaro.com/lesson/wUl3hc2wr1ms493bd23b/this-is-how-software-engineers-should-initially-learn/?playlistId=iBKHYpkjd0BRWWLKQG2j). You can actually continue this trajectory all the way to staff. This lesson teaches you:

- What the specialist staff engineer archetype is
- What it takes for a technical domain to be worthy of staff scope
- Examples of staff-level specialist domains

Grow From Senior To Staff Engineer [Part 11] - Define The Roadmap

All the way to senior, engineers are almost entirely given projects to work on. This dynamic flips as you grow from senior to staff. This lesson teaches you:

- The differences between senior and staff engineers when they participate in the team roadmapping process
- The skills you need to become a leading, staff-level influence in roadmapping

Asking Great Questions [Part 17] - Target Domain Experts

A big part of getting great answers quickly is to ask the right people, especially with technical questions as different engineers specialize in different things. This lesson teaches you:

- Why targeting makes it more likely for your question to be answered
- What it looks like when you target domain experts in your question, explained with an in-depth example
- How to find domain experts to tag in your questions

Asking Great Questions [Part 15] - RFC Pull Requests

As a software engineer, you're inevitably going to have to ask a lot of questions about your code. This lesson teaches you:

- A natural tactic you can use to share code context with engineers you want to help you
- What a quality RFC pull request looks like, shown with an in-depth GitHub example
- Why pull requests are so powerful and organic when it comes to attaching technical context 

Level up your code quality here: [\[Course\] Level Up Your Code Quality As A Software Engineer](https://www.jointaro.com/course/level-up-your-code-quality-as-a-software-engineer/)

Asking Great Questions [Part 12] - Length Balancing

One of the most important attributes of a question is its length, and the reason why is simple: The length of a question is directly connected to how much time a helper needs to spend answering it. This lesson teaches you:

- Why questions that are too short are bad
- Why questions that are too long are bad
- What to do if your question seems too short
- What to do if your question seems too long
- What a healthy question length is, demonstrated with a real-world example
- A tactic you can use to keep the length of your question in-check while still including a lot of important context

Tech Career Growth Videos

Our instructional videos provide software engineers with insights needed for career growth. Masterclasses go incredibly in-depth on a specific tech topic while case studies are real examples from established engineering leaders.