Paper Reading: AutoDev: Automated AI-Driven Development

Are you interested in tools like the recently announced Devin?\
\
Join us for an enlightening session as we delve into the cutting-edge world of agentic AI-driven software development, featuring a comprehensive exploration of the AutoDev framework, Microsoft's agentic AI-driven software development framework.

This reading event will elucidate how AutoDev will revolutionize the software development process. Through enabling autonomous planning and the execution of complex engineering tasks, AI agents such as AutoDev will be fully integrated into common tools and technologies such as VSCode and GitHub. Discover how these agents can perform multiple operations on a codebase, from file editing and testing to git operations, all within a secure Docker-based environment.

Moreover, we will examine AutoDev's impressive performance on the HumanEval dataset, achieving remarkable scores of 91.5% and 87.8% for Pass@1 in code generation and test generation tasks, respectively. We will also discuss the implications of this research for the future of AI-assisted software development.

This event is a must-attend for software developers, researchers, and tech enthusiasts eager to understand the future of agentic AI-driven software development and its potential impact on the software development landscape. Whether you're interested in improving your development workflow, exploring AI applications in software engineering, or simply curious about the latest advancements in AI-driven development tools, this paper reading offers valuable insights and discussions on the forefront of technology innovation.

Mark your calendars and join us for an exciting hour of "AutoDev: Automated AI-Driven Development" exploration!

Paper Reading: Self-Discover - Empowering Large Language Models with Autonomous Reasoning Structures

In this talk, we delve into Self-Discover, an innovative framework revolutionizing Large Language Models (LLMs) by enabling them to autonomously discern task-specific reasoning structures, thereby empowering them to tackle complex problems without explicit prompting. 

At the heart of Self-Discover lies a self-discovery process, wherein LLMs independently select and amalgamate various atomic reasoning modules, such as critical thinking and step-by-step reasoning, into explicit structures to guide their decoding process. The broad applicability of these self-discovered structures across diverse model families—from PaLM2-L to GPT-4 and from GPT-4 to Llama2—underscores the framework's versatility and adaptability, suggesting seamless integration with an even wider range of advanced AI models. This holds promise for a unified approach to autonomous reasoning in artificial intelligence.\
\
Whether you're an AI researcher, engineer, or simply curious about this achievement in AI research, join us as we explore the intricacies of Self-Discover and its potential implications for prompt engineering and advancing the capabilities of Large Language Models in complex reasoning tasks.\
\
Paper: [**https://arxiv.org/pdf/2402.03620.pdf**](https://arxiv.org/pdf/2402.03620.pdf)

Lucia on LinkedIn: <https://www.linkedin.com/in/lucia-mocz-ph-d-35a98a1b6/>

Taro Book Club: Designing Data Intensive Applications - Chapter 5 (Replication)

Topic: Chapter 5 - Replication

Session led by: [Rohit Katyal](https://www.linkedin.com/in/rohitkatyal/)

In this #taro-bookclub discussion, we'll be talking about Chapter 5 of [**"Designing Data-Intensive Applications"**](https://www.amazon.com/Designing-Data-Intensive-Applications-Reliable-Maintainable/dp/1449373321) by Martin Kleppmann.

Here's the agenda:

- Let’s get to know each other with some quick introductions.
- We’ll break down and discuss the key concepts from Chapter 5.
- There will also be time for any questions and answers you might have.

Please send your questions ahead of time to [**Rohit Katyal**](https://www.linkedin.com/in/rohitkatyal/) in the [**Taro Premium**](https://www.jointaro.com/membership/) Slack, so that they can be included in the discussion.

Get ready for another engaging and informative session!

Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases

Session led by [Dharin Shah](https://www.linkedin.com/in/dharinshah/)

In this session, we delve into the intricacies of Amazon Aurora, a high throughput, cloud-native relational database designed to meet the demanding scalability and performance requirements of modern applications without compromising on the simplicity of access via SQL. The paper presents a thorough examination of the design considerations that underpin Aurora's architecture, understanding how it achieves unparalleled levels of efficiency and reliability within the cloud. We will explore key concepts such as Aurora's distributed and decentralized approach, fault tolerance mechanisms, and the optimization of data storage and retrieval processes.

Taro Book Club: The Fundamentals of Data Engineering - Chapter 3 (Part 2)

Session led by [Gideon Blinick](https://www.linkedin.com/in/gideonblinick/)

Chapter 3 - Part 2: Designing Good Data Architecture

In our first session on Chapter 3, we learned the 9 principles of Good Data Architecture, which are largely based on Google's Principles of Cloud-Native Architecture and Amazon's Well-Architectured Framework.

Now we get less abstract and cover Major Architecture Concepts and Examples, including Domains and Services, Distributed Systems, Data Warehouses and Lakes, the Modern Data Stack, and so much more!

Paper Reading: Self-supervised Vision Transformers (Foundational Model)

Session led by [Mandar Deshpande](https://www.linkedin.com/in/mandardeshpande1995/)

Dino is a self-supervised learning approach for computer vision tasks. It leverages vision transformers without using any labeled data. Dino v2 introduces several improvements over its predecessor, enhancing performance and scalability. It achieves state-of-the-art results on various benchmarks. Notably, Dino v2 demonstrates strong performance on downstream tasks such as image classification and object detection.

We will be walking through

- Basics of computer vision
- Self supervised learning
- ViT (vision transformers)
- DINO
- DINO v2 model

Paper link - [**https://arxiv.org/abs/2304.07193**](https://arxiv.org/abs/2304.07193)

[Event link](https://www.jointaro.com/event/paper-reading-dino-v2-ie-foundational-model-for-computer-vision/)

Paper Reading: Refactoring vs Refuctoring: Advancing the state of AI-automated code improvements

Paper Discussion led by [Adi Pratheek](https://www.linkedin.com/in/pratheekadi/)

"**Refactoring vs Refuctoring**" is the conclusion of a benchmark study of the most popular Large Language Models (LLMs) and their ability to generate code for refactoring tasks. This paper aims to illustrate the current standards and limitations, and seek to show new methodologies with higher confidence results.

**What's in store?**

In this paper reading session, we will be examining:

- The performance of the most popular Large Language Models (LLMs) on code refactoring
- How does an AI err?
- A novel innovation by CodeScene for fact-checking the AI-output and augmenting the proposed refactorings by measuring the confidence levels

Companies build pretty substantial codebases over a period of time (sometimes decades). As and when the companies grow, a significant amount of time and energy are constantly invested by development teams on code refactoring (also covered in this paper).

But, how well do the existing AI tools/models actually perform this task? Are we there yet to confidently say "**AI will replace software developers**"? Join this session where we dive deep into the paper to find more answers.

Link to the paper: [**Refactoring vs Refuctoring: Advancing the state of AI automated code improvements**](https://codescene.com/hubfs/whitepapers/Refactoring-vs-Refuctoring-Advancing-the-state-of-AI-automated-code-improvements.pdf)\
Event Link: <https://www.jointaro.com/event/paper-reading-refactoring-vs-refuctoring-advancing-the-state-of-ai-automated-code-improvements/>

Paper Reading: AlphaGeometry: Advancing Automated Reasoning in Olympiad-Level Geometry

Paper Discussion led by [Lucia Mocz](https://www.linkedin.com/in/lucia-mocz-ph-d-35a98a1b6/)

Solving mathematical problems at the Olympiad level is incredibly difficult, requiring advanced problem-solving skills, a deep understanding of mathematical principles, and the ability to think critically and creatively under pressure. Those who excel in Olympiad-level problem solving usually have exceptional talent, undergo extensive preparation, and possess a unique problem-solving mindset. Achieving human-level automated reasoning in mathematical theorem proving, especially when competing with top performers, is a significant milestone in artificial intelligence research.

In this paper reading session, we introduce AlphaGeometry, a groundbreaking theorem prover tailored for Euclidean plane geometry. Unlike traditional methods that rely on human demonstrations, AlphaGeometry autonomously synthesizes millions of theorems and proofs across various complexities without human intervention. By combining a neural language model trained on vast synthetic data with a symbolic deduction engine, AlphaGeometry employs a neuro-symbolic approach to navigate intricate problem spaces. Notably, AlphaGeometry generates human-readable proofs and has successfully tackled challenging problems from previous International Mathematical Olympiads, even discovering new generalizations of established theorems.

Whether you're an AI researcher, engineer, or simply curious about this achievement in AI research, join us as we explore the intricacies of AlphaGeometry and its potential implications for automated reasoning in mathematics.\
\
Paper: [**https://www.nature.com/articles/s41586-023-06747-5**](https://www.nature.com/articles/s41586-023-06747-5)\
Event Link: <https://www.jointaro.com/event/paper-reading-alphageometry-advancing-automated-reasoning-in-olympiad-level-geometry/>

Blog: [**https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/**](https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/)

GitHub: [**https://github.com/google-deepmind/alphageometry**](https://github.com/google-deepmind/alphageometry)

Paper Reading Session: Simple Testing Can Prevent Most Critical Failures

Paper Discussion led by [Sudhi Awasthi](https://www.linkedin.com/in/sudhiawasthi/)

In this session, we will be discussing this paper titled "[Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf)".

Large, production quality distributed systems still fail periodically, and do so sometimes catastrophically, where most or all users experience an outage or data loss. This paper presents the result of a comprehensive study investigating 198 randomly selected, user-reported failures that occurred on Cassandra, HBase, Hadoop Distributed File System (HDFS), Hadoop MapReduce, and Redis, with the goal of understanding how one or multiple faults eventually evolve into a user-visible failure and how simple testing can prevent majority of them.\
\
Event Link: <https://www.jointaro.com/event/paper-reading-session-simple-testing-can-prevent-most-critical-failures/>

Paper Reading - Gemini: A Family of Highly Capable Multimodal Models

Paper Discussion led by: [Akshika Wijesundara PhD](https://www.linkedin.com/in/akshika/)

Abstract: Google researchers recently unveiled Gemini, a ground breaking family of multimodal AI models capable of understanding and reasoning across images, audio, video, and text. The largest Gemini model, Ultra, achieved human-expert performance on several benchmarks and advanced the state of the art across a range of multimodal tasks.

In this paper reading, we will discuss key details from Google's paper introducing Gemini models. Topics will include:

- The Gemini model architectures and how they achieve cross-modal understanding
- Performance benchmarks and state-of-the-art results across text, image, audio, and video tasks
- Applications and use cases enabled by Gemini's multimodal reasoning
- Approaches to responsible deployment of large language models like Gemini

This paper provides a comprehensive overview of the exciting new capabilities of Google's Gemini project. Whether you're an AI researcher, engineer, or simply curious about the state of the art in multimodal AI, join us for an in-depth look at this new model family.\
\
Paper link: [**https://arxiv.org/abs/2312.11805**](https://arxiv.org/abs/2312.11805)\
Event link:  **[https://www.jointaro.com/event/paper-reading-gemini-a-family-of-highly-capable-multimodal-models/](https://www.jointaro.com/event/paper-reading-gemini-a-family-of-highly-capable-multimodal-models/)**

In this session, we will be discussing this paper titled "[**Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems**](https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-yuan.pdf)".

Large, production quality distributed systems still fail periodically, and do so sometimes catastrophically, where most or all users experience an outage or data loss. This paper presents the result of a comprehensive study investigating 198 randomly selected, user-reported failures that occurred on Cassandra, HBase, Hadoop Distributed File System (HDFS), Hadoop MapReduce, and Redis, with the goal of understanding how one or multiple faults eventually evolve into a user-visible failure and how simple testing can prevent majority of them.

Presented by [Sudhi Awasthi](https://www.jointaro.com/p/sudhi-awasthi/), an engineer working at Bloomberg. Connect with her on LinkedIn: <https://www.linkedin.com/in/sudhiawasthi/>.

Demystifying LLM Benchmarks -- How To Compare Large Language Models?

In the recent launch of Google's Large Language Model (LLM) Gemini, Google claimed that "Gemini beats human level performance on MMLU" but you might be wondering:

- What are the different evaulation criteria? Massive Multitask Language Understanding (**MMLU**)? General Language Understanding Evaluation (**GLUE**)? **HumanEval**?
- How can we know which model is good at coding tasks?
- What is considered "human-level performance"?
- How do they actually get the results for these benchmarks?

In this talk, Sai breaks down all the benchmarks you need to know to be better informed when papers claim to beat benchmarks.

The next paper reading session is on January 8 at 5pm: [**https://www.jointaro.com/event/paper-reading-group-4-mistral-7b/**](https://www.jointaro.com/event/paper-reading-group-4-mistral-7b/)

**Your host:**

[**Sai Shreyas Bhavanasi**](https://www.linkedin.com/in/b-sai/) has worked in Computer Vision and Reinforcement Learning and published 2 first-author papers. He has also worked as an MLE and Data Analyst.

Paper Reading Group: Attention Is All You Need

Generative AI, specifically large language models, have taken the world by storm. But they wouldn't exist today without a key paper published by Google Brain: "[Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf)." The 10-page paper, published in 2017, introduced the Transformer - a critical innovation in neural networks.

**What's in store?**

- A deep dive and interactive discussion of the paper. We recommend reading (or at least skimming) it first: <https://arxiv.org/pdf/1706.03762.pdf>
- A look at what came before the paper (neural networks, embeddings), and what came after (BERT, GPT-2, and ChatGPT).
- **All software engineers welcome** - no AI/ML background necessary!

**Did you know?**

- The paper's authors have all gone on to found multiple billion-dollar AI companies including Cohere and [Character.ai](http://Character.ai).
- Since 2017, the paper has been cited over 86,000 times.
- The Transformer architecture is the **T** in "GPT".

**Your host:**

[Charlie Guo](https://www.linkedin.com/in/charlierguo/) is a seasoned founder and author with over 15 years of software experience and more than a decade in Silicon Valley. He is an alum of both Stanford and YCombinator, and currently publishes [Artificial Ignorance](https://www.ignorance.ai/about), an AI-focused newsletter for engineers and founders.

**The agenda:**

- Charlie will give a brief background, and then explain the key insights and context from the paper.
- We will open the group to some Q&A and a broader discussion.
- And we'll end with some discussion on which paper to read and review next.

The next paper reading group is on October 15 about [deep reinforcement learning](https://www.jointaro.com/event/paper-reading-group-human-level-control-through-deep-reinforcement-learning-deep-q-network/), led by Sai Bhavanasi.

Paper Reading Q&A and Videos

About Paper Reading

Paper Reading: AutoDev: Automated AI-Driven Development

Paper Reading: Self-Discover - Empowering Large Language Models with Autonomous Reasoning Structures

Taro Book Club: Designing Data Intensive Applications - Chapter 5 (Replication)

Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases

Taro Book Club: The Fundamentals of Data Engineering - Chapter 3 (Part 2)

Paper Reading: Self-supervised Vision Transformers (Foundational Model)

Paper Reading: Refactoring vs Refuctoring: Advancing the state of AI-automated code improvements

Paper Reading: AlphaGeometry: Advancing Automated Reasoning in Olympiad-Level Geometry

Paper Reading Session: Simple Testing Can Prevent Most Critical Failures

Paper Reading - Gemini: A Family of Highly Capable Multimodal Models

Paper Reading Session: Simple Testing Can Prevent Most Critical Failures

Demystifying LLM Benchmarks -- How To Compare Large Language Models?

Paper Reading Group: Attention Is All You Need