09-09, 11:00–11:20 (Asia/Jerusalem), Hall 1
Language: English
Speed up your pipelines by doing less! We’ll explore memory efficient caching, filtering, and take a deep dive into the often-overlooked Bloom filter — with practical examples to avoid unnecessary IO and computation.
Python has become powerful, but data pipelines, workloads, and API servers often suffer from unnecessary IO and redundant computation that slow things down.
In this talk, we’ll explore two essential techniques to speed up workloads by doing less: caching and filtering—and how to implement them efficiently, with relatively low memory overhead, along with real-world use cases where these techniques made an impact.
We’ll also take a closer look at an often overlooked and misunderstood tool: the Bloom filter. You’ll learn how it works, when it’s useful (and when it’s not), and how it helps you maintain a low memory footprint while effectively avoiding unnecessary database queries, API calls, or heavy computation—before they even happen.
Whether you're building data pipelines, APIs, or wrangling large datasets, this talk will give you practical insights and Pythonic tools to write smarter, faster, and more memory-conscious code.
Basic
Target audience –Developers, R&D
Former hardware hacker turned Python enthusiast, I’ve worked as a programmable hardware engineer and security researcher. Today, I’m a software engineering team lead at the Hexagate team in Chainalysis, developing a real-time high-scale system that’s based on a distributed micro-service architecture.