06-28, 10:30–10:50 (Asia/Jerusalem), Main Hall
Have you ever written a simple function, and added it to your pipeline only to discover it is WAY slower than it should be? In this talk, I will demonstrate how to sniff out functions that slow down your pipeline and be proactive about speeding up
Have you ever written a short and simple function, and added it to your pipeline only to discover it is WAY slower than it should be? Did you know some pandas functions are written in cython and work much faster than others? Python is not known to be the fastest language, but you can be proactive about speeding things up!
While Python is featureful and simple to write, it isn’t known as a fast language. Many of Python’s functions and one-liners have hidden complexity costs. Choosing the wrong ones can slow down your code and those costs definitely add up. For example - popping an item from the end of a list vs from the start of it, or using the “in” operator on a set vs a list. A few seconds running time difference in your local script could mean a few hours time difference on the production pipeline.
If you want to learn about what to look out for, how to overcome these pitfalls and make your code more efficient this is the talk for you.
English
Target audience –Developers
Maya Gershovitz Bar is a Software Engineer at Meta. She has been working on data pipelines for a number of years and the author of a well known Israeli tech blog about algorithms and complexity.