Our talks at PyCon DE & PyData Berlin
From 17th to 19th April Python lovers from all fields met again in Berlin as well as remotly. The Scieneers team participating this year was 10 heads strong and enjoyed both, many good presentation as well as socialising with the community. We also had the opportunity to participate actively with two talks.
Raised by pandas, striving for more: An optionated introduction to polars
The talk provided an overview of the Polars library. Polars serves as a fast and memory-efficient alternative to Pandas, built on the foundation of Apache Arrow and written in Rust. Both Rust and Apache Arrow were popular topics this year, with multiple talks discussing different aspects. Our presentation on Polars attracted significant attention and was the first of two consecutive talks on Polars in the main hall of the BCC. We specifically explained how Polars could achieve incredibly fast performance in benchmarks despite being a young and relatively small project compared to other DataFrame libraries. We highlighted its benefits of being close to hardware and using a language more inspired by Spark than by Pandas, enabling real parallelization and lazy query optimization.
Have a look at the slides
Who is an NLP expert? – Lessons Learned from building an in-house QA-system
Our second topic covered a popular topic as well. We presented our experience in developing a Slack bot capable of answering user questions by reading internal documents and Slack messages. Although developed in parallel to ChatGPT and not using any LLM yet, our system leverages recent NLP developments such as sentence-transformers, retriever-reader architecture, and a vector database. We also shared valuable insights gained from integrating diverse data sources (many potentially noisy and short Slack messages, along with long, information-packed documents) into a unified knowledge base. After the talk, numerous attendees who faced similar challenges reached out to us, leading to several engaging conversations.
Have a look at the slides
Nico Kreiling – Data Scientist
Alina Bickel – Data Scientist