Deep Learning Week Europe Agenda

Berlin - October 5-6, 2022

This keynote gives an overview of the current innovations within the realm of Natural Language Processing. It will cover the various applications which can be built using the Transformer architecture and dives into the importance of context for AI in today’s world. This will be followed by an in-depth overview of Semantic Search and how to build it from scratch and develop an API for it.

Despite the wealth of information available to job seekers, choosing one’s next job remains a complex endeavor. We will present an approach for creating job recommendations across industries by combining hand-annotated job data and a multivariate technique combining job title, job description, and skill set similarity. This method was used to scan 9 million job transitions to identify the best matches for any given job and uncover deep insights into how the labor market is organized today.

Global enterprises are increasingly relying on data analytics for decision making. To process data, firms leverage cloud-based data warehouses. As more on-prem data is moved to the cloud, the need for robust data governance controls to ensure data integrity, security, and regulatory adherence is mounting; however, existing governance processes are lagging. Here we present a zero-trust approach that can augment existing governance models and reduce exposure of sensitive data like PII.

As a manufacturer, visual inspection is a crucial part of keeping the quality of our products up to high levels. Above that many new applications can be done by making use of deep learning in combination with computer vision. Deep learning models can be industrialized with the Visual Perception Provider (VPP) which is a service developed in-house at Continental Tires. The talk is about the reasoning, why we decided to go that route, and about what we are doing. Also, some use cases done with the Visual Perception Provider will be demonstrated.

In recent years, there have been various efforts to product thinking and decentralized data loading using data meshes. However in deep learning, data loading is still challenging to master at scale. In this talk, we present our decentralized data loading solution and show why flexibility and collaboration is key to enable novel ML use cases. We hope to make large-scale model training accessible to a wider community and move towards more sustainable ML.

This session presents Anomaly Detection Neuroevolution (AD-NEv) – a multi-level optimized neuroevolution framework. The method adapts genetic algorithms for: i) creating an ensemble model based on the bagging technique; ii) optimizing the topology of single anomaly detection models; iii) non-gradient fine-tuning network parameters. The results prove that the models created by AD-NEv achieve significantly better results than the well-known anomaly detection deep learning models.

During this session, we will discuss the different risk-based categories of AI laid out by the EU’s Artificial Intelligence Act and find out how to become more admissible as per the Act. Thereafter, we will walk through the concrete steps, tools, and practices such as monitoring, explainability, model fairness, and compliance that are instrumental in achieving Responsible AI and building more risk-proof and market-friendly solutions

Being able to predict KPIs for new, not yet published articles is a key factor in process optimization to assist editors with their daily work flow. Modern NLP methods allow us to use text content efficiently and to understand the connection between natural language and the respective KPIs. By combining statistical models (GAM), modern NLP methods (BERT), and XAI tools (SHAP), we are able to understand specific connections between text content and our KPI.

Machine learning models are becoming obsolete and must be retrained – this is the current widespread tenor. Is this actually true? And if yes, which components does a CI/CD pipeline for machine learning really need – and which are optional? How can the whole thing be implemented without building a complete Machine Learning Platform team? And which challenges are still difficult to solve at present? A field report including (mis)decisions, which will help to choose the right path for your own challenges.

AI systems are fundamentially vulnerable – ignoring AI security risks will jeopardize safety of people and security of companies. This session presents a framework for secure AI development covering entire model lifecycle. It’s relevant for AI stakeholders across functions and levels in AI development, governance, and product security teams. The stages and activities described in the framework should enable AI stakeholders to implement a secure development lifecycle for their AI systems.

AI algorithms deteriorate and fail silently over time impacting the business’ bottom line. The talk is focused on learning how you should be monitoring machine learning in production. It is a conceptual and informative talk addressed to data scientists & machine learning engineers. We’ll learn about the types of failures, how to detect and address them.

The desire to embody Responsible AI practices requires an understanding of, the context around, and the impact on the end user. To achieve this, design and research are just as pivotal to the RAI conversation as ML. There is no bigger risk, and no greater irresponsibility, than to not interface with those who will be affected by your design. This talk will share how to navigate customer relationships to encourage end user contact and mitigate assumptions and therefore risk.

In my quest to become a Pokémon master, I need to learn a lot about their types. I’d rather create a model to do so for me. A simple one-off won’t do: to be ready for upcoming generations, I need a pipeline for experimenting with new datasets and configs. We will set up a codified ML pipeline using the open-source DVC library. This will help us adopt a reproducible, experiment-driven approach to ML, which will boost our ability to iterate over models and compare them to find the best one.

The changes in the ads ecosystem have led marketers to lean on existing aggregate experimentation tools that assume a predetermined treatment effect. Choosing the treatment group to ensure you have high chances of detecting an effect is non-trivial. Built by Meta Open Source, GeoLift solves this problem by building well powered geographical experiments. Join us to go over why geographical experiments are necessary and their implications in the marketing industry, along with a demo of GeoLift.

In this session, two open source libraries will be demonstrated to show you how you can quickly deploy custom Named Entity Recognition (NER) solutions. NERDA is an easy-to-use interface to apply pre-trained transformer models (e.g. based on Huggingface) to your own challenges. Combined with FastAPI, a lightweight webframework for Python, you can build your solution in short time

Survival analysis, the modeling of time-to-event data, is a statistical field with a long history and great potential to marketing and analytics. In this deep dive, you will learn about the brief origins of survival analysis and applications in the field of customer retention, which is of particular importance for subscription-based growth. Learn how to grow your understanding of customer churn, and learn how to better predict your customer’s lifetime, along with monetary aspects altogether.

This session is an introduction into Apache TVM – an end to end compiler framework for deep learning models. It can compile machine learning models from various deep learning frameworks to machine code for different type of hardware targets like CPU, GPU, FPGAs, microcontrollers. It provides bindings for different higher level languages like C++, Rust etc. and also has provision for autotuning the models for different hardware targets. You will learn how to use Apache TVM to deploy your models on different target systems.

One of the most profitable applications of ML/AI is the large-scale recommender systems. Whether it is e-commerce, ads targeting or ranking posts, recommender systems have become a big part of our lives. But whatever the application, the purpose of the recommender system is to suggest relevant items to millions of users at scale. The session will cover the principles and techniques used to build such recommender systems with case studies from video recommendation and NLP systems.

In this session, the power of compound sparsity for model compression and inference speedup will be demonstrated for NLP (HuggingFace BERT) and CV (YOLOv5) applications. The open source library SparseML will be used for applying compound sparsity onto dense models, utilizing techniques including structured + unstructured pruning (to 90%+ sparsity), quantization, and knowledge distillation. After sparsification, these models will be run on the DeepSparse engine, which is optimized for executing sparse graphs on CPU hardware at GPU speeds. The participants of the session will learn how to apply compound sparsity so that they can run inference at an order of magnitude faster than the original dense models without a noticeable drop in accuracy.

Classification is one of the most frequently solved problems using machine learning. Unfortunately, it cannot handle a case with a number of classes, varying over time, and require all the data to be labelled. There is another approach, designed to solve cases when we can’t perform full data annotation and/or would like to dynamically modify the number of classes. Similarity learning is capable of solving such problems even with extreme classification. We’re going to show how to use such models in production.

Many safety-critical systems, such as autonomous vehicles, rely on neural networks as state-of-the-art for image classification. Despite high accuracy, their final classification decision is difficult to verify. Uncertainty on how neural networks will behave is a challenge to safety. A known issue is providing high probabilities for unknown images. This deep dive will present some novel solutions to inspect certain parameters of trained neural network for detecting out-of-distribution data.

As far as people moved to sell and buy online because of covid-19, the importance of making right pricing decisions increased dramatically. Market is extremely competitive, supply chain is very complex. That’s why being top seller is much more complicated than it was before. This deep dive session will show how to build pricing agents that will react on different changes in a market and will keep your products on top of sellers.

Object detection with a monocular camera is extremely important for the automotive industry as obtaining LiDAR data is not only expensive but getting them labelled is extremely difficult. Previous works have tried removing dependencies of LiDAR but only for inference, they still needed LiDAR data during training. In our work, there are no requirements of LiDAR data annotations. Yet the major advancement in our work compared to that of the previous works is that previously 3D detections were initially performed by stacking two different deep learning networks i.e., a 2D object detection network followed by projecting them to Bird’s Eye View (BEV) to get the depth from a depth prediction network. The presented approach instead combines the two different deep learning networks in one single feed-forward pass with a common backbone network separating out at heads. Having two different heads with common backbone helps the backpropagation learn the weights by mutually improving the two different tasks of 2D object detection and depth prediction simultaneously, thus giving better and faster output as the previous works.