On April 24 2019, cncv will do a whole day dedicated to discussing machine learning pipelines.
Venue: A1 Main Hall, Lassallestraße 9, 1020 Vienna
Data crunching pipelines in distributed environments have come a long way. On CNCML day we are going to take a look at how those pipelines are tackled and implemented in 2019.
The event takes place on April 24 2019. The schedule is as follows:
|09:00||A1 Digital Breakfast||-|
|09:30||Welcome to CNCML 2019||Malte Fiala - cncv|
|09:45||Cloud Native and mission critical applications:
challenges and opportunities
|Andrea Calia - CERN|
|10:10||Using Elasticsearch as the
Primary Data Store
|Volkan Yazici - bol.com|
|11:10||The Cloud Native Journey for Big Data||Marcel Däppen - Cloudera|
|12:00||A1 Digital Lunch||-|
|13:00||How to make 5 billion predictions
in 2 days on a scalable infrastructure
|Andreas Böhm - ONE LOGIC|
|13:40||Building Brains - Parallel training strategies of
large-scale deep learning neural networks
|Romeo Kienzler - IBM|
|14:20||Flexible ML Pipelines (with Kubeflow)|| Hermann Wagner - Google
Yuriy Babenko - Google
|15:10||A1 Digital Coffee Break||-|
|15:30||Service Assurance 2020||Jürgen Moser - A1|
|16:00||Scaling your Python interactive applications
|Romeo Kienzler - IBM|
|16:40||This was CNCML 2019||Malte Fiala - cncv|
|17:00||A1 Digital Coffee To Go||-|
CERN is the European Organization for Nuclear Research and part of its mission is to give an answer to questions like: What is the nature of our universe? In order to study even the most minute bit of the matter, a particle accelerator complex is built to accommodate a variety of experiments.
The biggest accelerator is the Large Hadron Collider (LHC) that was able to produce the collisions that led to the discovery of the Higgs boson in 2012. The LHC can count with a sophisticated software infrastructure that is able to control each hardware device of the machine as well as providing the controls tools to the machine’s operators. Today, all the servers that are needed for operations are deployed on bare machines running a customized Linux distribution. In the view of taking advantage of the new cloud infrastructures available today, a Kubernetes prototype cluster has been setup to assess the viability of such container orchestration tools in CERN operational scenarios: high availability, fault tolerance, easy intervention, fast rollback and long term maintenance.
In this talk, some preliminary results will be presented along with considerations on the usage of container orchestration solutions in mission-critical systems.
The biggest e-commerce company in the Netherlands and Belgium, bol.com, set out on a 4 year journey to rethink and rebuild their entire ETL (Extract, Transform, Load) pipeline, that has been cooking up the data used by its search engine since the dawn of time. This more than a decade old white-bearded giant, breathing in the dungeons of shady Oracle PL/SQL hacks, was in a state of decay, causing ever increasing hiccups on production. A rewrite was inevitable. After drafting many blueprints, we went for a Java service backed by Elasticsearch as the primary storage! This idea brought shivers to even the most senior Elasticsearch consultants hired, so to ease your mind I’ll walk you through why we took such a radical approach and how we managed to escape our legacy.
Since the beginning of a distributed big data environment like Hadoop a little over a decade ago, the technology has been steadily improving. Memory and compute power have become much cheaper, but also the bandwidth has improved by several factors. We need all this technological innovation to handle the predicted 44 zettabytes of data by 2020. Most of the major trend such as cloud, containers, streaming, Machine learning & AI, and the Internet of Things (IoT) have direct impact on the next generation of the big data landscape - therefore we have to revise old design paradigms that make this shift possible.
MARKANT is the largest trade and industry collaboration for European food retail, working with over 14,000 industry partners and approximately 150 trade partners. Together with ONE LOGIC, MARKANT is developing a centralized forecasting platform for their trade and industry partners. The goals is to obtain precise sales forecasts up to 26 weeks in advance. With more than 200 article-location-combinations, this means that 5.5 billion forecasts have to be calculated every week within just 2 days’ time. Additionally, the statistical models must be able to take into account events like promotions and external effects like holidays. The results are optimized production, logistics, and sales processes, leading to significant savings and a reduction of environmental footprint.
DeepLearning is so powerful that it rapidly transforms the Machine Learning space. But DeepLearning is very resource and data hungry. In this talk we show how neural network training works and how it can be parallelized on large scale GPU clusters using inter-model, intra-model, data and pipelined parallelism. We’ll use TensorFlow 2.0 for demonstration purposes.
With more than 2000 open source projects Google has been a contributor to the open source community for a long time. While some of the Open Source projects we contribute are dedicated to AI and ML (like Tensorflow, Deepmind Lab and Kubeflow), others are focused on Apps, Infrastructure and CICD (like Kubernetes, Istio and Spinnaker).
At Google Cloud we are now trying to combine some of those technologies to tackle the following 3 big AI-related customer needs we see at the moment:
Finding AI/ML talent. Only a very small number of all SW developers are able to create a custom ML model.
Standardization and sharing. The market is fragmented without standard ways to build and define ML workflows. There exists no standard way to share your ML work with the rest of your organization either.
Efficiency. 80% effort of a typical ML workflow or project is spent on non-ML tasks, dealing with the lifecycle management of ML workflow.
This is why we are building an end-to-end ecosystem to standardize ML and enable acceleration of AI development. We want to enable you to (1) build upon the work of others by sharing data, models and experiments across your organization and outside, (2) easily access your companies data sources, (3) provision (ML) infrastructure on-demand and with no infrastructure management needed, (4) build powerful ML workflows to simplify and accelerate your model creation and deployment. In the talk we will show you in a real world scenario how you can use and combine Open Source Software, like Kubernetes, Istio, Kubeflow, Tensorflow, et cetera, to improve your agility, efficiency and time to market when you build the products and services of tomorrow.
Telco Companies are transforming to IT Companies, so does A1. A1 is the leading Service Provider and IT Outsourcing Partner in Austria and other European countries. The Service Assurance Domain in A1 guarantees the Service Quality and Customer Satisfaction.
In this session you will learn how we disrupt the Service Assurance legacy architecture with a new stack based on open source software, and move from data silos to a data centric approach.
Uncaged data is the bases to new possibilities like machine learning.
Jupyter Notebooks have become the "de facto" platform used by scientists and engineers to build Python interactive applications to tackle scientific and machine learning problems. However, with the popularity of big data analytics and complex deep learning workloads, there is a growing requirement to extend the computation across a cluster of computers in a parallel fashion. In this talk, we will describe how to use multiple Jupyter Notebook components to enable the orchestration and distribution of interactive machine learning and deep learning workloads across different types of computing clusters including Apache Spark and Kubernetes. This talk is intended to attendees interested in distributed platforms and scientists experiencing difficulties on scaling their scientific workloads across multiple machines.
We very much thank our sponsors who enable us to have this event!
aykit - Verein zur Förderung von Kunst, Kultur, Wissenschaft und Kommunikation
Mollardgasse 70D / 204