I’ve talked at Voxxed Days Buchares 2017 about how to apply some DevOps practices to Big Data applications by leveraging the power of Kubernetes and Openshift.
DevOps, continuous delivery and modern architectural trends can incredibly speed up the software development process. Big Data applications cannot be an exception and need to keep the same pace. In this talk, Nicola Ferraro (Red Hat) will show an overview of the next generation cloud-native Big Data systems based on Docker and Kubernetes. Switching from static Big Data platforms to infrastructure as code allows releasing robust applications in minutes by minimizing the gap between all the environments (development, testing, staging and production), a typical problem in Big Data application development. The presentation goes through the full release cycle of a Spark application alongside a fleet of microservices exchanging data in near real time. A fully featured continuous delivery pipeline, provided by the Fabric8 platform, will automate every step of the build and release process. Spark clusters will come to life from nowhere and disappear when they are no longer needed…
The slides are available here:
And here’s the recording:
