Over the past few years, society has seen a rebirth of interest in artificial intelligence (AI) and, more specifically, machine learning (ML) applications. As is typical in time periods of hardware performance spikes in which computer scientists can increase the throughput of their systems, researchers have been using computers to learn from patterns in data that once took an impractical amount of time to process. This capability has not been limited to large, corporate entities; the advent of graphics processing units (GPUs) has enabled even non-corporate equipment to process large data sets. On a corporate scale, machines with multiple GPUs are used in data centers to mine information and identify data patterns like never before.
How can engineers organize such systems to take advantage of all the practices proposed by Agile methodologies and the more recent DevSecOps developments? In this blog post, I will explore the machine learning (ML) and DevSecOps domains and propose ways to use them in collaboration for increased performance.
Understanding Pipelines
When software engineers say pipeline, they usually mean a set of tools chained together—into some form of framework—that provide the necessary glue-code to stitch them together. This glue-code usually has functions of storing and retrieving information from a data repository that is common to all tools, or at least moving data from one tool to the next in line.
The two domains we are talking about in this article are ML and DevSecOps. In this context, a pipeline may mean a learning ML pipeline that is used to process data for an ML framework. A pipeline may also be a DevSecOps pipeline that is used to transform source code into a built product, using build directives, as well as functional and security testing.
Machine Learning Pipelines
An ML pipeline is a chain of tools used to automate the processes in an ML machine learning workflow. These tools allow the transformation and correlation of data into a model that will be
This article is purposely trimmed, please visit the source to read the full article.
The post Aligning DevSecOps and Machine Learning appeared first on Carnegie Mellon University's Software Engineering Institute Blog.