There is growing interest today in incorporating artificial intelligence (AI) and machine-learning (ML) components into software systems. This interest results from the increasing availability of frameworks and tools for developing ML components, as well as their promise to improve solutions to data-driven decision problems. In industry and DoD alike, putting systems that include ML components into production can be challenging. Developing an ML system is more than just building an ML model: The model must be tested for production readiness, integrated into larger systems, monitored at run time, and then evolved as data changes and redeployed. Because of this complexity, software engineering for machine learning (SE4ML) is emerging as a field of interest.
This blog post describes how we at the SEI are creating and assessing empirically validated practices to guide the development of ML-enabled systems as part of AI engineering—an emergent discipline focused on developing tools, systems, and processes to enable the application of artificial intelligence in real-world contexts. AI engineering comprises development and application of practices and techniques that ensure development and adoption of transformative AI solutions that are human-centered, robust, secure, and scalable.
An ML-enabled system is a software system that relies on one or more ML components to provide capabilities. ML-enabled systems must be engineered such that
Integration of ML components is straightforward.The system is instrumented for runtime monitoring of ML components and production data.The cycle of training and retraining these systems is accelerated.
Many existing software engineering practices apply directly to these requirements, but these practices typically are not used in data science, the field of study that focuses on development of ML algorithms and models that are incorporated into software systems. Other software engineering practices will require adaptation or extension to deal with ML components.
ML Mismatch
In a blog post we published last June, Detecting Mismatches in Machine-Learning Systems, we observed that the ability to integrate ML components into applications is limited by, among other factors, mismatches between
This article is purposely trimmed, please visit the source to read the full article.
The post Software Engineering for Machine Learning: Characterizing and Detecting Mismatch in Machine-Learning Systems appeared first on Carnegie Mellon University's Software Engineering Institute Blog.