The last week of November is always fully packed with companies working to squeeze in as much productivity as possible between Thanksgiving (in the US) and the winter holiday season. Amazon makes the most of this with their Amazon Web Services (AWS) Re:Invent show that continues to grow. In 2018, this year, almost 50,000 people made their way to Las Vegas to hear from AWS CEO Andy Jassy and key strategic leaders about the advancements AWS is making to its cloud platform and insights into how to get the most value from using those capabilities.
Why is Amazon in the AI space to begin with? Perhaps not surprisingly, Amazon has billions of interactions a day, and makes millions of AI inferences every minute, from their product recommendations on the site to the voice-based interactions with Alexa, and many other AI-enabled capabilities in the supply chain, operations, logistics and beyond. Amazon has perhaps some of the broadest experience in the field, not just building underlying technology that others use, but also actually using it themselves, running one of the largest scale businesses across a wide range of industries. From this experience, the mission for AWS Machine Learning is to put machine-learning capabilities into the hands of every developer, regardless of their skills and experience level.
Amazon last year announced their foray into the world of AI and Machine Learning technology capabilities, adding significant capabilities to their cloud stack, and challenging rivals in the space including Google, Microsoft, IBM and others. Just one year ago, they launched Amazon SageMaker, 90+ new enhancements made in just one year, with over 200 new ML launches and major updates since the launch. This year, AWS has kicked things up a significant notch with a wide variety of features that cater to a range of user roles interested in bringing AI capabilities to their own environments.
Amazon’s Perspective on the AI Technology Stack
Amazon is particularly customer-focused, insisting that they only develop product features and capabilities based on actual customer demand and feedback. As such, they sort the world of AI-focused customers into three general personas:
·Expert practitioners & researchers that are already experienced with ML algorithms and AI approaches and are comfortable with tools and framework decisions. AWS aims to support them with infrastructure to increase the speed and lower their cost of development and operationalization
·Developers and Data Scientists that are looking to add AI and ML capabilities to their existing applications or enhance their data science activities. AWS continues to enhance their platform for app developers with the Amazon SageMaker suite and ML framework support.
·Application Developers with little ML knowledge or experience that want AI capabilities and ML but aren’t familiar with how these work or the details. For this audience, AWS has already announced in the past computer vision and natural language processing capabilities such as Rekognition, Polly, Transcribe, Lex, and other tools, but aims to increase overall support to give these developers more capabilities without requiring them to have more specific AI knowledge.
In general, the AWS AI specific announcements at Re:Invent 2018 aim to address three major challenges for AI and ML developers: reducing the cost of AI and ML development and operationalization at training and inference phases, helping AI practitioners better wrangle the vast amounts of data they need to deal with, and improving overall ease of use for AI and ML for the broader audiences detailed above.
AI Infrastructure Announcements
When Amazon first launched their foray into cloud computing almost 15 years ago, they were focused on two simple services: storage and compute. Since then, the AWS ecosystem has evolved significantly handling a wide range of workloads. AI and ML capabilities are now being added to this infrastructure tier to handle the emerging AI and ML use cases. Below are some key announcements from Re:Invent 2018
Amazon EC2 P3dn Instance
Amazon’s core cloud computing technology is their Elastic Compute Cloud (EC2) that executes general computing functionality in a wide range of configurations. To handle the heavy compute loads required for machine-learning training, the company announced the EC2 P3dn instance that is highly optimized for distributed ML training. With up to 100gbps of network throughput, the use of Nvidia V100 GPUs, clustered data sets, and 32GB of memory per GPU, the system can significantly reduce training time, improve GPU utilization, and support larger, more complex models. Amazon is also applying ML to its other EC2 offerings, using pattern analysis with ML to provide predictive scaling to help customers warm up server resources before they are actually needed.
Amazon Elastic Inference
The inference phase of AI projects is where most of the compute cost is, with over 90% of the compute time spent in inference and only 10% of overall AI project compute time spent on training. To help reduce the cost of inference, AWS announced Amazon Elastic Inference that matches inference capacity to actual demand. Users can add inference to any EC2 instance, and is integrated with the SageMaker tool. Elastic Inference has support for TensorFlow, Apache MXNet, with PyTorch coming soon.
To help make training and inference even more efficient, Amazon has invested in reworking TensorFlow to be optimized for the AWS environment. AWS claims 90% scaling efficiency and an over 50% reduction in training time with the optimized TensorFlow, as used within the SageMaker offering.
To further optimize the AWS AI and ML stack, Amazon is announcing the soon availability of the AWS Inferentia chip, a purpose-built, high performance machine learning chip which is a high performance alternative to GPUs for training. The solution promises to provide orders of magnitude cost reduction and is a response to Google’s TensorFlow Processor Units (TPUs) and the Microsoft Xilinx based FPGAs.
AWS SageMaker Neo
At the inference stage of AI projects, practitioners are challenged with running ML models in a wide range of environments and end points. To simplify the process of deploying ML at the edge and other endpoints, AWS announced SageMaker Neo that aims to provide a way to train the model once and run it anywhere. The solution compiles ML models to 10% of the size of the original frameworks and provides a deep learning compiler with 2x performance. The solution has broad support for hardware including from Intel, Nvidia, ARM, Xilinx, Cadence, and Qualcomm, and supports TensorFlow, MXNet, and Pytorch frameworks.
Accelerating AI Development
To have apps that can run on the AWS platform, they need to be developed first. To that end, Amazon has announced a significant addition to their development suite to help accelerate AI and ML app development.
Amazon SageMaker Ground Truth
Well-labeled, sufficient, high quality training data is the key to a successful ML project, and necessary for any form of supervised learning. To help accelerate and improve the labeling of data, Amazon announces SageMaker Ground Truth to improve labeling of data and make machine learning training data more easily and accurately labeled. The solution leverages Amazon Mechanical Turk as well as private and third-party human-powered labeling tools to start the labeling process, and builds an active learning model that the augments human labeling to provide machine-based automated labeling with degrees of confidence. In this way, labeling can be accelerated and performed at lower cost.
Amazon SageMaker RL (Reinforcement Learning)
In addition to supervised and unsupervised machine learning methods and algorithms, the AI toolbox includes reinforcement learning (RL) as a method to achieve goal-oriented learning tasks using a goal/reward system. AWS announces the Amazon SageMaker RL tool to simplify the building of RL solutions and provide reinforcement learning for developers and data scientists. In addition to core RL capabilities, the solution supports 2D and 3D environments for simulation, including the Amazon Sumerian AR and VR environment.
Many RL projects are in the robotics space, using RL as a way to train physical and virtual bots. Regardless of ML approach, Amazon announced the AWS RoboMaker tool that makes it easier to develop robotics apps and manage the lifecycle of robotics apps. The solution provides cloud extensions for the Robotics Operating System (ROS), a development environment, simulation environment, and fleet management tool
Higher Level AI Capabilities
Moving up the stack, Amazon is focused on helping even casual developers leverage the power of AI and ML. At AWS Re:Invent 2018, the company made the following product announcements:
AWS Comprehend Medical
Leveraging Amazon’s AI technologies, the company built out a healthcare industry-specific solution called Comprehend Medical that can extract health-related text and data from virtually any document, using natural language processing (NLP) extraction in unstructured data. This can identify health-related information and terms and automatically create relationships between healthcare data.
Amazon has a first-class product recommendation and personalization system that is often copied and envied by other online retailers. Now, Amazon is making the underlying technology available through AWS with the Personalize tool. The AI-powered personalization and recommendation engine uses experience on recommendation systems learned from the many years of operating Amazon.com, but it can be applied to any need that can benefit from ML-trained personalization and recommendation systems. Amazon insists that all the data remains the ownership of the customer as is not aggregated by Amazon. Why beat Amazon at the personalization game when you can join them?
In addition to personalization ML features, Amazon announced Forecast tools that provide ML-based forecasting capabilities using predictive analytics. Amazon claims it can improve forecasting accuracy by up to 50% at 10% the cost. The system can deal with irregular time-series data and many variables and lets customers deploy a customized forecasting API for their own forecasting needs.
Amazon also announced advanced natural language and character recognition capabilities with the Amazon Textract tool. The solution is touted as “OCR++” and can extract text and data from virtually any document, with strong support for text in a range of formats. It can detect tables and other document structure as well as extract information from forms.
AWS Marketplace for Machine Learning
Third-party vendors continue to broaden the AWS platform. In the case of machine learning, Amazon announced the AWS Marketplace for Machine Learning that provides third party algorithms and models available instantly to the SageMaker platform. The Marketplace now has over 150 models and algorithms, and will continue to grow.
Helping People Learn About AI
Amazon is also realizing that there is a serious, and possibly growth-limiting problem plaguing adoption of AI technology: the lack of talent. To avoid this talent crunch squeezing the AWS business, Amazon is investing in growing AI skills. At Re:Invent 2018 Amazon announced the following:
Amazon invested significant time and effort to train its own employees on AI and machine learning. They are now providing these training resources for free in its Amazon Machine Learning University. According to the company, you can learn ML the same way that Amazon trains its developers and engineers. The courses are free, although you need to implement the code using AWS services, which aren’t, and the exams cost money if you want to become certified.
A year ago, Amazon announced the AWS DeepLens as a way for developers to become familiar with the concepts around computer vision and build expertise in the area. In the same way, Amazon announced the DeepRacer as a way to become familiar with reinforcement learning and autonomous capabilities. The DeepRacer is a fully autonomous 1/18 scale race car that is optimized to run around a race track. The system is a hardware platform with camera, processor, and other technology that uses SageMaker RL. The company also announced the AWS DeepRacer League, the first global autonomous racing league, as a way to have developers compete, win prizes, and advance knowledge of RL and autonomous vehicles.
Amazon ML Solutions Lab
To finalize the roundup of major AI and ML announcements at AWS Re:Invent 2018, the company announced the Amazon ML Solutions lab as a way of transferring knowledge and experience to enterprises looking to adopt and gain value from AI and machine learning.
Amazon sees AI, machine learning, and cognitive technology not only as a core component of their AWS cloud computing platform, but as a strategic differentiator that will provide value not only for Amazon, but its customers in the long-term.
Ronald Schmelzer, columnist, is senior analyst and founder of the Artificial Intelligence-focused analyst and advisory firm Cognilytica, and is also the host of the AI Today podcast, SXSW Innovation Awards Judge, founder and operator of TechBreakfast demo format events, and an expert in AI, Machine Learning, Enterprise Architecture, venture capital, startup and entrepreneurial ecosystems, and more. Prior to founding Cognilytica, Ron founded and ran ZapThink, an industry analyst firm focused on Service-Oriented Architecture (SOA), Cloud Computing, Web Services, XML, & Enterprise Architecture, which was acquired by Dovel Technologies in August 2011.