AI as a Service
SERVERLESS MACHINE LEARNING WITH AWS
PETER ELGER EÓIN SHANAGHY
MANNING SHELTER ISLAND
For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road PO Box 761
Shelter Island, NY 11964
Email: orders@manning.com
©2020 by Manning Publications Co. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.
Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.
Manning Publications Co.Development editor:Lesley Trites
20 Baldwin RoadTechnical development editor:Al Krinker PO Box 761Review editor:Ivan Martinović
Shelter Island, NY 11964Production editor: Deirdre S. Hiam
Copy editor:Ben Berg
Proofreader: Melody Dolab
Technical proofreader:Guillaume Alleon
Typesetter and cover designer:Marija Tudor
ISBN 9781617296154
Printed in the United States of America
For my parents, Noel and Kay Eóin
For my daughters Isobel and Katie, my parents Jacky and Julian, and my brother Jonathon Peter
forewordxiii prefacexv acknowledgmentsxvii about this bookxix about the authorsxxii about the cover illustrationxxiii
PART 1FIRST STEPS
1 A tale of two technologies3
1.1Cloud landscape5
1.2What is Serverless?7
1.3The need for speed8
The early days8 ■ The Unix philosophy9 ■ Object orientation and patterns10 ■ Java, J2EE, .NET,11 ■ XML and SOAXML (Extensible Markup Language) SOA (service-oriented architecture)12 ■ Web speed12 ■ Cloud computing13 Microservices (rediscovery)13 ■ Cloud native services14 The trend: speed15
1.4What is AI?18
History of AI18 ■ Real world AI19 ■ AI services22 AI and machine learning23 ■ Deep learning24 AI challenges27
1.5The democratization of compute power and artificial intelligence27
1.6Canonical AI as a Service architecture27
Web application28 ■ Realtime services28 ■ Batch services29 ■ Communication services29 ■ Utility services29 ■ AI services29 ■ Data services29
Operational support29 ■ Development support30 Off-platform30
1.7Realization on Amazon Web Services30
2 Building a serverless image recognition system, part 133
2.1Our first system34
2.2Architecture34
Web application36 ■ Synchronous services37 ■ Asynchronous services38 ■ Communication services39 ■ AI services41 Data services41 ■ Development support and operational support41
2.3Getting ready41
DNS domain and SSL/TLS certificate42 ■ Setup checklist44 Get the code44 ■ Setting up cloud resources44
2.4Implementing the asynchronous services47 Crawler service47
3 Building a serverless image recognition system, part 257
3.1Deploying the asynchronous services58 Analysis service58
3.2Implementing the synchronous services62 UI service62 ■ Front end service67
3.3Running the system71
3.4Removing the system74
2TOOLS
4 Building and securing a web application the serverless way79
4.1The to-do list80
4.2Architecture80
Web application82 ■ Synchronous services83 ■ Asynchronous services84 ■ Communication fabric85 ■ Utility services85
AI services85 ■ Data services86 ■ Development support and operational support86
4.3Getting ready86
Getting the code86
4.4Step 1: The basic application87
Resources89 ■ To-do service90 ■ Front end94
Deploying step 199
4.5Step 2: Securing with Cognito101
Getting the code103 ■ User service104 ■ To-do service107
Front-end service107 ■ Deploying step 2109
5 Adding AI interfaces to a web application115
5.1Step 3: Adding a speech-to-text interface116
Getting the code117
■ Note service117 ■ Front-end updates118 ■ Deploying step 3121 ■ Testing step 3121
5.2Step 4: Adding text-to-speech122
Getting the code123 ■ Schedule service123 ■ Front-end updates126 ■ Deploying step 4127 ■ Testing step 4127
5.3Step 5: Adding a conversational chatbot interface128
Getting the code129 ■ Creating the bot129 ■ Front-end updates132 ■ Deploying step 5134 ■ Testing step 5134
5.4Removing the system135
6
How to be effective with AI as a Service137
6.1Addressing the new challenges of Serverless138 Benefits and challenges of Serverless138 ■ A production-grade serverless template139
6.2Establishing a project structure142
The source repository—monorepo or polyrepo142 ■ Project folder structure143 ■ Get the code143
6.3Continuous deployment144 Continuous deployment design144 ■ Implementing continuous deployment with AWS services146
6.4Observability and monitoring148
6.5Logs150
Writing structured logs152 ■ Inspecting log output153 Searching logs using CloudWatch Logs Insights156
6.6Monitoring service and application metrics158
Service metrics158 ■ Application metrics159 ■ Using metrics to create alarms163
6.7Using traces to make sense of distributed applications164
Enabling X-Ray tracing165 ■ Exploring traces and maps167 ■ Advanced tracing with annotations and custom metrics168
7 Applying AI to existing platforms170
7.1Integration patterns for serverless AI171
Pattern 1: Synchronous API174 ■ Pattern 2: Asynchronous API175 ■ Pattern 3: VPN Stream In176 ■ Pattern 4 VPN: Fully connected streaming177 ■ Which pattern?178
7.2Improving identity verification with Textract179
Get the code180 ■ Text Analysis API180 ■ Client code182 Deploy the API183 ■ Test the API184 ■ Remove the API185
7.3An AI-enabled data processing pipeline with Kinesis185
Get the code187 ■ Deploying the API188
7.4On-the-fly translation with Translate189
7.5Testing the pipeline192
7.6Sentiment analysis with Comprehend193
7.7Training a custom document classifier196
Create a training bucket197 ■ Upload training data197
Create an IAM role198 ■ Run training198
7.8Using the custom classifier199
7.9Testing the pipeline end to end201
7.10Removing the pipeline202
7.11Benefits of automation202
PART 3BRINGING IT ALL TOGETHER ........................ 205
8 Gathering data at scale for real-world AI207
8.1Scenario: Finding events and speakers208
Identifying data required209 ■ Sources of data211 Preparing data for training211
8.2Gathering data from the web212
8.3Introduction to web crawling212
Typical web crawler process213 ■ Web crawler architecture214
Serverless web crawler architecture217
8.4Implementing an item store219
Getting the code219 ■ The item store bucket219 Deploying the item store219
8.5Creating a frontier to store and manage URLs220
Getting the code220 ■ The frontier URL database220
Creating the frontier API222 ■ Deploying and testing the frontier224
8.6Building the fetcher to retrieve and parse web pages224
Configuring and controlling a headless browser225
Capturing page output226
■ Fetching multiple pages227
Deploying and testing the fetcher228
8.7Determining the crawl space in a strategy service229
8.8Orchestrating the crawler with a scheduler231
Grabbing the code232 ■ Using Step Functions232 Deploying and testing the scheduler234
9 Extracting value from large data sets with AI238
9.1Using AI to extract significant information from web pages239
Understanding the problem239 ■ Extending the architecture240
9.2Understanding Comprehend’s entity recognition APIs241
9.3Preparing data for information extraction244
Getting the code244 ■ Creating an S3 event notification244 Implementing the preparation handler245 ■ Adding resilience with a dead letter queue (DLQ)246 ■ Creating the DLQ and retry handler247 ■ Deploying and testing the preparation service249
9.4Managing throughput with text batches252
Getting the code252 ■ Retrieving batches of text for extraction252
9.5Asynchronous named entity abstraction253
Get the code253 ■ Starting an entity recognition job254
9.6Checking entity recognition progress255
9.7Deploying and testing batch entity recognition256
9.8Persisting recognition results257
9.9Tying it all together258
Orchestrating entity extraction259 ■ End-to-end data extraction testing261 ■ Viewing conference data extraction results262
9.10Wrapping up263
appendix AAWS account setup and configuration265
appendix BData requirements for AWS managed AI services275
appendix CData sources for AI applications277
appendix DSetting up a DNS domain and certificate279
appendix EServerless Framework under the hood285
index291
foreword
For the past two decades, AI has played an increasingly significant role in our lives. It has done so quietly behind the scenes, as AI technologies have been employed by companies around the world to improve search results, product recommendations, and advertising, and even to assist healthcare workers to provide a better diagnosis. AI technologies are all around us, and soon, we’ll all travel in cars that drive themselves!
With this rise in prominence came a rise in demand for relevant skills. Engineers with expertise in machine learning or deep learning are often hoovered up by the big tech companies at huge salaries. Meanwhile, every application on the surface of the earth wants to use AI to improve its user experience. But the ability to hire the relevant skillsets and acquire the necessary volume of data to train these AI models remains a significant barrier to entry.
Fortunately, cloud providers are offering more and more AI services that remove the need for you to steep yourself in the art of collecting and cleaning up data and training AI models. AWS, for instance, lets you use the same technologies that power product recommendations for Amazon.com through Amazon Personalize, or the speech recognition technology that powers Alexa with Amazon Transcribe. Other cloud providers (GCP, Azure, IBM, and so on) also offer similar services, and it will be through these services that we will see AI-powered features in everyday applications. And as these services become better and more accessible, there will be less need for people to train their own AI models, except for more specialised workloads.
It’s great to finally see a book that focuses on leveraging these AI services rather than the nitty-gritty details of training AI models. This book explains the important concepts in AI and machine learning in layman’s terms, and describes them for
exactly what they are, without all the hype and hyperbole that often accompany AI-related conversations. And the beauty of this book is that it is way more than “how to use these AI services from AWS,” but also how to build applications the serverless way. It covers everything from project organization, to continuous deployment, all the way to effective logging strategies and how to monitor your application using both service and application metrics. The later chapters of the book are also a treasure trove of integration patterns and real-world examples of how to sprinkle some AI magic into an existing application.
Serverless is a state of mind, a way of thinking about software development that puts the needs of the business and its customers at the forefront, and aims to create maximum business value with minimum effort by leveraging as many managed services as possible. This way of thinking leads to increased developer productivity and feature velocity, and often results in more scalable, resilient, and secure applications by building on the shoulders of giants such as AWS.
Serverless is not the future of how we build businesses around software; it’s the present and now, and it’s here to stay. This book will help you get started with Serverless development and show you how to integrate AI services into a Serverless application to enhance its user experience. Talk about hitting two birds with one stone!
YAN CUI
AWS SERVERLESS HERO INDEPENDENT CONSULTANT
preface
The fourth industrial revolution is upon us! The coming decade will likely see huge advances in areas such as gene editing, quantum computing, and, of course, artificial intelligence (AI). Most of us already interact with AI technology on a daily basis. This doesn’t just mean self-driving cars or automated lawn mowers. AI is far more pervasive than these obvious examples. Consider the product recommendation that Amazon just made when you visited their site, the online chat conversation you just had with your airline to re-book a flight, or the text that your bank just sent you warning of a possibly fraudulent transaction on your account. All of these examples are driven by AI and machine learning technology.
Increasingly, developers will be required to add “smart” AI-enabled features and interfaces to the products and platforms that they build. Early adopters of AI and machine learning have been doing this for some time, of course; however, this required a large investment in research and development, typically requiring a team of data scientists to train, test, deploy, and operate custom AI models. This picture is changing rapidly due to the powerful force of commoditization.
In his 2010 bestselling book, The Big Switch, Nicholas Carr compared cloud computing to electricity, predicting that eventually we would consume computing resources as a utility. Though we are not quite at the point of true utility computing, it is becoming clearer that this consumption model is fast becoming a reality.
You can see this in the explosive growth in the range and capability of cloud-native services. Commoditization of the cloud stack has given rise to the serverless computing paradigm. It is our belief that serverless computing will become the de facto standard architecture for building software platforms and products in the future.
In conjunction with the commoditization of the wider application stack, AI is also rapidly becoming a commodity. Witness the number of AI services that are available
from the major cloud providers in areas such as image recognition, natural language processing, and chatbot interfaces. These AI services grow in number and capability month by month.
At our company, fourTheorem, we use these technologies on a daily basis to help our clients extend and improve their existing systems through the application of AI services. We help our clients to adopt serverless architectures and tools to accelerate their platform development efforts, and we use our experience to help restructure legacy systems so that they can run more efficiently on cloud.
It is the rapid growth and commoditization of these two technologies, Serverless and AI services, along with our experience of applying them to real-world projects, that led us to write this book. We wanted to provide an engineer’s guide to help you succeed with AI as a Service, and we wish you luck as you begin to master this brave new world of software development!
acknowledgments
Ask any technical book author ,and they will tell you that completing a book takes a lot of time and effort. It also requires the fantastic support of others. We are incredibly grateful for the many people who made completing this book possible.
First we would like to thank our families for their support, understanding, and patience while we worked to complete the book. Eóin would like to thank his amazing wife, Keelin, for her unending patience, moral support, and indispensable technical reviews. He would also like to thank Aoife and Cormac for being the best children in the world. Peter would like to thank his daughters, Isobel and Katie, just for being awesome.
Eóin and Peter would like to thank fourTheorem co-founder Fiona McKenna for her belief in this book, and her constant support and expertise in so many areas. We could not have done it without you.
Starting a project like this is the hardest part, and we are grateful for the people who helped in the beginning. Johannes Ahlmann contributed ideas, writing, and discussion that helped to shape what this book became. James Dadd and Robert Paulus provided invaluable support and feedback.
We would also like to thank the awesome team at Manning for making this book possible. In particular, we want to thank Lesley Trites, our development editor, for her patience and support. We would also like to thank Palak Mathur and Al Krinker, our technical development editors, for their review and feedback. Thank you to our project editor, Deirdre Hiam; Ben Berg, our copyeditor; Melody Dolab, our proofreader, and Ivan Martinović, our reviewing editor.
We would like to thank Yan Cui for writing the foreword to this book. Yan is an outstanding architect and champion of all things serverless, and we are grateful for his endorsement.
A big thanks to all of the reviewers for their feedback and suggestions for improvement to the text and examples: Alain Couniot, Alex Gascon, Andrew Hamor, Dwight Barry, Earl B. Bingham, Eros Pedrini, Greg Andress, Guillaume Alleon, Leemay Nassery, Manu Sareena, Maria Gemini, Matt Welke, Michael Jensen, Mykhaylo Rubezhanskyy, Nirupam Sharma, Philippe Vialatte, Polina Keselman, Rob Pacheco, Roger M. Meli, Sowmya Vajjala, Yvon Vieville,
A special thanks to Guillaume Alleon, technical proofreader, for his careful review and testing of the code examples.
Finally we wish to acknowledge the broader open source community, of which we are proud to participate in. We truly do stand on the shoulders of giants!
about this book
AI as a Service was written as an engineer’s guide to building AI-enabled platforms and services. The aim of the book is to get you up and running, and able to produce results quickly, without getting stuck in the weeds. AI and machine learning are big topics, and there is an awful lot to learn if you wish to master these disciplines. It is not our intent to discourage anyone from doing this, however if you need to get results quickly, this book will help you get up to speed.
The book examines two growing and increasingly important technologies: serverless computing and artificial intelligence. We examine these from a developer’s perspective to provide a practical, hands-on guide.
All of the major cloud vendors are engaged in a race to provide relevant AI services, such as
Image recognition
Speech-to-text, text-to-speech
Chatbots
Language translation
Natural language processing
Recommendations
This list will only expand over time!
The good news is that you do not need to be an AI or machine learning expert to use these offerings. This book will guide you in applying these services in your day-today work as a developer.
In tandem with the grown of AI services, it is now possible to build and deploy applications with a minimum of operational overhead using the serverless approach. Our
Another random document with no related content on Scribd:
“Quocumque loco me vertam, semper se oculis meis cum suis ingerunt desideriis. Nec etiam dormienti suis illusionibus parcunt. Inter ipsa missarum solemnio, obscæna earum voluptatum fustasmata ita sibi penitus miserrimam captivant animam ut turpitudinibus illis magis quam orationi vacem. Quæ cum ingemiscere debeam de commissis, suspiro potius de amissis; nec solum quæ egimus, sed loca pariter et tempora in quibus hæc egimus ita tecum nostro infixa sunt animo, ut in ipsis omnia tecum agam, nec dormiens etiam ab his quiescam. Nonnunquam et ipso motu corporis, animi mei cogitationes deprehenduntur, nec a verbis temperant improvisis ... castam me prædicant qui non deprehenderunt hypocritam.”[26] ...
These expressions, scarcely equalled by the delirium of Sappho, succeed at length in rekindling the expiring passion of Abelard. He replies by quotations from Virgil, from Lucanus, and by passages from the Song of Solomon. To convince her that their sorrows are not unmerited, he reminds her on his side of their past pleasures, and among others, of a sacrilegious interview held in the refectory of the convent of Argenteuil, where he had visited her in secret.
He then, and more than once, enlarges in praise of eunuchs, and ends by enclosing a prayer he has composed for her and for himself.
This closes the amorous correspondence, for in the next letter Eloisa declares her resolution, to which she remains firm, of putting a restraint on the ardour of her feelings, although she cannot at the same time refrain from quoting some equivocal lines from Ovid’s Art of Love.
We must here once more ask whether, circumstanced as these two lovers were, and taking into consideration the piety and resignation apparent in all the writings of Abelard, he being at the time fifty-four years of age, and Eloisa thirty-three—and after fourteen years’ separation, it is credible or possible that the letters we have quoted, letters in which all modesty is laid aside, should have been written by Eloisa? Allowing that she had preserved Abelard’s correspondence, is it easy to suppose that Abelard, continually moving from place to place, should have preserved hers to the day of his death, so that
their letters might eventually be brought together?—letters, too, breathing an ardour so compromising to the reputation of both?
Is it likely that Eloisa should have kept copies of her own letters, the perusal of which, it must be confessed would not have tended to the edification of the nuns?
Remember also that all these events occurred in the first half of the 12th century, in an age when it was very unusual to make collections of any correspondence of an amorous nature.
We can then only arrive at the same conclusion as Messieurs Lalanne, Orelli, Ch. Barthélemy, and others, viz. that the correspondence which has given such renown to the names of Abelard and Eloisa as lovers, is in all probability apocryphal.
M. Ludovic Lalanne has another supposition, which is curious, and which appears to us not to be impossible:
“These letters,” says he, “are evidently very laboured. The circumstances follow each other with great regularity, and the vehement emotions that are traceable throughout, do not in any wise interfere with the methodical march of the whole. The length of the letters, and the learned quotations in them from the Bible, from the fathers of the church, and from pagan authors, all seem to indicate that they were composed with a purpose and with art, and were by no means the production of a hasty pen. Eloisa, we must remember, was a woman of letters, and a reputation for learning was of great value in her eyes. Did she, who survived her lover upwards of twenty years, wish to bequeath to posterity the memory of their misfortunes, by herself arranging and digesting at a later period, so as to form a literary composition, the letters that at divers times she had written and received? Or has perhaps a more eloquent and experienced pen undertaken the task? These are questions difficult to resolve. Anyhow, the oldest manuscript of this correspondence with which we are acquainted, is upwards of a hundred years posterior to the death of Eloisa. It is, as we have already said, the manuscript of the library of the town of Troyes.”
Let us now proceed to examine the authority for the so-called tomb of these lovers in the cemetery of Père la Chaise.
Two learned archæologists will enlighten us on the subject. Monsieur Lenoir,[27] in his Musée des Monuments français, and Monsieur de Guilhermy, in an article of the Annales Archéologiques de Didron for 1846.
During the French Revolution of 1792, the convent of the Paraclete, founded by Abelard, was sold. In order to protect the remains of the lovers from desecration, which was too common in those days, some worthy inhabitants of Nogent-sur-Seine, took possession of the coffins and deposited them in the church of that town. Seven years later M. Lenoir obtained the permission of the minister to transfer these remains to Paris, and it occurred to him at the same time, that it would be expedient to enclose them in a tomb of the period in which the lovers had lived. He was told that in the chapel of the infirmary of Saint Marcel-les-Chalons, Peter the Venerable had erected a monument to Abelard. Several denied this fact; but be that as it may, Monsieur Lenoir obtained possession of part of this monument, which had been purchased by a physician of the town in order to save it from destruction. M. Lenoir then constructed a monument with the fragments of a chapel of the abbey of St. Denis, and, as he tells us, placed the sarcophagus, which was of the style of architecture in vogue in the 12th century, in a room of the museum entrusted to his care.
The following information given by M. de Guilhermy[28] will show us how far M. Lenoir succeeded in his architectural device, and how far the sarcophagus contains the actual remains of Abelard and Eloisa:
“How many illusions,” says M. de Guilhermy, “would vanish into thin air if the pilgrims who came to visit the shrine of these celebrated lovers in the cemetery of Père la Chaise only knew, that in the construction of the sepulchral chapel there is not one single stone from the abbey of the Paraclete. The pillars, the capitals, the roseworks, which decorate the facings of the tomb belonged to the abbey of St. Denis. It does not require a very practised eye to discover that the sculptures are not in harmony, and were never intended to form
a whole It was the former director of the Musée des Monuments Français, who conceived the idea of putting together some fragments placed at his disposal, and with these to erect a monument worthy of receiving the bones of the two illustrious lovers of the 12th century.
“A wooden case sealed with the republican seal of the municipality of Nogent-sur-Seine, carried to Paris in 1799 the remains which were taken out of the grave in the Paraclete; but before depositing them in their new asylum, it was thought necessary to satisfy the amateurs of relics of this nature. The republicans opened the box, and all that was left of the bodies after a period of six hundred years was stolen out of it.” M. de Guilhermy says that: “Actually a tooth of Eloisa was offered for sale at the time. At any rate it was in the following manner that the tomb of Abelard was completed. A bas-relief which represented the funeral procession of Louis, the eldest son of Louis IX. of France, was taken from St. Denis, and it was decided that for the future this piece of sculpture should do duty for the mausoleum of Abelard. Two medallions, the work of a second-rate artist of the 16th century, represented Abelard with curled mustachios, and Eloisa under the form of a half-naked woman.”
“But this is not all. On the sarcophagus are two recumbent figures. One is draped in priestly robes and was purloined from one of the numerous cloisters demolished in Paris; the other is the statue of some noble lady in the costume and style befitting the 14th century, which once reclined on a tomb in the chapel of St. Jean de Beauvais in Paris.”
It is as well to recall such details as these in order to expose errors which, unless refuted, would from their long standing end by being accepted as truths. But after reading all the circumstances narrated above, can it be believed that Monsieur Guizot, who is so well acquainted with the real facts, or who at any rate ought to be acquainted with them, should, in order to gratify the public taste for sentiment, write as follows in the preface to a translation of the letters of Abelard and Eloisa:[29]
“Vingt-et-un ans après la mort d’Abailard, c’est-à-dire en 1163, agée de 63 ans, Héloïse descendit dans le même tombeau. Ils y reposent encore l’un et l’autre, après six cent soixante-quinze ans, et tous les jours de fraiches couronnes, déposées par des mains inconnues, attestent pour les deux morts la sympathie sans cesse renaissante des générations qui se succèdent!”
It would be difficult to find a more inflated style with which to decorate an historical error.