Introduction to AWS Services

Introduction to AWS Services

Introduction to AWS Services

hi all this is Hisham welcome to gulfwebhosting so, in this lecture, we are going to talk about aw is basic services so if you are not familiar with AWS and you don’t know much about AWS services this lecture is for you ok so before starting this lecture I want you to understand few things about AWS first is AWS global data centers so whenever we are we will be using some AWS services typically we will deploy it in some AWS geographical area now that geographic area is called AWS

regions like across the world there are different AWS regions available for us like in u.s. there are seven regions in India there is one region in Europe there are a couple of ofreasonsregions and in all, there are 20 regions at the moment till date and 5 more reasons coming soon in 2019 and when you deploy services we can choose which regions we need to deploy in now every region

further is comprised of typically two or more data centers that are for high availability of AWS services and those data centers are called availability zones we will learn more about it shortly also as in AWSglobal data centers there is something called edge locations now each location are

something like you can consider it like and caching devices which are there across100 plus cities across the world and your content like your media videos and pictures what you watch may be on theFacebook or YouTube they get cached to the nearest location and from there itis

delivered to the user so it basically improves the performance by lowering the latency network latency right so overall AWS has 130 plus services if you heard about see 2 s 3 these are like different AWS services and we are going to learn more about these services in this lecture ok so as

I said the region is one geographic area here the blue area you see is an AWS region and every region consists of typically two or more availability zones for high availability of your application so when you design your architecture typically you will keep your machines in different Easy’s so that

if one of the easy goes down for some reason you have your machine running in another easy and your application then have high availability okay so we’ll talk about more this in ec2stations which is a different course but as of now you just need to know about these things ok I hope at the region

and availability zone you’re familiar with now hope at the region and availability zone you are familiar with now let’s move ahead and now I want to talk about AWS services so before that just a quick overview of how these services regions and easy really map each other so first thing if you

have an AWS account AWS account is a top-level entity that means once you have an AWS account you can deploy your infrastructure in any of the AWS regions so as I said there are 20regions as of now and every region then for the comprises of two or moreEasy’s that is what’s

shown here now in AWS there are different services and they have a different scope with respect to the region or ez-zor account level, for example, say billing service it works at an account level that means at the end of the month you get one AWS bill which you have to pay I am which is

identity and access management it also works account level which means how many users you want to create you can create that and all these users would have access to all AWS regions and AZ’s and the services because of they work right and there are more services we’ll talk about shortly

and then some other services like s3 DynamoDB they work region level that means when you creates3 bucket you select in which region you want to create that s3 bucket right similarly dynamodb tables so and then there are further services like ec2which is a VM RDS databases

elastic block storage like a disk all this works at the easy level the scope of this services is a level that means one ec2instance cannot be in two Easy’s at the same time it would be either in a z1 orz2 or AZ 3 depending on where we are launching that machine same with the databases and the

disk so we will see more services but from this, I want you to understand different AWS services works at a different level and this is a scope where AWS account is a top-level entity under which we have AWS regions and then we have Easy’s in given AWS region now let’s move to

AWS services now there are so many AWS services as I said there are 130 plus AWS services and we can broadly categorize them into different kind of computing power or analytic services like this so in computing there are ec2 auto scaling lambda load balancers container service likewise for

data analytics there is sayEMR which is Hadoop service kinasesAthena so rather than talking about this services now in this fashion I would like to take some example and then so that you can map really how this fits into some architecture and that probably would help you recall what service

is used for what ok similarly there are other categories like storage services and databases services then there are some network related services and management services further you have application services and development services as well so still it does not really take care of all

AWS services but we have listed the widely used AWS services the popular AWS services okay with this what I want to do next is I want to build one application and we will see how to create the same architecture using AWS services so what we want to do is now to understand different AWS

services where they fit into any architecture we want to build a simple social media application may be a mini version of Facebook or an Instagram and then we will see how to design the same architecture using different AWS services okay so our application is FBcom for example we

our users will access it using this name so first thing if you want to deploy this application or in your on-premise data centers then the first thing you will need is one private network like every company has their private network we would also require something like this to make it secure

of course, the next thing you would require is a web service now to start with supposing we are a startup then we will probably build a small code in maybe PHP and we will run in some kind of application server or run the webserver and it should work maybe for at least 200users or lower

than that and it works fine and our users will access this application using IP address initially so maybe this VM has some public IP and users access it now what happens over the time is like you you want to now extend your application and you want to add some business logic some UI

stuff the login functionality and more so that’s where you need to then have a web server as well as application server so that all the front-end stuff is taken care by web server but all business logic suppose it’s a Facebook kind of application then maybe they are you connect with different

people adding that data and everything is taken care by the application server and of course further if you want to extend it you need some kind of database like relational database my sequel or an even you can have Oracle’s whatever you prefer right so if you have this kind of application it works

well it’s installed three-tier architecture and your users are using this application using an IP address right so this works well and considering the app is really doing good your website is really doing good and there is more traction from the users and somewhere then your webservers or an

application servers bottom becomes bottleneck maybe they are not able to handle the increased load on your application so what’s the solution typically we will scale now that scaling can happen vertical scaling that means you increase the capacity of these machines or it could you could

horizontal scaling so typically in three-tier architecture, you will see web servers and application servers are scaled horizontally that means you will bring more web servers and more application servers right as I have shown here okay that’s fine now I have multiple web servers and

multiple application servers but as you know there are multiple web server that means there are multiple IP addresses and now is the time where we need an intelligent entity who can really distribute the load to this web service and that’s where webring in the load balancer service so if you

heard about load balancers like a cheap proxy and engine inks they do something like this a user hits the request to the load balancer then and then evenly distribute that to aback-end servers like this and as you know now we have load balancers also and your application is really catching

typically you don’t want your application to be accessed using the IP address you want people to access your application with the domain name something called say a bit calm and that’s where you need some DNS service where you can map your DNS this domain name to load

balance probably right okay so so far so good this works fine right your application is three-tier and it is working well now catches a further and you are now having a lot of data or say you have a number of friends or the growing number of connections or the growing number of posts are growing

and that’s where your relational database cannot really solve this kind of maybe data storing data you can not do that in relational databases for this you need a scalable database and also for connection information and all it makes sense to rather going for no sequel databases so what you

will do bring in the no sequel database like MongoDB or Cassandra anything that you want to have so some part of data is stored in relational databases and other is storing non-relational or no sequel databases but still, your relational databases could be a performance bottleneck maybe

there are very read-heavy operations happening on this database and for that typically you will bring in one more component which is called database caches ok so you bring in some database cache engines like radius or a Memcache T where you can query the frequently

accessed data so that your application servers don’t hit the database but all the requests are served from this cache engine ok so this is fairly better architecture what we saw where we started with now next thing as you know Facebook might be getting millions of pictures uploaded

daily and the videos daily now this disk which is attached to the VM are not really capable of extending on the fly they have size limitations and that’s where all these media pictures are never stored typically on this web servers or application servers for you need for this you need

some unlimited kind of storage and that’s external storage and it could be it should not necessarily block storage like your disk can be file storage like a sheer filesystem or something or some external storage like Google Drive if you are aware of right so you need some

externalstorage where you store this informationokay so that makes your storage thatsolves you a storage capacity problem ifyou use external storage that’s fine sofar so good now next what happens iswhen you upload a videos or photos youneed some kind of content filters likemaybe

oh you are uploading videos andthat video has some some content whichare objectable or there are somepictures we have which has some nudityso you need some content filter whichcan do this on the fly and then thosepictures we do should be actually storedhere in the external storage

so we bringin one more component there rightokay that’s fine now you also knowFacebook also throws a lot of ads and itis continuously watching what activitiesyou are doing on while you are on theFacebook page or maybe what kind ofproducts you are liking what kind ofposts you are liking

and based on thatit gives you suggestions and the friendrequest will throw a lot of ads right sothis is called clickstream analysisevery click is getting capturedsomewhere and it is getting analyzed inreal time so you need some kind ofclickstream analysis engine there rightlet us take an

example Twitter what alltweets are going on in the market whatis the mood of the people currently allthis is done using the clickstreamanalysis on Facebook also you havesomething like this now all this datawhat this clickstream analysis enginecaptures it has to further

storesomewhere in the external storage rightand you need an external storage forthis like this storage for storing thisdata and further you want to take thisdata and do some data operations likeyou need to run some maybe aggregationsyou need to sort your data and you tofind some

meaning out of that data andthat’s where you need some kindof Hadoop platform which can perform thecomputing on distributed systems rightso you need some kind of a Duke platformand you would also require over the timeone data warehousing why because maybeat the end of the

year or Facebook doesa lot of data analytics right maybe atthe end of the year they want to findout which kind of users are accessingFacebook more what are their age limitwhat are their age in which region theycome from how particular feature ofFacebook date so that they

canconcentrate more on those kind offeatures what is trending all thisinformation is taken out by storing thisinformation in some kind of datawarehousing engine and then doing somekind of business intelligence on top ofit so you need some businessintelligence tool which can query

thisdata analyze this data and then thereare reports generated out of which thenFacebook can take decisions like nextyear maybe this is our strategy or wewill focus on this area or that area sosome business decision you can drivebased on what analytics results come outof this ok so

this all about this ismore on a back-end size which end-userdoes not really know but this ishappening there ok so far so good so wehave extended our architecture now nextwhat we have is all these photos andvideos they can be directly served overthe internet because you consider

thisas like a Google Drive so you candirectly maybe stream your videos andwatching pictures directly from thisstorage so users might come from the webbrowser and they may watch whatever postsuppose you have posted a video so theycan watch that video here but

sometimesyour users come from using mobiledevices nowadays they will watch yourvideos through mobile and in that caseyou need the same videos but in probablydifferent format that’s because mobiledevice might play a different format ofthe video and for this typically we willneed

some kind of video converter inbetween so whenever anyuser upload some videos maybe theyshould be immediately converted into amobile friendly format all right so youneed some kind of computing power hereas well okay so we will introduce thatas an Video Converter here next

allthese photos and videos are typicallyserved from as I said from the externalstorage but you know whenever some videoget viral right millions of users watchthat video now every time if that videois fetched from this location this willthis might become a bottleneck or youmay pay a price

because your data isflowing out to the Internet and thereare a lot of usage of your videos soforth to solve this problem you need tohave something called CDN contentdelivery network which is nothing butwhich caches these videos and picturesto the nearest caching devices fromwhere the

user is accessing your videosright so that all the users in thatgeography when they want to watch thesame video it is served from here it isnot really served from here so userexperience the low latency and betterexperienceso in applications like Instagram andFacebook or and

YouTubelargely they would have lot of contentdelivery networks through which thecontents are solved okay so so far sogood we have extended architecturefurther now you know Facebook also sendsyour mobile notifications right there isa new friend request or there is a likeson your post

now for this we need andsome kind of notification service rightmaybe you get an SMS or mobile pushnotifications so you need that servicealso it sends you emails right forvarious activities you can disable thatbut yeah there is options to off foremail service as wellright and further you can

also chat withyour friends and for this typically aqueue is used no messaging queue if youheard about like RabbitMQ Gemma’s queuesIBM MQon cue services which enables the kindof first-in first-out and that kind ofdata structure so for chatting maybe yourequire some kind of cue service

as wellokay so if we consider all theseservices it’s a bare minimum kind ofsocial media application I’m sure theremust be much many more components but weare just sticking to this as of now andfinally if you want to deploy thisarchitecture and monitor it continuouslylike how my

VMs are doing how mydatabases are doing how my storage isdoing how much storage is there all thisyou need some kind of monitoring serviceand a dashboards like productiondashboard where you can monitor healthof your application okay so overall thiswill be your architecture and

thisprobably be deployed on on premises andnow let’s see if you want to do the samething on AWS then how we will do this wewant to do this now on AWS so let’s seefirst thing this private network whatyou see here in AWS world it is calledbe PC virtual private cloud so it is notexactly

the way it is shown here becauseall some of these services are outsideVPC but I cannot accommodate that in adiagram but consider be PC has oneprivate isolated network that AWS givesyou and then you would have to manageall the public curl network for webservers and load

balancer and a privatenetwork for databases that is a separatepart of discussion but the VPC is anetwork service now all these VMs thatwe are talking about these are nothingbut ec2 machines right and the disk thatwe attached it’s called EBS elasticblock storage and they have limitationof

maximum size so easy to any be asolves your problem of the VMS thattypically will deploy your applicationson whether web servers or app serversnow further you can have an auto scalingenabled for easy tools that means if theload increases on this ec2 they canscale

horizontally automatically and ifthe load decreases they can scale downmaybe from two machines they can go- ten machines from 10 they can comeback to two machines depending on theload that you can configure using autoscaling feature of AWS ec2 further fourdatabases

relational databases there isa service called RDS and for no sequeldatabases there is a service calleddynamo DB for DB caches there is aservice called elastic caches serviceand it comes with a Redis and memcachethe engines in that okay further as yousee there is a load balancer so



inAmazon there is a service called ELBelastic load balancer service which cutdistwhich can distribute the incomingtraffic to multiple back-end ec2machines like this and for that if youwant to have your domain name mapping toyour load balancer then you need a DNSservice which is

called route 53 okgreatnow let’s talk about the other stuffthat we have like for external storageit is an s3 service of Amazon simplestorage service right which tick whichis unlimited storage you can just go ondumping the data and it is accessibleover the internet directly and there isno size

limitation how much data you canstore in your s3 buckets also you needsome content filter so there isn’tservice called recognition which canfind out an object able images and itcan filter it out before you upload itto the say s3 buckets okay now as I saidyou need some kind of service where

yourvideos from one format get converted toanother format like mp4 to some mobilefriendly format now for this one optionis you run some ec2 machines whichcontinuously watch your s3 buckets fornew videos as the new video comes theydownload it here convert it and put itback into

another bucket that’s oneoption but there isn’t better option forthis like a lambda service no lambda isa service service of Amazon where youjust write a code in that code youspecify how to maybe convert a video andyou can execute this lambda functionwhenever there is a new

uploadhappening into your s3 so new videocomes lambda gets triggered it willconvert your video and maybe you haveput in logic that put that video in toanother s3 bucket so now here there areno servers to manage everything is takencare by lambda functions and this

scaleautomaticallyokay so we got lambda there now let’stalk about this clickstream analysis nowfor clickstream analysis there is aservice called kinases which cancaptures your click stream data and thenyou can analyze that data you can evenfor the store that data in s3 and youcan do

much more with whatever data youcapture right now for this park orHadoop platform there is a servicecalled EMR and for iya models likeoperations like aggregation sorting andyou can run distributed jobs SPARC jobsfilling jobs all this you can run inthis manage Hadoop cluster and you

alsoneed to do ETL transactions from yourDynamoDB tables like maybe you want todo what all friends are their friendsfriend what activities they are doingthat you want to continuously push newpost on your wall now all this is donein real time using clickstream analysisand at the end of

the year maybe youwant all this data to be extractedconverted into different format datacataloging and then further do some dataprocessing using EMR so you need thisglue service for doing this extracttransfer transform and load operationsETL operations right and then finallyall this data

what use process or whatdata you have you can store it in thedata warehouse service which is nothingbut redshift in Amazon so redshift is adata warehousing service which can storepetabyte

scale of data and they can youcan perform the analysis on the data andto perform this analysis and see theresults you need some BI tools whichlike there are various BI tools in themarket but in Amazon you will use Amazonquick site or you can also use awhich is an SQL query interface so

youcan pull data from s3 perform maybe anSQL operation on that and all thoseresults can be viewed in a quick siteyou build some graphs some charts andyou get insides of your data based onthat you will take some businessdecisions so it’s a bi service fromAmazon ok so far so good

we introducedlot of AWS services here now let’s moveto this side now as I said there is acontent delivery network which can cachea you’re a static content and for thisin Amazon there is something calledcloud front service and CloudFrontstools or caches your data in agelocations like

I said their edgelocations are there in across citiesacross 100 plus cities across the worldand when you use cloud front service allyour data from s3 or whether your storyor data it gets it get cached in thenearest edge location from where theuser is coming and the data is alwaysserved

from that edge location for allthe users in that geography ok so that’sa cloud front service now let’s talkabout this side also as I said you needto send an messages and mobile pushnotification in Amazon that is a servicecalled SNS simple notification servicefor that and if you want to send

emailsa bulk email then there is an SESservice simple email service now formessaging queues for chattingapplication Amazon has built its ownqueue service which is called sqs simplequeue service and finally to monitor allthis infrastructure how my EC tools aredoing was the CP

utilization of EC toolshow is database is doing all these canbe monitored in real time using aservice called cloud watch even you canset alarms like if an average siputilization goes beyond say this personsend an email or older to administratoror take some action do some auto

scalinghere all this can be doneusing this cloud watch alarm there okayso I think we have completely replacedwhat we did on premises with all withAWS services and I hope you got someidea about all these basic AWS servicesokay next we want to see some more AWSservices

and let’s see some applicationservices now as you know it’s a Facebookor Twitter or any other web services oreven Amazon itself it exposes all theirservices through API calls so thatdifferent third-party application canintegrate with these applications andfor that they need an REST API

servicewhere they can expose all their api’s soin Amazon you can have managed APIgateways very takes care of scalingthrottling everything so you just writea code for your api’s definitions ofyour api’s and api gateway it can bedeployed in api gateway also as themobile usage is

increasing most of yourusers the web users you need to managetheir identities like when you developan application you must sign up yourusers must sign up to your applicationright and that means you need to manageyour user pools their accesses andeverything and for that you need

someuser management service so in AWS thatservice is called cognitive right sothese are more application services thatwe can use here now let’s move head andtalk about the security services in thisarchitecture now as you know there isone primary service for managing allaccesses in your

AWS like all your AWSusers what access they have whatservices they can use even when say oneAWS service like ec2 wants to upload adata to s3 then easy to needspermissions to do thatnow all these accesses and authorizeattenti keishon and authorization ismanaged using

Amazon’s Iem service identity and accessmanagement it’s one of the mostimportant service for securing your AWSaccount as well as services next whatyou can also do is you can encrypt yourdata which is there which is stored atvarious storage locations like EBS is ablock storage like

a disk attached tothe ec2 you can encrypt that data datawhich is stored in s3 which is stored inEMR redshift Q messages databases cachesall this data you can encrypt usingAmazon’s kms key management service soit manages all the encryptions key foryou you don’t need to have your

ownsecure location where you can store yourkeys and do the encryptions further asyou know this application will beaccessed probably over HTTPS which isSSL enabled connection because obviouslyif users are doing some transactions orthey don’t want to lose their importantinformation

you would secure thatcommunication and for this you needdigital certificates right so that youeither deploy on load balancers or youmay deploy it on cloud front so thatyour communication is secures for thisAmazon has a service called ACM Amazoncertificate manager okaynext as you

know we can also have theapplication firewallsnow those application firewalls arecalled BAFF Web Application Firewallnow that take care of any attacks it canprevent like cross-site scripting SQLinjections even the DDoS attacks whichare happening RAF can protect yourapplication from

this attacks and youwill typically deploy it on cloud frontor in back also on load balancers or infront of your API gateways that we sawin earlier side so that you are safe andother various ways is to secure VPC thepublic and private that will seein detailed vp cessation the networkingin

a Douglass lectures but here we aretalking about application levelfirewalls so that’s laughs and if you’rethis machines if you are going for somekind of compliance for example PCI DSScompliance or say you’re going for anHIPAA compliance so your machines needto be

patched properly they should befree from vulnerabilities right or CVEas you know and for that there is aservice called AWS inspector what itdoes it puts an agent inside yourmachines and it scans your machine forany known vulnerabilities and then itwill give you reports saying like

youknow all these machines out of thesemachines we found these vulnerabilitiesgo and fix those so inspector can giveinsights about what’s there inside ourmachines okay so these are primarilyused of security services there are morebut I think will restrict ourdiscussions to only

these services as ofnow next we want to see some developmentand DevOps services now as you see thisarchitecture it has lot of AWS servicesand all are connected so when you wantto deploy everything by handmaybe manually I I would say it willtake maybe couple of days to do

thiswithout making any errors or ordetecting the errors and fixing it allthis has to be done manually then itwill take two or three days probablybut with AWS it gives you ability tocode your infrastructure that’s calledinfrastructure as a code so you can havean service like cloud formation

what itdoes it takes kind of a template formfrom you which is in JSON or Tamilformat and it will just create thisinfrastructure from scratch for you andthat to within maybe 30 minutesdepending on what size you have buttypically I have seen like 30 30 minutesmaximum it will create all

theseresources for you it’s a very powerfulservice which can provision yourinfrastructure from the scratchright and now this cloud formationtemplate will be written by sometear-offs people and at the same timeyou would have your developers and a QAwho developers are writing code for

yourproduct and maybe Q is are writing or QAtest cases automation test cases noweverybody needs some kind of coderepository like a get code repositoryfor that AWS has a core commit servicewhere they can check in the code so eventhis cloud formation template is nothingbut a

JSON or a Yaman code so these guysyour dubs guys will write that as antemplate cloud formation service willtake that template and create thisinfrastructure now once you have this instress in infrastructure up you requireyour actually product to be build andfor that you need

code build service soAmazon code build will take the sourcecode with your language you have writtenin Java or whatever and it will buildthat using some kind of build tool likeant or maven and it produces also whilebuilding it will do some unit tests andfinally it will produce some artifactsno

artifacts are like your exe Zorobinaries actually your applicationexecutables basically so the code willwill do that will test it and then youhave to deploy this that means whateverit produces you have to put these X’sand binaries in ec2 machines where yourapplications is actually running so

youwill require a deployment and for thisyou have a code deploy service alrightso if you know about the DevOps youheard about the term CI and the CDs sothis is your CI pipelines continuousintegration pipeline or a continuousdelivery pipeline you can say and if youwant to

have this automated like youknow developers are writing the codechecking in in it automatically getsbuild it automatically tested andautomatically deployed intocorresponding application serversrunning in ec2 then you can have a corepipeline service right so you cancompletewe build

your CI CI platform here usingthis three and a code pipeline servicenow if you want to further integrate allthese things with project managementtools like maybe a JIRA some bugtracking tool how you are how yourwhat’s the speed of your development andall the management tools now it

iscalled a course star service which verywell integrates with Atlassian JIRA andother tools so you have complete sdlccontrol now if you use these of thesedevelopment and the devops services okayso I think this is clear now where thesedevelopment and deployment services areused okay

so if you have come up to thisyou know about most of the AWS coreservices now for compute analyticsstorage security application anddeployment services thank you

read more

What is AWS amazon web services