Transcription
Building and deploying microserviceswith event sourcing, CQRS and DockerChris RichardsonAuthor of POJOs in ActionFounder of the original nethttp://[email protected]
Presentation goalShare my experiences with building and deploying anapplication using Scala, functional domain models,microservices, event sourcing, CQRS, and [email protected]
About [email protected]
About ChrisFounder of a buzzword compliant (stealthy, social, mobile, big data, machinelearning, .) startupConsultant helping organizations improve how they architect and deployapplications using cloud, micro services, polyglot applications, NoSQL, .Creator of http://[email protected]
AgendaWhy build event-driven microservices?Overview of event sourcingDesigning microservices with event sourcingImplementing queries in an event sourced applicationBuilding and deploying [email protected]
Let’s imagine that you are building abanking [email protected]
Domain t(amount)@crichardson
Traditional application architectureWAR/EARSpring MVCACIDBanking countsSpringHibernateTransfersRDBMSCustomersSimple [email protected]
Problem #1: monolithic architectureIntimidates developersObstacle to frequent deploymentsOverloads your IDE and containerObstacle to scaling developmentModules having conflicting scaling requirementsRequires long-term commitment to a technology [email protected]
Solution #1: use a microservice architectureBanking UIStandaloneservicesAccount Management ServiceMoneyTransfer ManagementServiceAccountDatabaseMoneyTransfer [email protected]
Problem #2: relational databasesScalabilityDistributionSchema updatesO/R impedance mismatchHandling semi-structured [email protected]
Solution #2: use NoSQL databasesAvoids the limitations of RDBMSFor example,text searchSolr/Cloud Searchsocial (graph) dataNeo4Jhighly distributed/available [email protected]
Different modules use differentdatabasesIEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @[email protected]
But now we have problems with [email protected]
Problem #3: Microservices distributeddata managementEach microservice has it’s own databaseBusiness transactions must update data owned by multiple services,e.g. Update MoneyTransfer and from/to AccountsSome data is replicated and must be kept in syncTricky to implement reliably without [email protected]
Problem #4: NoSQL ACID-free, denormalized databasesLimited transactions, i.e. no ACID transactionsTricky to implement business transactions that update multiple rows,e.g. Update MoneyTransfer and from/to Accountse.g. http://bit.ly/mongo2pcLimited querying capabilitiesRequires denormalized/materialized views that must be synchronizedMultiple datastores (e.g. DynamoDB Cloud Search ) that need to be kept in [email protected]
Solution to #3/#4: Event-basedarchitecture to the rescueMicroservices publish events when state changesMicroservices subscribe to eventsMaintains eventual consistency across multiple aggregates (in multipledatastores)Synchronize replicated [email protected]
Eventually consistent money ServiceMoneyTransferfromAccountId 101toAccountId 202amount 55state COMPLETEDINITIALDEBITEDAccountid 101balance 195250Accountid 202balance 180125Subscribes ribes age [email protected]
To maintain consistency a service mustatomically publish an eventwhenevera domain object [email protected]
How to reliably generate eventswhenever state changes?Database triggers, Hibernate eventlistener, .Reliable BUTNot with NoSQLDisconnected from the businesslevel eventLimited applicabilityAd hoc event publishing code mixedinto business logicPublishes business level eventsBUTTangled code, poor separation ofconcernsUnreliable, e.g. too easy to forget topublish an event
How to atomically update the datastoreand publish event(s)Use 2PCUse datastore as a message queueGuaranteed atomicity BUT1. Update database: new entity state & eventNeed a distributed transaction manager2. Consume event & mark event as consumedDatabase and message broker must support2PCEventually consistent mechanismSee BASE: An Acid Alternative, http://bit.ly/ebaybaseImpacts reliabilityNot fashionable2PC is best avoided BUT Tangled business logic and eventpublishing code Difficult to implement when using a NoSQLdatabase :-(
AgendaWhy build event-driven microservices?Overview of event sourcingDesigning microservices with event sourcingImplementing queries in an event sourced applicationBuilding and deploying [email protected]
Event sourcingFor each aggregate:Identify (state-changing) domain eventsDefine Event classesFor example,Account: AccountOpenedEvent, AccountDebitedEvent, AccountCreditedEventShoppingCart: ItemAddedEvent, ItemRemovedEvent, [email protected]
Persists eventsNOT current stateXAccount table101Account450Event [email protected]
Replay events to recreate [email protected]
Two actions that mustbe atomicBefore: update state publish eventsNow: persist (and publish) eventsSingle action that canbe done [email protected]
Aggregate traitsApply event returning updatedAggregateMap Command to [email protected]
Account - command processingPrevent [email protected]
Account - applying [email protected]
Request handling in an event-sourced applicationMicroservice ApastEvents TPHandlerAccountnewEvents @crichardson
Event Store publishes events - consumed by otherservicesMicroservice [email protected]
Persisting eventsIdeally use a cross platform formatUse weak serialization:enables event evolution, eg. add memo field to transfermissing fieldunknown fieldprovide default valueignoreJSON is a good [email protected]
Optimizing using snapshotsMost aggregates have relatively few eventsBUT consider a 10-year old Accountmany transactionsTherefore, use snapshots:Periodically save snapshot of aggregate stateTypically serialize a memento of the aggregateLoad latest snapshot subsequent [email protected]
Event Store APItrait EventStore {def save[T : Aggregate[T]](entity: T, events: Seq[Event],assignedId : Option[EntityId] None): Future[EntityWithIdAndVersion[T]]def update[T : Aggregate[T]](entityIdAndVersion : EntityIdAndVersion,entity: T, events: Seq[Event]): Future[EntityWithIdAndVersion[T]]def find[T : Aggregate[T] : ClassTag](entityId: EntityId) :Future[EntityWithIdAndVersion[T]]def findOptional[T : Aggregate[T] : ClassTag](entityId: }def subscribe(subscriptionId: crichardson
Business benefits of event sourcingBuilt-in, reliable audit logEnables temporal queriesPublishes events needed by big data/predictive analytics etc.Preserved historyMore easily implement future [email protected]
Technical benefits of event sourcingSolves data consistency issues in a Microservice/NoSQL-based architecture:Atomically save and publish eventsEvent subscribers update other aggregates ensuring eventual consistencyEvent subscribers update materialized views in SQL and NoSQL databases(more on that later)Eliminates O/R mapping [email protected]
Drawbacks of event sourcingWeird and unfamiliarEvents a historical record of your bad design decisionsHandling duplicate events can be trickyApplication must handle eventually consistent dataEvent store only directly supports PK-based lookup (more on that later)@crichardson
Example of an eventual consistencyproblemScenario:1. Create the user2. Create shopping cart3. Update the user with the shopping cart’s idThe user temporarily does not have a shopping cart id!Client might need to retry their request at a later pointServer should return status code [email protected]
Handling duplicate eventsIdempotent operationse.g. add item to shopping cartDuplicate detection:e.g. track most recently seen event and discard earlier [email protected]
AgendaWhy build event-driven microservices?Overview of event sourcingDesigning microservices with event sourcingImplementing queries in an event sourced applicationBuilding and deploying [email protected]
The anatomy of a microserviceXyz RequestHTTP RequestmicroserviceHTTP AdapterXyz AdapterCmdAggregateCmdEventsEvent AdapterEventsEvent [email protected]
Asynchronous Spring MVC [email protected]
MoneyTransferServiceDSL concisely specifies:1.Creates MoneyTransfer aggregate2.Processes command3.Applies events4.Persists [email protected]
MoneyTransfer [email protected]
Handling events published byAccounts1.Load MoneyTransfer aggregate2.Processes command3.Applies events4.Persists [email protected]
AgendaWhy build event-driven microservices?Overview of event sourcingDesigning microservices with event sourcingImplementing queries in an event sourced applicationBuilding and deploying [email protected]
Let’s imagine that you want to displayan account and it’s recent [email protected]
Displaying balance recent credits anddebitsWe need to do a “join: between the Account and the corresponding MoneyTransfers(Assuming Debit/Credit events don’t include other account, .)BUTEvent Store primary key lookup of individual aggregates, .Use Command Query Responsibility [email protected]
Command Query ResponsibilitySeparation ery-sideCommand-sideEventsEventsEvent [email protected]
Query-side microservicesHTTP GET RequestUpdater - microserviceViewStoreView ryView Query ServiceReader - microserviceEventsEvent [email protected]
Persisting account balance and recenttransactions in MongoDB{}Currentid: "298993498",balance: 100000,transfers : [{"transferId" : "4552840948484","fromAccountId" : 298993498,"toAccountId" : 3483948934,"amount" : 5000}, .],changes: [{"changeId" : "93843948934","transferId" : "4552840948484","transactionType" : "AccountDebited","amount" : 5000}, .]balanceTransfers that update theaccountThe sequence of debits andcreditsDenormalized efficient [email protected]
Other kinds of viewsAWS Cloud SearchAWS DynamoDBText search as-a-ServiceNoSQL as-a-ServiceView updater batches aggregatesto indexOn-demand scalable - specifydesired read/write capacityView query service does text searchDocument and key-value datamodelsUseful for denormalized, UI orientedviews
Benefits and drawbacks of CQRSBenefitsDrawbacksNecessary in an event-sourcedarchitectureComplexitySeparation of concerns simplercommand and query modelsSupports multiple denormalized viewsImproved scalability and performancePotential code duplicationReplication lag/eventually consistentviews
Dealing with eventually consistent viewsScenario:Client creates/updates aggregateClient requests view of aggregateProblem:The view might not yet have beenupdatedSolution:Create/Update response containsaggregate versionQuery request contains desired versionOut of date viewwait or return “outof date view” error codeAlternatively:“Fake it” in the UI until the view isupdated
AgendaWhy build event-driven microservices?Overview of event sourcingDesigning microservices with event sourcingImplementing queries in an event sourced applicationBuilding and deploying [email protected]
My application architectureService 1API gatewayService 2Service .NodeJSEvent ArchiverS3IndexerAWS CloudSearchEventStoreScala/Spring [email protected]
Jenkins-based deployment pipelineBuild & TestmicroserviceBuild & TestDockerimageDeploy Dockerimageto registryOne pipeline per [email protected]
Building Docker imagesdocker/build.shcp ./build/libs/service. {1}.jar build/service.jardocker build -t service- {VERSION} [email protected]
Smoke testing docker imagesPOST /containers/createPOST /containers/{id}/startDocker daemoncreatesSmoke testGET /healthDocker daemon must listen on TCP [email protected]
Publishing Docker imagesdocker/publish.shdocker tag service- {VERSION}:latest \ {REGISTRY HOST AND PORT}/service- {VERSION}docker push {REGISTRY HOST AND PORT}/service- {VERSION}@crichardson
CI environment runs on ifactorycontainerEC2 Instance/artifactory-homeEBS [email protected]
Updating production environmentLarge EC2 instance running DockerDeployment tool:1. Compares running containers with what’s been built by Jenkins2. Pulls latest images from Docker registry3. Stops old versions4. Launches new versionsOne day: use Docker clustering solution and a service discovery mechanism,Mesos and Marathon Zookeeper, Kubernetes or [email protected]
SummaryEvent sourcing solves key data consistency issues with:MicroservicesPartitioned SQL/NoSQL databasesUse CQRS to implement materialized views for queriesDocker is a great way to package [email protected]
@crichardson [email protected]? Let’s talk at the Open [email protected]
Building and deploying microservices. @crichardson The anatomy of a microservice Event Store HTTP Request HTTP Adapter Aggregate Event Adapter Cmd Cmd Events Events Xyz Adapter Xyz Request microservice. @crichardson Asynchronous Spring MVC controller.
